My own thoughts on it are that it's entirely crazy that we focus so much on "real world" fixed benchmarks.

I should write an article on it sometime, but I think the incessant focus on data someone collected from the mystical "real world" over well designed synthetic data from a properly understood algorithm is really damaging to proper understanding.