Well, that's encouraging. "Pourvu que ça dure !" (as Letizia Bonaparte said).
It's funny, but older machine learning papers (most of what was published throughout the '70s, '80s and '90s) was a lot less focused on beating the leaderboard and much more on the discovery and understanding of general machine learning principles. As an example that I just happened to be reading recently, Pedro Domingos and others wrote a series of papers discussing Occam's Razor and why it is basically inappropriate in the form where it is often used in machine learning (or rather, data mining and knowledge discovery, since that was back in the '90s). It seems there was a lively discussion about that, back then.
It's funny, but older machine learning papers (most of what was published throughout the '70s, '80s and '90s) was a lot less focused on beating the leaderboard and much more on the discovery and understanding of general machine learning principles. As an example that I just happened to be reading recently, Pedro Domingos and others wrote a series of papers discussing Occam's Razor and why it is basically inappropriate in the form where it is often used in machine learning (or rather, data mining and knowledge discovery, since that was back in the '90s). It seems there was a lively discussion about that, back then.
Ah, the paper:
https://link.springer.com/article/10.1023/A:1009868929893
Not innovation, exactly- but not the SOTA rat race, either.