I mean I'm with you on the side of "ML needs more math" because we're about as sciencey as psychology, but I'm not quite on the "aww poor babies" side (yet). It's just the nature of how sciences evolve. The truth is that with deep rigorous theory, things like physics are easier. I'd say the same for psych too. It's a crazy amount of stats you need to do to account for all the unknowns and uncertainties but I'm not sure why these sciences push the theory people away when they can't even be bothered to add an epsilon to their models (which are always linear), but I digress. The thing that concerns me more though is the push against math. We see it a lot in ML but weirdly enough reviews will reject you if you don't have equations so you get tons of papers just copy pasting math that adds nothing to the paper. It's really weird and a clear indication of the laziness of reviews and area chairs.
What I tell my students is "you don't need math to make good models but you need math to know why your models are wrong." This seems to be becoming more important than ever. Hell, how many people know about the Whitney embedding theorem? That itself helps answer half my undergrad questions and I think it's relatively unknown.
But you need both. The thinkers and tinkerers. Referring to my longer rant, industry has captured the research field. I'm really happy they're involved but we're responding in insane ways. Let academia be the thinkers and let them make small models and don't expect a $100M budget for every paper. It's the railroading that annoys me and I think it's anti science. But no one wants to even acknowledge it's existence. I'll go ahead and say there's not much use to GPT papers unless you work for OpenAI (not to be confused with LLM papers). Let people explore new architectures and don't make them compete with sota models that have billions of dollars and man hours put into them. Hype is killing our field and it's hard to see because we're a high functioning addict.
What I tell my students is "you don't need math to make good models but you need math to know why your models are wrong." This seems to be becoming more important than ever. Hell, how many people know about the Whitney embedding theorem? That itself helps answer half my undergrad questions and I think it's relatively unknown.
But you need both. The thinkers and tinkerers. Referring to my longer rant, industry has captured the research field. I'm really happy they're involved but we're responding in insane ways. Let academia be the thinkers and let them make small models and don't expect a $100M budget for every paper. It's the railroading that annoys me and I think it's anti science. But no one wants to even acknowledge it's existence. I'll go ahead and say there's not much use to GPT papers unless you work for OpenAI (not to be confused with LLM papers). Let people explore new architectures and don't make them compete with sota models that have billions of dollars and man hours put into them. Hype is killing our field and it's hard to see because we're a high functioning addict.