I think this is a way to really piss off statisticians:
Stop talking about minima. Stop talking about how your optimization algorithm behaves around a minimum. Nobody ever trains their model remotely close to convergence. ADAM doesn’t even provably converge. All real models diverge! You are nowhere close to a minimum! Stop talking about minima already goddamnit! Why even think about minima?! Minima are a myth! Everybody proves their results for minima of a convex function.
Actually arguing for divergent models. I mean, I love it, just because it's so very very cruel.
