Yep, at that level, there is a ton of competition. The self-driving industry is almost like the video game industry and actually has a ton of employees who previously worked in video games. They are definitely much better compensated in ADAS but the work culture is similar (anecdotally so YMMV).
ML is the sexy wing of the tech industry, so it tends to attract the people who are willing to put in the hours (for interviews as well as towards work).
This is because there isn't really much demand for genuinely innovative ML in the tech world. It's a third-circle nice-to-have, not part of the core business.
It enables the business to say they invest in R&D (tax writeoffs, marketing) and it also makes it easier to hire the grunts who'll do the scut work, thinking they'll one day be working on something more interesting (which they won't be).
Competition for bullshit "data science" jobs isn't that tight. For genuine research that actually matters, it is, and that's because most of these positions exist as recruiting tools (you're getting paid to let the company say you work there) and that's always going to be a thin ledge to try to perch on.
The OP actually started his career as a ML engineer (or SWE even) and transitioned into research without a PhD. Can you elaborate on your "which they won't be" comment?
The fact that it is possible to jump from MLE to a research role in Deep Learning means that this pathway is accessible to anyone who is a decent SWE and has some math skills. But there are much fewer research positions. So there is an element of luck and being at the right place at the right time.
Today, the competition for research positions in a "sexy" STEM field (think quantum computing, black holes, DL, RL) is quite high. DL just happens to be a field where it is possible to get into research without investing too much time while also making a lot of money, so this is a dream job for many.
ML is the sexy wing of the tech industry, so it tends to attract the people who are willing to put in the hours (for interviews as well as towards work).