Something else entirely. Similar to an LLM, V12 of FSD is trained on on tens of millions of clips of people driving. The model takes in the history of frames over some period of time and spits out a path for the car to take, conditioned on the route from the regular route planner. As with LLMs, this approach has the benefit that it scales well according to a set scaling law: add more data, add more training and inference compute and your model will get predictably better. Tesla has the ability to sample driving behavior from almost any of its 6+ million cars world wide, based on certain triggers. The video (usually 30 seconds long) is uploaded to Tesla along with the actions that the human driver took during that time. Training a model like this is called Imitation Learning, or Behavior Cloning, and it's all the rage in robotics right now. But Tesla also has direct intervention data from its 0.5m users running FSD. Whenever a user intervenes (by disengaging the system through the steering wheel or brake, or by pressing the gas), the clip gets sent back. Tesla can then train on this clip directly to correct bad behavior, this is called Expert Intervention Learning, and it's mathematically proved to fix certain problems with raw Behavior Cloning.
