The point is that approximating a distribution causally downstream of the game (text-based descriptions, in this case) produces a predictive model of the underlying game mechanics itself. That is fascinating!
Yes, the one is formally derivable from the other, but the reduction costs compute, and to a fixed epsilon of accuracy this is the situation with everything we interact with on the day to day.
The idea that you can learn underlying mechanics from observation and refutation is central to formal models of inductive reasoning like Solomonoff induction (and idealised reaoners like AIXI, if you want the AI spin). At best this is well established scientific method, at worst a pretty decent epistemology.
Talking about sampling all of the game states is irrelevant here; that wouldn't be possible even in principle for many games and in this case they certainly didn't train the LLM on every possible Othello position.
> This is astrology: to attribute to the patterns of light hitting the eye some actual physical property in the universe which corresponds to those patterns. No such exists.
Of course not - but they are highly correlated in functional human beings. What do you think our perception of the world grounds out in, if not something like the discrepancies between (our brain's) observed data and it's predictions? There's even evidence in neuroscience that this is literally what certain neuronal circuits in the cortex are doing (the hypothesis being that so-called "predictive processing" is more energy efficient than alternative architectures).
Patterns in measurements absolutely reflect properties of the objects being measured, for the simple reason that the measurements are causally linked to the object itself in controlled ways. To think otherwise is frankly insane - this is why we call them measurements, and not noise.
Yes, the one is formally derivable from the other, but the reduction costs compute, and to a fixed epsilon of accuracy this is the situation with everything we interact with on the day to day.
The idea that you can learn underlying mechanics from observation and refutation is central to formal models of inductive reasoning like Solomonoff induction (and idealised reaoners like AIXI, if you want the AI spin). At best this is well established scientific method, at worst a pretty decent epistemology.
Talking about sampling all of the game states is irrelevant here; that wouldn't be possible even in principle for many games and in this case they certainly didn't train the LLM on every possible Othello position.
> This is astrology: to attribute to the patterns of light hitting the eye some actual physical property in the universe which corresponds to those patterns. No such exists.
Of course not - but they are highly correlated in functional human beings. What do you think our perception of the world grounds out in, if not something like the discrepancies between (our brain's) observed data and it's predictions? There's even evidence in neuroscience that this is literally what certain neuronal circuits in the cortex are doing (the hypothesis being that so-called "predictive processing" is more energy efficient than alternative architectures).
Patterns in measurements absolutely reflect properties of the objects being measured, for the simple reason that the measurements are causally linked to the object itself in controlled ways. To think otherwise is frankly insane - this is why we call them measurements, and not noise.