This is not at all obvious to me. Symbolic reasoning feels quite different from picking the next word. Using physical demonstrations (or mental models of physical demonstrations) feels quite different from picking the next word.
Over the years I’ve come to believe that claims that something is “obvious” tell you more about the claimant’s state of mind than about the thing being claimed.
Symbolic reasoning isn't an atomic action. I can't speak for you or anyone else, but at least for me, symbolic reasoning involves either cached conclusions, or a fuck ton of talking to myself in my head - and that part is effectively LLM-like.
Which is why I'm still bewildered people expect LLMs to solve math and symbolic issues directly, when they're clearly (see e.g. "chain of thought") better treated as "inner voice" and used accordingly.
A lot of this kind of reasoning is very visual to me and involves no inner monologue of any kind - just visualizations flying around in my brain in complete silence. The translation into words happens later as a separate step. I wonder if this is not a universal experience.
Some people have no inner monolog, something that blew my mind.
When I work on problems I don't understand I'll monolog it internally. Now when I'm doing things I understand well I have some kind of internal shortcut language (my own token set I guess), that can sometimes make it difficult to explain to other people.
Depends on what the code is doing! Typically I “see” the underlying concepts or data structures interacting with each other. Imagine those block and arrow diagrams you would have in a textbook explaining how pointers work, except it’s completely automatic for me. My day to day involves a lot of low level systems stuff, so reading a page of code could spawn an image of how various event loops, memory allocations, etc. interact. Then to explain what I’m thinking of to a colleague is sort of like describing what I’m already “seeing” in my mind’s eye. Language to me is just an interface to this inner visual world.
This is not at all obvious to me. Symbolic reasoning feels quite different from picking the next word. Using physical demonstrations (or mental models of physical demonstrations) feels quite different from picking the next word.
Over the years I’ve come to believe that claims that something is “obvious” tell you more about the claimant’s state of mind than about the thing being claimed.