Compared to an LLM, the corpus they use to become proficient at language (and countless other things alongside) is achingly small. It’s also embodied so it can relate sounds and utterances to the world and actions. Thus comparing outputs between to two entities is really like comparing apples to oranges, it’s not that the pronouncements of linguists and philosophers are useless in the face of it, they’re just in a different domain.
don't agree. the amount of written words we feed in, yes. but a tottler is a multimodal system including not only audio/images but actually video, plus other senses like touch or smell. This complex input 24/7 is a lot of data that happens, and some brain relations to classify the world in form before it can spell out a label (word) for it.
Even just looking at a simple word (like "apple"), the human already has a lot of contextual information about it, mostly how its presented and how the presenter frames it (like yummy, tasty fruit! like other foods in the fridge, ...), and then people say the word physically pointing at the thing.
My guess thats an order of magnitude more total input than our LLMs get solely via text or other standalone channels for training.
>Compared to an LLM, the corpus they use to become proficient at language (and countless other things alongside) is achingly small.
But that's only true if you assume humans start at zero and the millions of years of natural selection that gave us our incredible brains gives no advantage to learning, which seems facially absurd. The toddler probably sees less data in that five years than the LLM sees in a few months, but it's worth remembering how rich human sensory data is, smell is heavily tied to memory and is incredibly complex, taste and touch sensitivity in the mouth is high enough that toddlers stuff new things in their mouth to understand them better, human hands are incredibly sensor dense, and that's before we get to the classics of 1000fps vision consisting of partially high definition and partially upscaled low definition video, or audio that contains huge amounts of information not only about material properties, energy magnitudes, or space layouts and shapes, but also a huge amount of semantic data through speech and language for most children. But unlike the LLM, they didn't start with a random selection of numbers, they started with a human brain which is incredibly impressive. Frankly, it is genuinely surprising that LLMs are able to rival us as well as they do considering how much less complex they are than our brains.
>it’s not that the pronouncements of linguists and philosophers are useless in the face of it, they’re just in a different domain.
I go back and forth on whether this is true. I see the argument, but it also makes intuitive sense that study of an AI system whose intelligence comes entirely from human language can probably yield insights about human language. It doesn't seem any less plausible than studying the way slime molds feed on oat flakes to gain insights into public transport design and infrastructure, which is something that we have done to significant success.
A philosopher would likely retort something along the lines of “it doesn’t come from human language, it comes from statistically modelling a giant corpus of readymade text, which isn’t the same thing as language but an encoding of it”.
