"However, we must not forget that AI needs to learn as well from vast sources."
Well, that's actually the problem. This current wave of AI is not "learning" anything really. An AI with any sort of generalizable reasoning ability would just need basic sources on programming syntax and semantics and figure the rest out on its own. Here, instead, we see the need to effectively memorize variations of the same thing, say, answers to related programming questions, so that they can be part of a intelligent sounding response.
I was dubious at the value of GenAI as a search tool at first, but now see that it's actually well suited for the role. These massive models are largely storing information in a compressed form and are great at retrieving and doing basic rewrites. The next evolution in Expert Systems I suppose, although lacking strong reasoning.
Exactly, imagine thinking you can learn how to program Java or C just from being handed the language specification, or even learn how to play chess just by being told the rules of the game.
Humans don't learn anything of substance just from being told the strict rules, we also learn from a wealth of examples expressed through a variety of means some of which is formal, some poetic, some even comedic.
Heck, we wouldn't even need Stack Overflow to begin with if we could learn things just from basic sources.
> imagine thinking you can learn how to program Java or C just from being handed the language specification
Throw in the standard library documentation, and that's exactly how many of us learned how to program before projects like stackoverflow or even the web existed for the public. We took those rules, explored the limits of them using the compiler, and learned.
Stackoverflow is, IMO, a shortcut through portions of the exploration and learning phase. Not a bad thing, but importantly it's not required either.
As the only general intelligences we know of so far, I'd say it's support for the assertion that an AI with general reasoning abilities wouldn't need SO or other examples to figure out how to do specific tasks.
Well, that's actually the problem. This current wave of AI is not "learning" anything really. An AI with any sort of generalizable reasoning ability would just need basic sources on programming syntax and semantics and figure the rest out on its own. Here, instead, we see the need to effectively memorize variations of the same thing, say, answers to related programming questions, so that they can be part of a intelligent sounding response.
I was dubious at the value of GenAI as a search tool at first, but now see that it's actually well suited for the role. These massive models are largely storing information in a compressed form and are great at retrieving and doing basic rewrites. The next evolution in Expert Systems I suppose, although lacking strong reasoning.