> It is hard to believe that our brain is just a bunch of attention layers and neural nets.
Our brain isn't, but I'd wager the architectural complexity of a physical, neuronal brain is not optimized for all useful mental tasks, and has perhaps a fair amount of local maxima that are near vestigial in overall positive impact on cognition. Just because the human brain model of cognition is the only way nature has been able to create GI doesn't mean it's the only way GI can be attained.
The best kind of machine is the simplest one needed to produce a desired outcome.
I agree that GI can have a different implementation compared to our human brain, but one thing is for sure: as of right now, the human brain can become more creative with a fraction of the data consumed by GPT-4.
GPT-4 could be AGI, but it feels like cheating to achieve AGI by feeding the entire internet. If someone can build AGI with only the data that humans consume in their lifetime, then that, imho, is the real AGI.
I guess the challenge here is that the human mind is not a blank slate, and has been optimized first by billions of years of evolution.
If it takes all the data on the internet (or more) to bootstrap AGI, but that system is then capable of leveraging its knowledge to solve new out-of-distribution tasks, that seems like a fair test to me.
I agree with the article that we see "sparks" of this generality with GPT4.
become more creative with a fraction of the data consumed by GPT-4
not if you understand the input stream of vision as an equivalent input stream of semantic tokens as in multimodal models. under that definition people looking around for 10 years receive much more training data than large language models and thus perform a bit better at zero shot inference.
or better yet, chuck it in an open plain and see how long it takes to figure out how to attach a rock to a stick and fight a gazelle to refuel it's energy supply.
Our brain isn't, but I'd wager the architectural complexity of a physical, neuronal brain is not optimized for all useful mental tasks, and has perhaps a fair amount of local maxima that are near vestigial in overall positive impact on cognition. Just because the human brain model of cognition is the only way nature has been able to create GI doesn't mean it's the only way GI can be attained.
The best kind of machine is the simplest one needed to produce a desired outcome.