> Atari games just from the raw pixels on the screen
It's important to distinguish between what sorts of games work well under this method and what sorts do not. Games that are variations of pole balancing, like Pong, fare better than more complex games like Asteroids, Frostbite or Montezuma's Revenge.
> Saying that AlphaGo is limited because it only knows to play one game, is like saying that humans are limited because Lee Sedol could only master at world level one game.
It's nothing of the sort. AlphaGo is a machine in the Turing sense. The neural network is a program that is the result of a search for a function specialized to playing Go. This machine, the program that the parameters across the edges in the graph represent, is logically unable to run any other program. Lee Sedol is a Universal Machine in the Turing sense, any statement contradicting this makes no mathematical sense.
> We limit software to specific domains only on account of efficiency, not because algorithms are fundamentally limited.
It is well known within the literature that these models do not make best available use of information when learning. They are exceedingly inefficient in their incorporation of new information. Issues include improper adjustment of learning rates, not using side information to constrain computation, having to experience many rewards before action distributions are adjusted in the case of reinforcement learning, samples per example in supervised learning. Note that animals are able to learn without explicit labels and clear 0/1 losses.
Humans and animals generally, even in the supervised regime, are vastly more flexible in the format the supervision can take.
For an example, look into the research on how children are able to generalize from ambiguous explanations as "that is a dog" and why difficulty in learning color from this kind of "supervision" shows just what priors are being leveraged to get that kind of learning power.
That's a great paper but it does this by minimizing prediction error drift by comparing before and post performance on the old task while learning the new. I do not know that this method will scale with increasing task numbers, considering Neural Networks are already difficult and energy-time consuming enough to train as is.
> It's important to distinguish between what sorts of games work well under this method and what sorts do not. Games that are variations of pole balancing, like Pong, fare better than more complex games like Asteroids, Frostbite or Montezuma's Revenge.
That's what I'm getting at really: AIs that are expert/genius level at something niche and fall apart when applied to a similar task a human wouldn't have trouble adapting to. Once an AI is easily adaptable to many different domains without manual tuning people will be hard pressed to deny it is intelligent.
> My cellphone can play hundreds of games well, I don't think that's an indicator of intelligence.
I don't think it is either. I'd want to see many more domains than is demonstrated in playing most games (e.g. conversation, object recognition, planning, maths)
That's not really my point. If software X can do handwriting recognition then having software Y call software X is usually fairly easy and has little to do with intelligence.
IMO, we already have intelligent AI. It's just not intelligent the way we are used to dealing with. People don't want AI, they want a human brain in a box.
Some of these functions are easy to compose to get something impressive, true. But that is not often the case. Take the case for games and imagine we wanted a meta-algorithm to select an algorithm to apply to each game. The intelligence would then shift into, how does one select the correct algorithm for the current game in the shortest time possible?
There was a recent blog post covering this and the difficulties involved:
I also posted a link (https://arxiv.org/pdf/1604.00289v2.pdf) above which is an easily readable exposition on just how current approaches fall short. It's nothing so trivial as "it's just not what we're used to".
> Take the case for games and imagine we wanted a meta-algorithm to select an algorithm to apply to each game.
I take it you haven't seen the previous accomplishment of Deep Mind before they tackled Go. They used a Reinforcement Learning algorithm to play 50 Atari games - the same algo - with great results. They really created a generic learning algorithm.
I'm fully informed about this area of research. Including other research that found simple linear methods could also get good results over a large number of games and DeepMind's recent work where far less computationally involved methods as random projections and Nearest neighbors outperformed Deep Reinforcement learners at the more complex 3D mazes and Frostbite.
But like I keep emphasizing, you can't take a neural net trained on space invaders and have it play Asteroids because each is a task specialized program that was the result of a search. While the search method is more general, the resulting program is not. You can use a single algorithm as simple as linear methods based reinforcement learning and get great results across a wide swathe of tasks but you can't claim to have found a universal learner.
> That's not really my point. If software X can do handwriting recognition then having software Y call software X is usually fairly easy and has little to do with intelligence.
Ah, I understand you now. I don't see the relevance though as I don't see why it's important how many algorithms, programs and computers is used to implement the AI. I imagine your cellphone is unable to do many things a regular human can do such as hold a basic conversion and learn to play new games which is why I wouldn't call it intelligent.
> The neural network is a program that is the result of a search for a function specialized to playing Go. This machine, the program that the parameters across the edges in the graph represent, is logically unable to run any other program. Lee Sedol is a Universal Machine in the Turing sense
I am sure he is using different neurons for playing Go than for playing poker. His Go-related neural net is only able to play go.
That doesn't make sense. The brain is highly interconnected and there's no such thing as a "Go" or poker area. Many of the same structures are recruited for different tasks. For example, the part for recognizing faces will also be recruited for reading letters or playing Go. But the important thing here is that AlphaGo is a fixed program. It's no different than chrome or firefox in that it can no longer do learning based modifications on itself. In a sense, it's actually more limited than a browser in that it cannot simulate arbitrary Turing Machines. As a feedforward network, it's not Turing Complete.
Lee Sedol meanwhile is at least as capable as a Universal Turing Machine and was learning far more per game and modifying himself while also doing the highly complex tasks of vision integrated motion planning.
It's important to distinguish between what sorts of games work well under this method and what sorts do not. Games that are variations of pole balancing, like Pong, fare better than more complex games like Asteroids, Frostbite or Montezuma's Revenge.
> Saying that AlphaGo is limited because it only knows to play one game, is like saying that humans are limited because Lee Sedol could only master at world level one game.
It's nothing of the sort. AlphaGo is a machine in the Turing sense. The neural network is a program that is the result of a search for a function specialized to playing Go. This machine, the program that the parameters across the edges in the graph represent, is logically unable to run any other program. Lee Sedol is a Universal Machine in the Turing sense, any statement contradicting this makes no mathematical sense.
> We limit software to specific domains only on account of efficiency, not because algorithms are fundamentally limited.
It is well known within the literature that these models do not make best available use of information when learning. They are exceedingly inefficient in their incorporation of new information. Issues include improper adjustment of learning rates, not using side information to constrain computation, having to experience many rewards before action distributions are adjusted in the case of reinforcement learning, samples per example in supervised learning. Note that animals are able to learn without explicit labels and clear 0/1 losses.
Humans and animals generally, even in the supervised regime, are vastly more flexible in the format the supervision can take.
For an example, look into the research on how children are able to generalize from ambiguous explanations as "that is a dog" and why difficulty in learning color from this kind of "supervision" shows just what priors are being leveraged to get that kind of learning power.
See here for an excellent overview of limitations in our current approaches to AI: https://arxiv.org/pdf/1604.00289v2.pdf
> "Learning without forgetting"
That's a great paper but it does this by minimizing prediction error drift by comparing before and post performance on the old task while learning the new. I do not know that this method will scale with increasing task numbers, considering Neural Networks are already difficult and energy-time consuming enough to train as is.