Just to note that the AI has complete access to the NES's RAM (and also backtracks whenever it gets into trouble), which is not quite fair to compare to other approaches to AI or playing against a human. I'd be pretty damned good at battleship if I could see your board and rewind whenever I missed.
yes, but the point is also that the program doesn't really know the goal, it just tries to increase bytes that look like counters. that's the fun thing.
you should compare it with a game whose rules you don't know, being able only to understand your score. like trying to play spider solitaire on windows without knowing how the game works (it allows you to undo your moves).
i agree that it is an interesting result, however having access to everything is weird from an AI point of view and really muddies any conclusions to be drawn; the headline seems to make it sound like this is a breakthrough in AI, when it is using a cheat that won't apply to any sort of real situation.
notice that in your example of solitaire, knowing the values of hidden cards really changes the nature of the game and makes playing the game much less strategic
Not sure why TechCrunch made it seem that way ("a breakthrough in AI"), when the author explicitly says he's submitted this to SIGBOVIK 2013 -- an April 1 conference that usually publishes fake research. (http://sigbovik.org/2013/)
This is from the paper. "I tried again, and it was a huge breakthrough: Mario jumped up to get all the coins, and then almost immediately jumped back down to continue the level! On his fi rst try he beat 1-2 and then immediately jumped in
the pit at the beginning of 1-3 just as I was starting to
feel suuuuuper smart (Figure 7). On a scale of OMFG
to WTFLOL I was like whaaaaaaat?"
The interesting part wasn't the algorithm that played the game, but another algorithm that learned what the goals and subgoals of the game were just from watching a single playthrough. The AI that played the game was just to prove it worked.
But the AI isn't making strategic decisions. It's making brain-dead ones. It analyses a play-through looking to find sections of RAM that increase as time increases. It them locks in on these sections of RAM and says "increasing the values in these sections of RAM is how I determine success." It them brute-forces a play-through with the ability to rewind time (to try a different set of inputs).
This is why he mentions that Karate Kid didn't work well, because one of the factors of a successful play-through was that your opponents health is decreasing, which is something that this AI doesn't look for.
The ability to rewind time wouldn't make for a very interesting play-through of Battleship, but depending on how game state is stored, it might not even be able to accurately deduce a good game state from a bad one.
This is why he mentions that Karate Kid didn't work well, because one of the factors of a successful play-through was that your opponents health is decreasing, which is something that this AI doesn't look for.
I don't believe that was the case. From the video he mentioned it using the power kicks up all on the first enemy because they produced the most favorable outcome for that game, at the expense of later games when more powerful moves are needed (since the AI can't plan that far ahead).
From the paper, written by the same person who made the video:
"The result is not impressive at all; the main goal here is to reduce the opponent's health, but our objective function can
only track bytes that go up."
I can't watch the video at the moment but I imagine the paper goes much farther in-depth with the internals of this AI.
Yeah, I'm wondering how it might work if instead of using a sequence of byte configurations in memory it used a sequence of pixel configurations from the screen.
