I imagine for the bots you mentioned, that they are programmed with specific logic for those games. As for the NES emulator capable of playing the second player, I imagine that would be a similar story.
The research here is not a computer program that plays games. Rather, it's a computer program that (attempts to) play any game, without having any understanding of what it means to play that game, purely by watching the buttons that a human player would press while playing that game.
4 directions and two buttons. You could point a crappy machine learning algo at it, and it would figure it out. That is far fewer variables to manipulate than what a Quake bot has to deal with.
I would bet money I could zap a pigeon every time it died in mario and it would learn to be a master in 12 hours. It won't master CoD anytime soon.
How would it figure out what it means to be "winning"? It might be possible, but would require using OCR to read the 'score' (and guessing as what is meant by the score) and would also need some way to determine if the goal is to "go to the right", etc. (also possible). Doing this would be interesting and more practical in that it could work with games where you do not have access to the memory, but seems like it would require more work in interpreting how the game's state has changed and wouldn't allow going back in time.
A Quake bot is easier to develop because it is written for a specific game.
I accept that bet for training a pigeon to play SMB :), although the scenario you described isn't comparable (it is only one game and you would be giving the pigeon extra feedback about how to play it. It still seems hard and I think it would be notable if you achieved it.)
There are Guitar Hero Bots taht "OCR" (your words) the screen to play the guitar. Not such a hard trick to read a score or see a turtle. (you know we have self driving cars right?)
I think trying to understand what is going on in a game by looking at the screen would be more challenging than trying to understand by looking at the game's memory if you want to write a bot that can play any generic game.
The examples you gave are each specific to particular domains.
This whole topic reminds me of how it is much easier to develop natural language processing for use in a particular domain compared to generically.
I write "generic" Natural language processing for a living.
Also "nintendo" is a particular domain. When you only have a "language" of 8 directions, A, and B, and only time to deal with in response to a screen that side scrolls in only one direction the programming is easy.
A chess engine has more potential decisions for a bishop than this has for a given move. And unlike the the NES, the chess engine has to adjust for changes in the behavior. Given the same input the NES would make the same choices. You can macro through most the games.
1. This was a paper submitted for a joke conference. It is in no way a giant leap in AI. It is something that some nerds find amusing. Like an anti-Rube Goldberg machine. It is trying to accomplish something quite complex using a ridiculously stupid approach.
2. Its stupid because it is looking at the memory state as just a list of numbers that change during the gameplay. It has no knowledge of the actual game. It doesn't parse the memory to figure out game state or anything. It doesn't even have any generic idea of what games involve. It figures out "oo, the number that starts at location 243 goes up in the training data. I'll mash buttons to try to make memory location 243 go up".
You're taking this way too seriously. Think of it as someone tinkering with a specific algorithm who happens to overlay it ontop of NES games to show that it's working.
It's a cool project. If this is nothing compared to the things you've done, you're welcome to post them- we'd love to see them!
That won't work, the pigeon will stop playing. But a pigeon will recognize paintings by a certain painter, or steer a missile to a target. Relative to the state of the art in machine learning pigeons are fairly sophisticated, so it means little to claim that a pigeon might learn to do something.
The CoD / Quake bots were programmed to play those games.
The emulator was presumably programmed to play those 40 games, too.
But the real measure of intelligence is to throw something at the AI that it's never seen before, and see how it performs. And that's the magic of this one: it's a general-purpose video game playing AI. You can find a new game that it wasn't specifically designed to play, and it will quite often do a decent job playing it.
I can't find a reference but I recall an Emulator for NES that used to be able to be player 2 for something like 40 NES games.