The video is long but worth all 16 minutes. I like how he isolated such a dead simple heuristic for success in video games. It's not surprising, in retrospect, that "numbers going up" would correlate with winning, but to prove it in practice is pretty neat.
He isn't very explicit about it, but the "time travel" that is mentioned basically allows it to go back in time if things turn out badly. This is what allows it to make such unbelievable trick moves: in short timespans it is almost a brute-force search, but over the longer term it can't backtrack as much.
The video is long but worth all 16 minutes. I like how he isolated such a dead simple heuristic for success in video games. It's not surprising, in retrospect, that "numbers going up" would correlate with winning, but to prove it in practice is pretty neat.