It sounds to me like it's basically how deterministic a game is that determines whether this works. For a Tetris with a defined order that the pieces appear in, I bet this algorithm could do pretty well.
For the purposes of this AI player, all NES games are fully deterministic. Even what hardware the NES had that could be used for random number generation is under the control of the emulator and being run deterministically. There's no "random" in the usual sense here.
It seems like the problem is more that it can't look far enough into the future to see that grabbing lots of points now (by stacking blocks) is canceled out by losing (that is, not gaining) points later.
It would be interesting to see how it responded when the 'Pause' function was disabled. It seems like it would need to track 'past future' state to find a solution.
Edit: Or maybe try to filter out 'constant' sources of increasing value. Treat the stacking points as noise.
