Uninformed speculative question here: could the algorithm be improved with some sort of momentum term in the input selection?
It seems to succeed with move-to-the-right (position counter) but fall into what looks like Brownian motion on Pac-Man. It seems to me that putting momentum into its strategy (and changing when the objective function declined) might reduce the implicit branching tree.
What it simulates, then, is the tendency for people to fall into "a groove" and depart from it when their objective function ceases increasing.
I'd really like to see a fuller analysis of what Pac-Man was doing there. As I understand the algorithm, it should have seen into the future where the dot gets eaten, and not turned back.
But maybe the program failed to discover the bytes which hold the score, and so it didn't think eating dots was a particular notable activity.
It seems to succeed with move-to-the-right (position counter) but fall into what looks like Brownian motion on Pac-Man. It seems to me that putting momentum into its strategy (and changing when the objective function declined) might reduce the implicit branching tree.
What it simulates, then, is the tendency for people to fall into "a groove" and depart from it when their objective function ceases increasing.