Continues the brilliant trend of neuroscience hubris and ignorance of the human mind. "How X is generated by the brain". Answer: "lol we have no fucking clue but here are some brain regions that light up in our machine that are vaguely associated with this phenomenon. We'll still say that we explained 'how' this happens in the title of our paper though".
Reinforcement learning has a pretty useful framework for thinking about this called the exploration-exploitation dilemma.
In order for an agent to fully exploit it's environment it first has to explore that environment, and this is where curiosity becomes useful. In fact, last year there was a paper that tried to introduce curiosity to RL-based learning models [1]
I suspect some sort of cross-pollination of these two areas of research would be useful.
I view intrinsic curiosity as different from exploitation-exploration, the latter of which has its roots as far back as WWII and bandit problems.
Imho, curiosity is something fundamentally different than randomly trying new stuff in hopes of finding something with higher reward. Some of the artificial curiosity/intrinsic motivation work in RL seems to suggest this as well.
This is the first I've heard about this topic, but what's different about intrinsic curiosity to me is that its a record, theres some of yourself in there, you are testing your own experiences against the world from a beginning point of recalled experience in the moment, rather than I guess exploitation-exploration is throwing things at the wall? But I'd guess that still comes from a place of intrinsic curiosity even if we can't see it.
If I knew then I would be in a pretty powerful position :)
But at least some starting point: exploration-exploitation is basically chasing something with a better outcome, for example, if I'm working on some project, I might look at random websites for hopes of finding something useful for that project, on the other hand, curiosity would be me just looking at things that are "interesting". Right now in RL artificial curiosity research we use novelty as a heuristic for "interesting" but we haven't really solved what "interesting" is yet I think.
Right! Curiosity could very well be about optimising the structure of one's existing knowledge rather than finding out new things per se. A distinctly non-random process.
See also this recent blog post + paper by OpenAI, they designed curiosity in an agent by having it test how well it can predict the output of a randomly initialized neural network from the game's current frame + input. The better the prediction, the more its mental model of the world is correct, and the less it gets rewarded (encouraging it to go and find unexplored situations). https://blog.openai.com/reinforcement-learning-with-predicti...
