"The sentiment neuron within our model can classify reviews as negative or positive, even though the model is trained only to predict the next character in the text."
If you look closely at the colorized paragraph in their paper/website, you can see that the major sentiment jumps (e.g. from green to light-green and from light-orangish to red) occur with period characters. Perhaps the insight is that periods delineate the boundary of sentiment. For example:
I like this movie.
I liked this movie, but not that much.
I initially hated the movie, but ended up loving it.
The period tells the model that the thought has ended.
My question for the team: How well does the model perform if you remove periods?
My point would be to see how much the model is relying on punctuation. It could provide insight as to why character-based models outperform word-based models for sentiment analysis.
Note that sentiment tends to also jump at the ends of grammatical phrases. For example, "Seriously, the screenplay AND the directing were horrendous" [sudden drop in sentiment without punctuation] "and clearly done by people who could not fathom what was good about the novel."
This seems to have to do with a pretty deep understanding of grammar; the model waits until it the low-level neurons have something to pass up (decoding of a complete unit of meaning) before using that to update its sentiment neuron.
A lot of next-character or next-word prediction ends up working like this - internally, the model keeps some state and makes big changes to its understanding at points that have to do with the structure of the stream.
"The sentiment neuron within our model can classify reviews as negative or positive, even though the model is trained only to predict the next character in the text."
If you look closely at the colorized paragraph in their paper/website, you can see that the major sentiment jumps (e.g. from green to light-green and from light-orangish to red) occur with period characters. Perhaps the insight is that periods delineate the boundary of sentiment. For example:
I like this movie. I liked this movie, but not that much. I initially hated the movie, but ended up loving it.
The period tells the model that the thought has ended.
My question for the team: How well does the model perform if you remove periods?