Great tutorial. I see active learning as a halfway house between supervised learning and reinforcement learning, because requesting labels is an action (as in RL), but of a very limited, predefined type.
A lot of problems which we initially model as supervised learning are in reality, in a live situation, more like active learning. Going the whole hog to RL may be unnecessary. But still, bandit-type models may be the right fully general setting.
These topics are very much in the air at the moment with a lot of new interest in bandit models and algorithms.
They are difficult to compare because they are intended to solve different problems. Typical learning algorithms have access to the entire, pre-labeled training set, which they can repeatedly iterate over.
Online learners are still trained using a completely-labeled data set, but they cannot access the entire data set at once. Instead, examples arrive one at a time and cannot be saved or replayed.
In active learning, the labeled data can be processed in batches, but the learner either has access to additional unlabeled data or can generate new examples itself. During training, it can request that some of these examples be labeled. Imagine asking a teacher for help.
Online learning makes sense when you have so much data that you cannot possibly store it all; active learning makes sense when you have less labeled data and the labeling step is expensive.
A lot of problems which we initially model as supervised learning are in reality, in a live situation, more like active learning. Going the whole hog to RL may be unnecessary. But still, bandit-type models may be the right fully general setting.
These topics are very much in the air at the moment with a lot of new interest in bandit models and algorithms.