Understanding Active Learning

Some of the fundamental concepts behind Active Learning.

Image Source: https://www.researchgate.net/figure/The-active-learning-loop-In-active-machine-learning-data-from-experiments-informs-a_fig1_317295417

I recently started an AI-focused educational newsletter, that already has over 150,000 subscribers. TheSequence is a no-BS (meaning no hype, no news etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:

TheSequence

The traditional concepts supervised learning imply that a model will be trained using a static and previously selected dataset of labeled data. That approach has resulted challenging as many of the most successful supervised learning techniques require very large amounts of data in order to achieve any learning milestones. Many times, building the best machine learning solutions comes down to the effectiveness of annotating unlabeled datasets. But how to prioritize? Give a set of unlabeled records, it is hard to predict which ones could have the biggest impact in the training of a given machine learning models. Welcome to the world of active learning?

A classroom environment is a good analogy to understand active learning. In a typical student-teacher setting, the teacher will present certain concepts to the students and they can, in turn, ask questions to understand the material better. In other words, the students are selectively adapting the material to their learning process. Adapting our analogy to the machine learning space, imagine scenarios in which the models can select which training instances it wants to learn from. That’s the main idea behind active learning. A form of semi-supervised learning(see The Sequence Edge 14), active learning attempts to proactively selects a subset of training examples to be labeled from a set of unlabeled instances. The ultimate goal of active learning is to produce models that can learn faster while using fewer examples.

One of the core components of active learning algorithms is the mechanism for querying and selecting the data to be labeled in the training…