Member-only story
Inside Data2vec 2.0: Meta AI New Self-Supervised Model for Vision, Speech and Text
The new model presents major performance improvemetns over its predecessor.
I recently started an AI-focused educational newsletter, that already has over 150,000 subscribers. TheSequence is a no-BS (meaning no hype, no news etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers and concepts. Please give it a try by subscribing below:
Earlier last year, Meta AI unveiled Data2vec, one of the first self-supervised learning models to ever master tasks across different domains such as speech, text and vision. The model was one of the first iterations in Meta AI’s self-supervised architectures that emulate human learning processes using different sensorial inputs. A few weeks ago, Meta AI followed up with Data2vec 2.0, a new version of the models that shows 16x performance improvement.
The original Data2vec architecture based on a student and a teacher network. The…
