Best of Enemies: Some Thoughts About Adversarial Training

Training deep learning models is one of the fastest growing areas of innovation in the artificial intelligence(AI) ecosystem. Just like ew deep learning algorithms appear every month in research journals, new methodologies and techniques fro training models are an active area of research in the deep learning space. One of the disciplines that has been gaining a lot of momentum in that segment is known by the catchy name of adversarial training.

To understand adversarial training, it may be useful to rely on some analogies about how humans learn and master different skills. In many areas, knowledge training is drastically improved by creating regular and arbitrarily complex challenges. Let’s take an example from the world of chess where a grandmaster can devote months to master the theory behind a specific opening that he or she is planning to use in an upcoming tournament. Despite his robust theoretical foundation, the grandmaster is likely to complement his preparations by playing numerous games against his trainign staff in which they are forced to look for “novelties” or moves that fall outside the main line. In other words, his “adversaries” are contributing to improve his knowledge of the opening by introducing variation in the main knowledge base (opening theory).

Another more extreme example of adversarial training has transcended centuries. In ancient Rome, Emperor Nero became obsessed with assassinating his mother Agrippina which just happens to be Caligula’s sister. You know, to make things more interesting. Knowing her son’s intentions Agrippina decided to start drinking small portions of every poison available in order to build immunological defenses against a potential assassination attempt. Agrippina’s methods help to teach her immune system how to build resiliency to the poisons by recognizing the effects they provoke in smaller doses. The strategy proved to be successful and Nero’s attempt failed forcing him to rely on more traditional methods and asks some of his guards to slay her.

The metaphor of both examples is that we can constantly resolve to create adversarial situations in order to improve our knowledge of a specific domain. Adversarial training works in similar ways by introducing small variations to the input dataset creating new datasets that are likely to cause errors in the target model. In deep learning theory, we often refer to the modified input as the adversarial dataset. In practice, researchers have observed that even deep neural networks with 100% success rate with an input dataset can produce high error rates when using adversarial training.

There are many areas in which adversarial training is been widely applied today. The field of image classification is full of training models that add small vectors to the training dataset causing misclassifications in the original neural network. The modifications will be imperceptible to the human eye and the adversarial dataset will look identical to the original training set but the small linear variation will cause the classification model to regularly fail. Cybersecurity is another area that has notoriously benefited from the use of adversarial training as small variation in the code of a malware have been able to throw off the most sophisticated detection algorithms.

Mathematically, the success of adversarial training has been associated with the excessive linearity of many deep learning models. High dimensional, linear models are susceptible to produce large variations in the output when introducing small variations in the input.

CEO of IntoTheBlock, Chief Scientist at Invector Labs, I write The Sequence Newsletter, Guest lecturer at Columbia University, Angel Investor, Author, Speaker.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store