Memory in Deep Learning Systems: The Cognitive Psychology Perspective
This is the second part of an essay that explores the intricacies of memory in deep learning systems. the ultimate objective of this essay is to highlight the key areas of human memory that are trying to be imitated in deep learning algorithms. In order to get there, we are exploring memory through the lenses of different schools of though such as neuroscience or cognitive psychology. In the first part of this essay, we presented the neuroscience theory of memory. Today, I would like to deep dive into the ideas that help to explain memory from the perspective of cognitive psychology.
When discussing the neuroscience theory of memory , we talked about the “binding problem” as the main theory that explains how scattered memory fragments can be recalled into cohesive memories. It turns out that, in order to explain the binding problem, we need to expand beyond our architecture of our brain and evaluate all sorts of psychological contextual elements that deeply influence how memories are recalled. One of the main theories in cognitive psychology that tries to explain the associative nature of memory is known as the Priming Effect.
Associative Memory and the Priming Effect
Like all good theories in cognitive psychology, let’s try to explain the Priming Effect in the context of experiments. Think about the first thing that comes to mind when you hear the word DINNER. Was it wine( for me it was), dessert, maybe Saturday night date? AS you can see, something as simple as a word can evoke a mixed set of emotions and even other related words. We are effectively recalling associated memories.
One of the most remarkable results of the prior experiments is to notice how fast you were able to retrieve the those related words or memories. That happens because associated memories are part of what Economics Nobel Prize winner Daniel Kanehman calls System 1; they happen quickly and they produce a series of related emotional and physical responses. In psychology, that type of phenomenon is known as Associatively Coherent.
Going back to our word game; the fact that the word DINNER evokes the idea of WINE or DESSERT is known as a priming effect in the sense that “dinner primes dessert”. Priming has an important role explaining how memory works. The priming effect does not only applies to words but also to emotions, physical reactions, instincts and other cognitive phenomenon’s. In the context of memory, the priming effect tells us that memories are not only recalled by associated ideas but by “primed ideas”.
The Availability Heuristic
Another important element of the cognitive psychology theory of memory covers how we recall the frequency of events. For instance, if I ask you “how many concerts have you attended in the last decade? “ you are likely to overestimate the number if the answer feels fluent or you have recently attended a concern. Otherwise, if you don’t enjoy your last concert experience, the number might be too low. This cognitive process is known as the Availability Heuristic and explains how our memories are deeply influenced by the rapid availability of an answer.
By now we have an idea of how we can think about memory in the context of the brain(neuroscience) and our social settings(cognitive psychology). How are those theories imitated in deep learning algorithms. That will be the subject of tomorrow’s post.