PROJECT SUMMARY
Until recently, it was thought that our memories occur as a reactivation of perception – reconjuring the same
visual information and neural patterns (dubbed the sensory reinstatement hypothesis; e.g., Buckner &
Wheeler, 2001). However, in recent work, we observed surprising differences between perception and
memory, with separate brain voxels and patterns involved in each process (Bainbridge et al., 2021b). We
observed that memory representations in the brain lose rich high-level visual information, and that the neurons
sensitive to memory are anterior to those sensitive to perception. Here, we hypothesize that memories
become more semanticized representations of perception, resulting in an anterior neural shift towards
more semantic regions (Popham et al., 2021); we dub this the semantic transformation hypothesis. This
hypothesis runs counter to a recent proposal that memory and perception are supported by two separate yet
adjacent networks (the separate networks hypothesis, Steel et al., 2021). By testing these three hypotheses
in conjunction, we will be able to identify the specific neural mechanisms that support visual memory
and its relation to visual perception. Crucially, this project has critical importance for understanding memory
disorders such as Alzheimer’s Disease, for which recent work has shown that visual and semantic information
in an image can predict its eventual memory fate in a patient (Bainbridge et al., 2019a; Grande et al., 2021).
Towards this question, we utilize innovative methods pioneered by our laboratory, including a direct read-out
of a participant’s perception and memory through drawings that are quantified through online crowd-
sourcing (Bainbridge et al., 2019a; Bainbridge, 2021). In Aim 1, we use drawings to identify the direct
influences of visual and semantic information on visual memory representations. In Aim 2, we use
representational similarity analyses (Bainbridge et al., 2021b) combined with functional magnetic resonance
imaging to test the roles of visual and semantic information on memory patterns in the brain. Importantly, in
addition to testing typical individuals, we also study the newly characterized phenomenon of congenital
aphantasia (2-5% prevalence), which describes individuals with no visual memory yet intact visual perception
and semantic memory (Zeman et al., 2015; Keogh & Pearson, 2018). Aphantasia serves as natural “knock-
out” case to observe behavioral and neural differences between perception and memory, and our
laboratory is the only laboratory in the U.S. to have studied the condition thus far (Bainbridge et al.,
2021a). Armed with this unique population, in Aim 3, we will identify the neural substrates that underlie their
dissociation of visual memory and vision, and test the role of semantic vs. visual content on memory.
In sum, this study promises to resolve our understanding of how what we see becomes what we remember,
by identifying the causes of these striking differences observed in the brain between perception and memory.