CRCNS: Developing and Testing Language Models with Cognitively Plausible Memory - Language makes it possible to communicate complex thoughts by composing together simpler elements. How does the brain accomplish this process? AI language models based on transformers, such as OpenAI’s GPT-4, appear to provide a potential answer to this question: like the brain, their behavior emerges from a complex interaction of a massive number of simple units, and they can learn from experience to process language surprisingly well. Yet they also differ from the brain in crucial ways; in particular, their working memory capacity is vastly greater than that of humans. Because they operate under very different constraints than humans, it is unlikely that their representations will match the ones used by humans, and as such they do not immediately constitute good candidates for models of the brain. This project will address this gap, following two main aims. (1) We will develop language models with working memory constraints that better approximate those of humans, and compare their behavior to that of humans in memory retrieval and language processing experiments. We will also test the hypothesis that memory constraints facilitate the emergence of structure by comparing the models’ language acquisition trajectory and efficiency to children’s. (2) We will conduct experiments with human participants, using fMRI and intracranial recordings, as they listen to or read sentences that require maintaining dependencies between words across multiple other words, taxing working memory. The sentences will either be embedded in a story, or presented in isolation (enabling greater experimental control). We will determine to what extent the models we developed in Aim 1 are able to explain these new neural data, as well as existing data from EEG, MEG and fMRI experiments. Overall, this project will develop neuroscientifically plausible AI language models constrained by data recorded from the human brain. Such models can then be used to deepen our understanding of language in the brain, which could advance treatments for language impairments. As such, the project is closely aligned with NIBIB’s mission to develop biomedical technologies that integrate engineering with the physical and life sciences to solve complex problems and improve health. RELEVANCE (See instructions): AI technology has a tremendous potential to help understand how the brain processes language and to address language processing deficits, but to realize this potential, we need AI systems that are similar to humans. We plan to create AIs that, unlike most standard AIs, have working memory that mimics that of humans. The development of this technology will be tightly integrated with existing data from human neuroscience as well as data from neuroimaging and intracranial recording collected for this project.
