Acquiring cognitive maps: how brains learn hidden structure - Project Summary: Animals perform goal-directed behaviors in complex environments, without the need for extensive experience, by harnessing an internal model of the world. These internal models, or cognitive maps, are central to model-based decision making. Importantly, understanding the neural circuitry of model-based decisions holds promise for improving treatments of neuropsychiatric disorders in which decision-making goes awry. An outstanding question in reinforcement learning (RL) is how cognitive maps are learned from experience, and what neural substrates support them. The orbitofrontal cortex (OFC) has been implicated in representing cognitive maps, and this project proposes to characterize the emergence of a cognitive map in rodent OFC during a value-based decision making task. This project asks three core questions: 1) How does a model-based RL agent learn a cognitive map using purely model-free RL methods? Here recent advances in meta-RL in state-of- the-art recurrent neural network (RNN) models will be trained with model-free RL, in which the emergence of a cognitive map can be fully characterized. 2) How does the OFC represent cognitive maps? Here partially trained rats will be implanted with Neuropixels probes, and neural recordings will capture emerging representations of cognitive maps in population-level OFC activity as rats learn the task structure. 3) Can poor learning of cognitive maps be bolstered with structured behavioral training? An attractive therapeutic approach for improving decision making strategies is through behavioral training alone, and here an RNN model of rodents will be used to characterize modes of poor learning and develop prescriptive training, and will then be employed in rats to uniquely addresses specific learning deficits. This proposed project employs both experimental and computational techniques as part a comprehensive career development plan toward becoming an independent investigator. Specific experimental training for electrophysi- ological recordings from behaving animals complements computational training on modeling neural activity with deep neural networks. The career development plan also includes structured opportunities for collaborating with experimentalists. It incorporates a breadth of science communication experiences through research conferences and public talks within the local training institution. Importantly, the career development plan incorporates targeted preparation for independent investigator applications, including chalk-talk opportunities, workshops to develop unique research questions, and exposure to the faculty search process. The Constantinople, Savin, and Glim- cher labs at New York University’s Center for Neural Science are a training environment uniquely positioned to deliver this interdisciplinary training: their cutting-edge research into decision-making, animal behavior, reinforce- ment learning, and systems neuroscience make it an ideal institution and set of labs to develop a health-related career studying decision-making strategies in neuroscience.
