CRCNS Data Sharing Proposal: One hundred thousand mental images - Program Director/Principal Investigator (Last, First, Middle): Naselaris, Thomas PROJECT SUMMARY We propose to amass a large sampling brain activity as humans internally generate mental images. The proposed Mental Imagery Database (MID) will use high-resolution 7T fMRI to achieve large-scale sampling of brain activity during mental imagery, a timely project directly motivated by the rise of generative AI. Of the many use cases we envision, we see two as especially urgent: (1) MID will be used by neuroscientists and AI researchers to gain insights into how the brain transforms text into novel visual representations. This operation is central to current AI research and distinguishes humans from other biological intelligences. Although AI image generators have become better at making pictures than most humans, they still struggle with visual interpretations of written information that humans find easy. Very little is currently known about how human brains do this. MID will provide the needed neuroimaging data at a scale that is matched to the complexity of the problem. (2) MID will advance recent work on visual decoding that has made great strides reconstructing seen images. Such work is hampered by the lack of data for cross-generalization to mental imagery. Using MID to fine-tune vision decoders for mental imagery would bring us closer to viable technology for externalizing visual thoughts. This technology could be deployed by clinicians to help patients interrogate and control intrusive mental images of traumatic events, and aid diagnosis and communication for unresponsive patients whose consciousness is not readily apparent through standard behavioral assessments. Decoding such patients' mental images could, in principle, affirm consciousness and aid in accurate diagnoses. RELEVANCE We expect this work to deliver a method for producing high-quality reconstructions of visual mental images. This work will impact the diagnosis and treatment of mental health disorders that are driven by dysregulated visual mental imagery. Little is known about the brain systems that mediate mental imagery's role in mental health, in part because of the paucity of high-quality mental imagery data, which the proposed Mental Imagery Database will provide.
