Computational attribution and fusion of vocalizations, social behavior, and neural recordings in a naturalistic environment - PROJECT SUMMARY Social vocalizations and movement-generated sounds often provide pivotal knowledge about an animal’s identity, location, or state, yet most studies of natural behavior fail to integrate acoustic information with simultaneous recordings of high-dimensional neural activity and behavioral dynamics. This proposal will develop novel experimental and computational methods to attribute vocal and non-vocal sounds to individuals in a naturalistic, acoustically complex, multi-animal environment. By integrating this rich acoustic information with simultaneous video and wireless neural recordings, we seek to predict auditory cortical responses to auditory cues, as a function of social context and individual identity within the family. Aim 1 will develop new tools with which to attribute vocal and non-vocal sounds to individual animals in a multi-animal setting (i.e., the “who said what” problem). In Aim 1A, we will collect, curate, and publicly release a range of benchmark datasets containing simultaneous camera and microphone array recordings of multi-animal interactions with ground truth labels of sound sources. We will use these benchmarks to validate new models for sound localization. In Aim 1B, we will develop and release deep learning models that localize sounds with calibrated confidence intervals, using synchronized video measurements to enhance predictions. Aim 2 will use these tools to identify archetypal, acoustically-driven social behaviors. We will establish a new experimental paradigm that permits months-long monitoring of rodent social behavior in a large, naturalistic environment with simultaneous camera and microphone array recordings. Using this data, we will develop novel data analytic approaches that leverage synchronized audio and video data streams to identify social interaction sequences. A key goal is to assess individual differences in social behavior across families. Aim 3 is a proof-of-concept experiment in which we determine how acoustically-driven social behaviors (established in Aim 2) predict auditory cortex responses to both vocal or movement-generated sounds. To accomplish this, we will make continuous wireless electrophysiological recordings from the auditory cortex of adolescent and adult gerbils within their naturalistic family environment. We will build regression models to infer our ability to predict neural responses from auditory/behavioral covariates (encoding models).
