PROJECT SUMMARY
Social vocalizations and movement-generated sounds often provide pivotal knowledge about an animal’s
identity, location, or state, yet most studies of natural behavior fail to integrate acoustic information with
simultaneous recordings of high-dimensional neural activity and behavioral dynamics. This proposal will develop
novel experimental and computational methods to attribute vocal and non-vocal sounds to individuals in a
naturalistic, acoustically complex, multi-animal environment. By integrating this rich acoustic information with
simultaneous video and wireless neural recordings, we seek to predict auditory cortical responses to auditory
cues, as a function of social context and individual identity within the family. Aim 1 will develop new tools with
which to attribute vocal and non-vocal sounds to individual animals in a multi-animal setting (i.e., the “who said
what” problem). In Aim 1A, we will collect, curate, and publicly release a range of benchmark datasets containing
simultaneous camera and microphone array recordings of multi-animal interactions with ground truth labels of
sound sources. We will use these benchmarks to validate new models for sound localization. In Aim 1B, we will
develop and release deep learning models that localize sounds with calibrated confidence intervals, using
synchronized video measurements to enhance predictions. Aim 2 will use these tools to identify archetypal,
acoustically-driven social behaviors. We will establish a new experimental paradigm that permits months-long
monitoring of rodent social behavior in a large, naturalistic environment with simultaneous camera and
microphone array recordings. Using this data, we will develop novel data analytic approaches that leverage
synchronized audio and video data streams to identify social interaction sequences. A key goal is to assess
individual differences in social behavior across families. Aim 3 is a proof-of-concept experiment in which we
determine how acoustically-driven social behaviors (established in Aim 2) predict auditory cortex responses to
both vocal or movement-generated sounds. To accomplish this, we will make continuous wireless
electrophysiological recordings from the auditory cortex of adolescent and adult gerbils within their naturalistic
family environment. We will build regression models to infer our ability to predict neural responses from
auditory/behavioral covariates (encoding models).