Summary
The aim of this proposal is to develop an innovative new system, including hardware assemblies and machine
learning algorithms, for continuous, high-resolution 3D quantification of behavioral and eliciting stimulus
dynamics in a natural mouse prey capture paradigm. The system will satisfy a critical unmet need for an easily
adoptable, modern behavioral measurement technology that extends well beyond current offerings, which are
difficult to set up and limited largely to measuring spontaneous animal movement in impoverished, static
environments. Our system consists of a 3D convolutional neural network processing multi-perspective video
recordings to provide detailed measurements of both predator (mouse) and prey (cricket) spatiotemporal
movement patterns within an enclosed, compact apparatus permitting precise control over the visual
environment. To reduce implementation complexity and enhance usability in other labs, the system will use only
a single commercial video camera and a set of low-cost mirrors to provide the multiple perspectives required for
3D behavior tracking. By using only a single camera, we also reduce the instrument’s physical footprint, thus
facilitating high-throughput studies across multiple setups. Furthermore, our 3D tracking algorithm will be built to
support out-of-the-box generalization to cloned setups, meaning other labs can immediately start doing science
with the instrument without laborious data labeling and training steps. As part of our system, we will also develop
new methods for analyzing the rich 3D mouse and cricket data to isolate key kinematic and action variables
along with comprehensive characterization of stimulus-behavior relationships. We will then investigate how these
new measurements can be used to better understand retinitis pigmentosa and Parkinson’s disease. Preliminary
experiments have been quite successful and illustrate the promise and power of our approach to collect large
amounts of quantitative behavior data and identify new phenotypes of motor disorders. As our vision is to make
as large of an impact as possible, our system and datasets will be shared openly with community to catalyze a
wide range of new research into brain function and treatments for neurological disease.
