Automated Assessment of Infant Sleep/Wake States, Physical Activity, and Household Noise Using a Multimodal Wearable Device and Deep Learning Models - Sleep and physical activity/sedentary behavior are physiological and behavioral processes that are intricately intertwined. Their Interconnectedness is particularly pronounced during early infancy when these systems are rapidly developing in concert with neurobiological changes. Yet, sleep and physical activity/sedentary behavior are often studied in isolation and with little attention to the home environment in which they occur. Further, current state-of-the-art methods, including wearables and mobile sensing devices that automate assessment of sleep, physical activity, and sedentary behavior, have been developed and validated predominantly with adults, adolescents, and school-age children. These adult-based methods, however, do not translate to infant populations and the unique challenges posed by this development period. With these issues in mind, our overarching aim is to advance methodological tools that provide valid, automated, objective, and fine-grained assessments of infant health behaviors in real-world environments. In doing so, we will use LittleBeats, an infant multimodal wearable device engineered by our team, that integrates a microphone to collect audio data, a 3- lead electrocardiogram (ECG) to assess infant cardiac physiology, and an inertial measurement unit (IMU) sensor to assess infant motion and posture. LittleBeats can be worn by infants for extended periods of time (8- 10 hours) in their natural environments without researchers present. We will leverage high-density data from this infant wearable to address three specific aims. First, we will develop and validate multimodal deep learning (DL) algorithms that use audio, ECG, and motion data as input to detect infant sleep/wake states, including quiet sleep, active sleep, drowsy, quiet alert, active alert, and crying states. Second, we will develop and validate DL algorithms that use ECG and motion data as input to detect infant physical activity (i.e., tummy time) and sedentary time (e.g., time restrained in a car seat). Third, because environmental noise, including loudness, variability, and number of sound sources have been associated with negative physiological, behavioral, and cognitive outcomes during the first year of life, we will leverage audio data from the LittleBeats device worn by the infant to detect noises in the home environment. Our development and validation will occur across two samples of infants under six months of age. DL algorithms will be validated against (a) annotations by trained and certified human coders, (b) ecological momentary assessments provided by infants’ primary caregivers, and (c) polysomnography, the gold-standard for sleep. By bringing together assessments of infant sleep, physical activity, sedentary behavior, and household noise under a single platform, we aim to advance research capacity to investigate the interdependencies and transactions among these core infant health behaviors and the environments in which they occur. Ultimately, such tools may aid in early detection, monitoring, and intervention among infants at risk for sleep disturbances, obesity, and other poor health outcomes.
