Advancing the Science and Practice of Ecological Momentary Assessment - Pressing Issues Page 5 of 49 Abstract The use of repeated, momentary, real-world assessment methods known as the Experience Sampling Method and Ecological Momentary Assessment (EMA) have been broadly embraced over the last few decades. These methods have extended our assessment reach beyond lengthy retrospective self-reports as they can capture everyday experiences in their immediate context, including affect, behavior, symptoms, and cognitions. In this paper we evaluate nine conceptual, methodological, and psychometric issues about EMA with the goal of stimulating conversation and guiding future research on these matters: the extent to which participants are actually reporting momentary experiences; respondents’ interpretation of momentary questions; the use of comparison standards in responding; efforts to increase the EMA reporting period beyond the moment to longer periods within a day; training of EMA study participants; concerns about selection bias of respondents and selection bias of moments; the reliability of momentary data; and, for which purposes EMA might be considered a “gold standard” for assessment. Resolution of these issues should have far-reaching implications for advancing the field. 119 Pressing Issues Page 6 of 49 Introduction Behavioral science research has resulted in scores of psychometrically sound self-report instruments about thoughts, opinions, feelings, events, and behaviors that are intended to be summaries of significant time periods (retrospective reports) or to characterize a person’s usual levels and dispositions across situations or over time (trait or global reports). In light of some limitations of these traditional assessment options, researchers have long voiced desire for methods to collect higher resolution data with greater ecological validity (Brunswik, 1941). Higher resolution is desired so that associations between immediate contexts and experiences can be examined and so that dynamic processes occurring over relatively short time periods (minutes, hours, days) can be explored; greater ecological validity is desired so that observed associations and processes are representative of respondents’ everyday lives. These aspirations have yielded a collection of within-day momentary data capture methodologies. Selected examples of, and reviews about, momentary research are presented in the following papers: (Bolger, Davis, & Rafaeli, 2003; Conner & Barrett, 2012; Csikszentmihalyi & Hunter, 2003; Degroote, DeSmet, De Bourdeaudhuij, Van Dyck, & Crombez, 2020; DeVries, 1987; Ebner-Priemer & Trull, 2009; C. D. Fisher & To, 2012; Gorin & Stone, 2001; Hamaker & Wichers, 2017; Heron, Everhart, McHale, & Smyth, 2017; Kirtley, Lafit, Achterhof, Hiekkaranta, & Myin-Germeys, in press; Myin-Germeys et al., 2009; Reis & Gable, 2000; Scollon, Kim-Prieto, & Diener, 2009; S. Shiffman, A. A. Stone, & M. R. Hufford, 2008b; J. Smyth, Juth, Ma, & Sliwinski, 2017; J.M. Smyth & Stone, 2003; Stone & Broderick, 2007; Stone, Obbarius, Junghaenel, Wen, & Schneider, 2021; Stone & Shiffman, 1994; Stone, Shiffman, Atienza, & Nebling, 2007; Trull & Ebner-Priemer, 2020). Several sources provide summaries and discussion of the analytic techniques used for these complex data (Bolger & Laurenceau, 2013; Mehl & Conner, 2011; Schwartz & Stone, 2007; Shiffman, 2014). A core feature of Ecological Momentary Assessment (EMA) methods is the brief periods about which respondents report; this is meant to reduce bias and error attributable to inherent memory limitations and to limit the use of cognitive heuristics in self-reports. Relatively unobtrusive in-the-field data collection as respondents go through their everyday lives is intended to ensure ecological validity. Following a first generation of paper-and-pencil methodologies (DeVries, 1987), advances in smart phone, Internet, and computer-assisted applications have further increased the appeal of 120
