Augmented Reality Platform for Telehealth Rehabilitiation - Efforts to keep the most vulnerable individuals with chronic medical conditions from being exposed to COVID-
19 have triggered an unprecedented decline in the number of visits to ambulatory practices. The repercussions
have impacted not only those with the disease, but the many millions of older persons in need of healthcare who
forego in-person visits in fear of infection or for socioeconomic reasons. While the precipitating need for
alternative healthcare delivery methods has hastened the adoption of software solutions, such as Zoom,
traditional videoconferencing services fail to compensate for the lack of direct physical evaluations with a patient
that is needed for evaluating musculoskeletal (MSK) deficits, planning therapeutic interventions, and guiding
exercise compliance—essential components of evidence-based practice among rehabilitation practitioners. To
overcome these shortcomings, our team of computer vision and human movement engineers is partnering with
orthopedic rehabilitation specialists at Massachusetts General Hospital (MGH) to develop a telehealth platform
that fuses high resolution RGB and Depth (RGD-D) video data readily obtainable from a modern smartphone
to facilitate quantitative, MSK assessment. The innovation builds upon our work in computing movement
outcome measures from vision-based body tracking algorithms, and our skills in augmented reality (AR)
software development to enhance a clinician’s assessment and exercise instruction capabilities. Our pilot data
demonstrate that accurate quantitative rehabilitation outcomes are obtainable using RGB-D body tracking
algorithms during a sub-set of knee activities. Phase I will advance these capabilities by deriving and validating
the accuracy of 3D body tracking and rehabilitation outcome measures during a wider set of activities used
clinically for assessing knee mobility, alignment, posture, balance, strength, and function from depth enabled
smartphone video recordings in control subjects (Aim 1). Aim 2 will develop a proof-of-concept AR telehealth
platform with the help of the MGH team that delivers an enhanced telehealth experience through real-time
synchronized audio-visual processing, real-time display of quantitative rehabilitation outcomes for the therapist
to assess deficits or guide exercise compliance, and instructional animations for the patient to safely carry out
the rehabilitation activities. The proof-of-concept prototype will undergo feasibility testing in Aim 3 among n=5
physical therapists and n=10 patients with knee OA during a simulated telehealth session to achieve high ratings
for usability, accessibility, and effectiveness. The results will inform the user-requirements of a more complete
Phase II telehealth platform designed in close collaboration with industry partners to provide secure cloud based
communication for seamless interoperability between devices; additional examination tools (e.g. gait analysis);
a broader range of baseline assessment and therapeutic exercise protocols for additional MSK conditions; and
HIPAA-compliant deployment and electronic documentation management. The final prototype system will be
evaluated during actual telehealth visits at multiple clinical sites to promote safe and effective clinical care.
