PROJECT SUMMARY
Poor adherence to oral medications is highly prevalent in 40-50% of patients with chronic diseases worldwide.
Close monitoring of medication adherence is a critical part of successful routine patient care and drug efficacy trials but
traditional observation methods of medication monitoring have had limited reliability and scalability. Thus, there is an
urgent need for effective methods of remote patient monitoring to ensure proper medication adherence. Moreover, this need
for remote monitoring and healthcare delivery was accentuated during the COVID-19 pandemic. Our proposal seeks to
advance the field of medication adherence monitoring and support.
A promising novel technology-based intervention, video directly observed treatment (VDOT), has been shown to
improve medication adherence monitoring and support in patients with TB and HIV. VDOT enables patients to use a
smartphone app to record and submit videos of daily medication intake. The health workers are able to access and manually
review the videos remotely to confirm adherence and take follow up actions when doses are missed. Although VDOT is
highly acceptable, effective and cost-saving, its large-scale adoption is limited by the requirement for the manual task of
reviewing daily medication videos. Harnessing the potential of artificial intelligence (AI), we propose to create an innovative
machine learning (ML) model that enhances the efficiency of the VDOT by eliminating the need for manual review of the
medication intake videos by the health workers. We seek to build on our promising results from a ‘proof of concept’ model
that was based on a small sample of video images from TB medication. We will develop a robust and more effective model
that focuses on fine-grained medication ingestion behaviors by patients, not captured in previous ML models.
We have access to ~20,000 medication intake videos collected from our recently completed VDOT study of
consenting patients with TB in Uganda. This large video image dataset will facilitate the training of a robust machine
learning model for proper detection of medication adherence.
Specific Aim 1: To construct a large, high-quality dataset of video images for training AI models for recognition of fine-
grained patient actions using existing TB medication videos and open source human activities videos.
Specific Aim 2: To develop, train and validate a novel machine learning model for robust recognition of fine-grained patient
behavior.
The results of this study will provide a robust AI-based model that will be further validated in routine clinical
settings for optimum performance. We expect the new model to serve as tool that will accelerate the adoption of VDOT and
be adapted for various chronic diseases.