The broad long-term objective of this project is to improve the success rate of novel mood
disorder therapeutics by enhancing the reliability and generalizability of Clinician Reported
Outcomes (ClinROs) in clinical trials. This will be achieved by developing and validating the
Depression and Anxiety AI-COA™, a machine learning-based drug development tool that
analyzes audiovisual recordings of clinical interviews to infer HAM-D and HAM-A scores, and
through doing so enhances the effective reliability of primary endpoints.
The specific aims of this project will reduce uncertainties around the design and dimensioning of
a prospective confirmatory trial which will be proposed in the Qualification Plan (QP)
1) Augment the pilot dataset: Assess 96 new participants to enhance the
representativeness of the dataset, targeting 80% male and 82% non-white participants.
Re-assess 30% of the new sample after 12 weeks to confirm sensitivity to change, and
have each rating re-rated by two additional raters for enhanced reliability.
2) Assess overall model performance change: Re-train the AI-COA™ model with the
augmented pilot dataset and evaluate the impact on performance. Adjust the assumed
sample size based on the updated model's ICC to achieve the 95% CI around the
expected ICC in the trial.
3) Evaluate generalization across biological sex: a) Assess biological sex generalization
using a non-linear regression model with fixed-effects, b) experiment with normalizing
features between biological sexes, and c) develop mixed-effects models (e.g., Maximum
Likelihood Estimation, Bayesian hierarchical models) with gender nesting. If the model
generalizes well across biological sexes, loosen biological sex split and sample
requirements for the Qualification Plan.
Relevance to the Mission of the Agency: This project addresses the critical need for improved
reliability and generalizability in mood disorder clinical trials, which have historically suffered
from low FDA approval rates. By enhancing the quality of ClinROs, this research aligns with the
FDA and HHS’ missions to protect and promote public health and accelerate the development
of effective, safe, and innovative therapeutics for mood disorders. The AI-COA™ tool aims to
increase study power, reduce sample size requirements, and shorten trial durations, ultimately
contributing to more efficient and successful drug development processes.