DDT-IST-000014: Progressing towards the Qualification Plan of AI-COA™ for Automated Depression and Anxiety Severity Measurement - The broad long-term objective of this project is to improve the success rate of novel mood disorder therapeutics by enhancing the reliability and generalizability of Clinician Reported Outcomes (ClinROs) in clinical trials. This will be achieved by developing and validating the Depression and Anxiety AI-COA™, a machine learning-based drug development tool that analyzes audiovisual recordings of clinical interviews to infer HAM-D and HAM-A scores, and through doing so enhances the effective reliability of primary endpoints. The specific aims of this project will reduce uncertainties around the design and dimensioning of a prospective confirmatory trial which will be proposed in the Qualification Plan (QP) 1) Augment the pilot dataset: Assess 96 new participants to enhance the representativeness of the dataset, targeting 80% male and 82% non-white participants. Re-assess 30% of the new sample after 12 weeks to confirm sensitivity to change, and have each rating re-rated by two additional raters for enhanced reliability. 2) Assess overall model performance change: Re-train the AI-COA™ model with the augmented pilot dataset and evaluate the impact on performance. Adjust the assumed sample size based on the updated model's ICC to achieve the 95% CI around the expected ICC in the trial. 3) Evaluate generalization across biological sex: a) Assess biological sex generalization using a non-linear regression model with fixed-effects, b) experiment with normalizing features between biological sexes, and c) develop mixed-effects models (e.g., Maximum Likelihood Estimation, Bayesian hierarchical models) with gender nesting. If the model generalizes well across biological sexes, loosen biological sex split and sample requirements for the Qualification Plan. Relevance to the Mission of the Agency: This project addresses the critical need for improved reliability and generalizability in mood disorder clinical trials, which have historically suffered from low FDA approval rates. By enhancing the quality of ClinROs, this research aligns with the FDA and HHS’ missions to protect and promote public health and accelerate the development of effective, safe, and innovative therapeutics for mood disorders. The AI-COA™ tool aims to increase study power, reduce sample size requirements, and shorten trial durations, ultimately contributing to more efficient and successful drug development processes.
