Enhancing the quality of CBT in community mental health through AI-generated fidelity feedback - Each year, millions of Americans receive evidence-based psychotherapies (EBPs) such as cognitive behavioral therapy (CBT) for the treatment of mental and behavioral health problems. Yet, at present, there is no scalable method for evaluating the quality of psychotherapy services. In research settings, human-based behavioral coding methods are used, but these are time consuming, costly, and rarely used in real-world clinical settings. Thus, EBP quality and effectiveness is unmeasured and unknown. The current, fast-track STTR proposal will develop and evaluate an AI-based software system (LyssnCBT) that will automatically estimate CBT fidelity from an audio recording of a CBT session. Importantly, the current work builds from Lyssn’s previous, successful work in developing an automated system for evaluating motivational interviewing (MI), and previous research showing that AI algorithms can accurately estimate CBT fidelity. Lyssn.io, Inc., (“Lyssn”) is a start-up developing AI-based technologies to support training, supervision, and quality assurance of evidence-based counseling. Our goal is to develop innovative health technology solutions that are objective, scalable, and cost efficient. Lyssn offers a HIPAA-compliant, cloud-based platform for secure recording, sharing, and reviewing of therapy sessions, which includes AI-generated metrics for MI. The proposed LyssnCBT tool will build from and be integrated into this core platform. Lyssn is partnering with Dr. Torrey Creed and the Penn Collaborative, which has a 14+ year track record of gold-standard CBT training and supervision, including more than 100 community agencies with almost 900 providers. The expertise, relationships, and amassed data -- more than 8,000 recorded sessions and more than 3,000 rated for CBT fidelity -- form the clinical foundation for the current research. Phase I will work from an existing AI-CBT prototype to develop LyssnCBT. Core activities include user-centered design focus groups and interviews with community mental health (CMH) therapists, supervisors, and administrators, which will inform the design and development of LyssnCBT. LyssnCBT will be evaluated for usability and implementation readiness in a final stage of Phase I. Phase II will conduct a field-based usability trial and a stepped-wedge, hybrid implementation-effectiveness randomized trial (N = 1,850 CMH clients) to evaluate the effectiveness of LyssnCBT to improve therapist CBT skills and client outcomes, and to reduce client drop-out. Analyses will also examine the hypothesized mechanism of action underlying LyssnCBT. The research is strongly aligned with NIMH’s 2020 Strategic Plan and its emphasis on a computational approach to scaling up treatment delivery and monitoring. Successful execution will provide automated, scalable CBT fidelity feedback for the first time ever, supporting high-quality training, supervision, and quality assurance, and providing a core technology foundation that could support a range of EBPs in the future.
