Tuesday, July 1, 2025 7/1/2025

Measuring and Understanding Diagnostic Quality from Large-Scale Data

Award Number: R01HS027990
ORGANIZATION: AGENCY FOR HEALTH CARE RESEARCH AND QUALITY
OPDIV: AHRQ
AWARD CLASS: DISCRETIONARY
AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)
PERIOD OF PERFORMANCE START DATE: 09/01/2022
PERIOD OF PERFORMANCE END DATE: 06/30/2027

Group Awards By:

View Award Description

Measuring and Understanding Diagnostic Quality from Large-Scale Data - For decades, diagnostic errors have constituted a blind spot in the effort to improve health care quality. Compared with the multitude of metrics available to assess the quality of treatment, clinicians and policymakers have few tools with which to measure and improve the quality of diagnostic decisions. Without better methods to systematically measure the quality of diagnostic decisions at the clinician level, it will continue to be difficult to identify patterns in diagnostic errors, categorize types and causes at scale, and develop and evaluate interventions to prevent them. Our long-term goal is to develop tools to measure diagnostic quality across clinical providers from large-scale data, and to build frameworks and knowledge to translate those measures into appropriate interventions. The objective of this application is to apply and validate a system for measuring diagnostic quality across radiologists in the setting of pneumonia diagnosis among 5.5 million visits with chest X-rays in Veterans Health Administration (VHA) emergency departments (EDs). In this project, we will address three challenges fundamental to any data-driven approach to measuring quality of diagnostic care. The first is a lack of observable ground truth against which to benchmark diagnoses, particularly in large-scale data. This challenge is particularly problematic when policies seek to balance type I errors (false positives) against type II errors (false negatives). Second, rates of diagnostic errors depend on the underlying prevalence of disease in the patient population, which may be incompletely observed. Third, small case numbers per clinician can complicate comparisons between clinicians, since measured differences may reflect underlying diagnostic quality or may arise from random noise. We will address these challenges with a novel combination of methods from statistical classification and applied economics, building on prior work. We propose the following specific aims: (1) We will validate data-driven measures of pneumonia diagnoses and diagnostic outcomes. In prior conceptual work building on the econometric literature of selection, we show that we may infer relative differences in diagnostic quality—as differences in type I error rates and type II error rates—even if individual type I errors are unobservable, under quasi-experimental assignment of cases to radiologists; (2) We will interpret provider-level rates of type I error and type II error in a receiver-operating curve (ROC) framework in which diagnostic errors may arise from incorrect diagnostic thresholds (trading off type I and type II errors) or poor diagnostic accuracy (incurring both too many type I errors and type II errors); and (3) To explore the determinants of clinician diagnostic quality, we will correlate our measures of radiologist diagnostic quality with their characteristics and actions across thousands of radiologists. To assess the potential consequences, we will study health outcomes of patients quasi-experimentally assigned to radiologists of differing diagnostic quality. Our project will lay the groundwork for data-driven measurement of diagnostic quality across clinical providers, a necessary first step in understanding and improving the diagnostic performance of our health care system.


Issue Date FY	Funding FY	Legal Entity Name	Legal Entity Address	Legal Entity City	Legal Entity State	Legal Entity Zip Code	Legal Entity COUNTY	Legal Entity COUNTRY	Assistance Listing	Award Code	Budget Year	Action Date	Action Type	Action Amount

Issue Date FY: 2025 ( Subtotal = $391,714 )
2025	2025	REGENTS OF THE UNIVERSITY OF CALIFORNIA, THE	1608 4TH ST STE 201	BERKELEY	CA	94710	ALAMEDA	USA	Research on Healthcare Costs, Quality and Outcomes	002	5	6/16/2025	NON-COMPETING CONTINUATION	$381,165
2025	2024	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Research on Healthcare Costs, Quality and Outcomes	000	3	3/13/2025	NON-COMPETING CONTINUATION	-$267,016
2025	2024	REGENTS OF THE UNIVERSITY OF CALIFORNIA, THE	1608 4TH ST STE 201	BERKELEY	CA	94710	ALAMEDA	USA	Research on Healthcare Costs, Quality and Outcomes	001	4	3/13/2025	CHANGE OF GRANTEE / TRAINING INSTITUTION / AWARDING INSTITUTION	$277,565
														Subtotal = $391,714

Issue Date FY: 2024 ( Subtotal = $386,063 )
2024	2024	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Research on Healthcare Costs, Quality and Outcomes	000	3	6/18/2024	NON-COMPETING CONTINUATION	$386,063
														Subtotal = $386,063

Issue Date FY: 2023 ( Subtotal = $397,870 )
2023	2023	THE LELAND STANFORD JUNIOR UNIVERSITY	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Research on Healthcare Costs, Quality and Outcomes	000	2	6/7/2023	NON-COMPETING CONTINUATION	$397,870
														Subtotal = $397,870

Issue Date FY: 2022 ( Subtotal = $399,999 )
2022	2022	LELAND STANFORD JUNIOR UNIVERSITY, THE	450 JANE STANFORD WAY	STANFORD	CA	94305	SANTA CLARA	USA	Research on Healthcare Costs, Quality and Outcomes	000	1	7/19/2022	NEW	$399,999
														Subtotal = $399,999

Grand Total All Awards = $1,575,646

Top

All Categories

About

Search

Reports

Data Submission

Award Information

Measuring and Understanding Diagnostic Quality from Large-Scale Data

Award Number: R01HS027990

ORGANIZATION: AGENCY FOR HEALTH CARE RESEARCH AND QUALITY

OPDIV: AHRQ

AWARD CLASS: DISCRETIONARY

AWARD ACTIVITY TYPE: SCIENTIFIC/HEALTH RESEARCH (INCLUDES SURVEYS)

PERIOD OF PERFORMANCE START DATE: 09/01/2022

PERIOD OF PERFORMANCE END DATE: 06/30/2027

Federal Websites

Department of Health & Human Services

HHS Operating Divisions

HHS Staff Divisions

Download A Document Viewer