Sunday, June 1, 2025
6/1/2025
Development of a new weighted ROC (WROC) analysis - Resubmission - 1 - ABSTRACT The broad and long-term objective of our research inquiry is to develop receiver operating characteristic (ROC) analysis towards a broadly applicable, practical, accurate, precise, efficient, and user-friendly method for evaluation of diagnostic performance in medical imaging and beyond. The objective of this project is to develop an innovative weighted ROC (WROC) analysis. The central hypothesis is that, by introducing a case weighting factor, WROC analysis can mitigate and eliminate bias in ROC analysis from non-random case samples and infer clinical performance without bias. Specific Aims are: (1) develop WROC algorithms and share analysis software with the research community; (2) develop and validate three WROC analysis applications; and (3) investigate with WROC analysis pivotal ROC study inference bias from non-random case samples. Research design, based on contemporary ROC methodologies and preliminary studies, will be to expand the basic ROC theory by introducing a weight factor to every case, and to develop WROC estimation algorithms for common ROC models including the non-parametric, conventional binormal, and proper binormal models, to develop new WROC software, which will supersede existing ROC software, and to make the new software available to the research community by developing an open, easy-to-use, feature-rich, and publication-friendly online calculator. New WROC algorithms will be used to develop three new applications: to compare meaningfully ROC studies of non-random and non-identical case samples, to design ROC studies with stratified case samples and then apply WROC analysis to model case sample distributions to match random sampling and infer random-sample ROC performance without bias, and to estimate aggregate ROC performance of multiple readers by averaging individual-reader ROC curves weighted by clinical case volume. Finally, WROC analysis will be used to investigate bias in the inference to clinical performance from multi- reader multi-case (MRMC) studies of non-random case samples and WROC analysis as a means to overcome this bias. Methods to be used include mathematical derivation of maximum-likelihood estimations, software development, and validation with Monte Calo simulations. The proposed WROC analysis is premised on weighing cases unequally. This simple addition of a case weight will add a new dimension to ROC analysis. Practical benefits include added analysis flexibility, improved clinical performance inference from laboratory studies, and new ways to design better reader studies. The importance and health relatedness of this research is that once developed, validated, and made available to and used by the research community, the new development will be one step that advances ROC analysis towards a broadly applicable, practical, accurate, precise, efficient, and user-friendly method for diagnostic performance evaluation.
HHS’ Tracking Accountability in Government Grants System (TAGGS) website provides access to detailed descriptions of grants, loans, aggregated direct payments and other types of financial assistance awarded by the HHS Operating Divisions (OPDIVs) and the Office of the Secretary Staff Divisions (STAFFDIVs).