Addressing differential performance by socioeconomic status in Artificial Intelligence (AI) models for childhood asthma - Augmented computing power, storage capability, and predictive analytics have accelerated adoption and deployment of Artificial Intelligence (AI) in health care delivery including for pediatric asthma care, the most common pediatric chronic disease with significant health disparities by socioeconomic status (SES), rurality, and demographics. Major concerns exist regarding the possibility that AI models perform systematically worse on disadvantaged populations, and the American Academy of Allergy, Asthma & Immunology, along with the Joint Commission, the National Academy of Medicine, and other non-government agencies, have expressed concerns that large-scale application of such unfair AI models may have substantial impact on health disparities. Ensuring fairness in AI is a crucial step toward improved asthma care for all children. Despite these concerns, the science of AI fairness in health care settings is still in its infancy. In our existing work, we applied current approaches of measuring and mitigating differential AI performance to an AI model for pediatric asthma care. We found the current measurement approaches to be inadequate, and the mitigation strategies to be unsuccessful. Three key limitations of the current approach for measuring and mitigating differential AI model performance in health care are: 1) statistical power and uncertainty measurement are not considered when assessing AI fairness metrics; 2) a lack of understanding of the role of EHR quality by sensitive attributes (e.g., SES) on differential AI performance; and 3) a lack of a suitable mitigation framework and strategies to reduce disparities by addressing the underlying sources of the disparities (EHR quality). Our study aims to address each of these knowledge gaps. Aim 1 is to develop a framework and tool for measuring differential AI performance using a statistical approach. We take our existing AI models for pediatric asthma exacerbation and asthma prognosis, and test for statistically significant differences by sensitive attributes. In addition to rurality and demographics, we will assess the role of SES in differential AI performance as the primary sensitive attribute by using the individual HOUsing-based SES (HOUSES) index, a validated, standardized, objective, and nationwide individual-level SES measure. We will do so via in-progress analytic and computational extensions of biostatistical techniques to estimate uncertainty and statistical power around AI fairness metrics. Aim 2 is to assess the role of EHR data quality by sensitive attributes (e.g., SES) on AI model performance. Using explanatory regression modeling (primary approach) and direct standardization (secondary approach), we will measure the strength of association between SES and AI model errors and see if this association is mediated by metrics of EHR quality. Aim 3 is to determine whether addressing EHR quality can achieve AI model fairness. This endeavor will enable AI to be used in fair and responsible ways to deliver optimal health care and achieve improved health for all children with asthma.
