Project Summary
Physicians in the cardiac intensive care unit (CCU) make decisions in an increasingly data- and
knowledge- rich world, yet often they get little help. Currently, each physician makes decisions based on his or
her mental model of the patient’s physiology, together with mental predictions of the patient’s response to
intervention. This approach can lead to a range of behaviors that compromise patient outcomes, including
oversimplification of the physiology, errors due to cognitive overload, and physician to physician variability in
decision making. A computational tool equipped with quantitative knowledge of physiology, the ability to
systematically evaluate all the data, and informed by a database of past action-outcome events could aid the
physician with valuable suggestions for action.
We propose to train an algorithm to make decisions about dosing vasoactive medications and initiating
mechanical support in patients with cardiogenic shock due to decompensated heart failure. This focused set of
decisions entails calculations about the physiology that are normally performed in a physician’s head. We frame
the decision problem as optimizing cardiovascular function to preserve oxygen delivery, and we apply tools from
optimal control. Rather than hand-design a CCU controller we will use reinforcement learning (RL) techniques
to “fit” one. The field of RL has experienced explosive growth over the past few years, with notable advances in
strategic decision problems and robotics. A key challenge in the clinical environment is that the exploration phase
of learning (“trial and error”) would be unethical in real patients. A second challenge is that the availability of
patient data, while growing, is likely to be a bottleneck. We will leverage state-of-the-art model-based RL to train
an algorithm using a combination of simulation and off-policy learning from historical data. We will use a model
of cardiovascular physiology that underlies cardiac simulators in use today for the training of cardiologists.
Historical patient data will come from the Massachusetts General Hospital Clinical Data Animation Center which
has recorded real-time telemetry waveform data in addition to standard electronic medical record data from all
CCU patients spanning several years. This is one of the largest and most complete datasets of its kind. The
complexity of managing cardiogenic shock will continue to escalate as tools become more sophisticated and
patients live longer, with more extensive comorbidities. Advanced decision support tools could help tame this
complexity, improving the quality of care as well as democratizing it.