Title: Algorithms that reinforce racial biases, and algorithms that fight them
Abstract: There is increasing worry that algorithms reproduce racial and gender disparities, via the people building them or through the data used to train them. Empirical work is giving bite to these concerns, including our own work, which shows a widely used family of algorithms in health care exhibits significant racial bias. The bias arises because of label choice: the algorithm predicts health care costs rather than illness, but unequal access to care means that we spend less money caring for Black patients than for White patients. Thus, despite health care cost appearing to be an effective proxy for health by some measures of predictive accuracy, large racial biases arise. This result also suggests that identifying less biased proxies for ground truth is a fruitful enterprise, both for algorithm accuracy and also for equity. We illustrate this with examples from our ongoing work with health systems and insurers, to fix the racial bias we identified, and also in new work: By training algorithms to listen to patients, rather than predict doctors' judgment, we can dramatically reduce racial biases in the diagnosis of pain.
Bio: Ziad Obermeyer is an Associate Professor at UC Berkeley, where he does research at the intersection of machine learning, medicine, and health policy. He was named an Emerging Leader in Health and Medicine by the National Academy of Medicine, and received numerous awards including the Early Independence Award -- the National Institutes of Health’s most prestigious award for exceptional junior scientists -- and the Young Investigator Award from the Society for Academic Emergency Medicine. Previously, he was an Assistant Professor at Harvard Medical School. He continues to practice emergency medicine in underserved communities. His work has been published in Science, The New England Journal of Medicine, JAMA, The BMJ, and Health Affairs, and his research has been supported by the National Institutes of Health, Schmidt Futures, the Gordon and Betty Moore Foundation, the Robert Wood Johnson Foundation, and the Laura and John Arnold Foundation. Prior to his career in medicine, he worked as a consultant to pharmaceutical and global health clients at McKinsey & Co. in New Jersey, Geneva, and Tokyo. He is a graduate of Harvard College (magna cum laude) and Harvard Medical School (magna cum laude), and earned an M.Phil. from Cambridge.
This event is a part of our Bias^2 Series