Forbes covers the work of Dandelion Health, a startup sparked by the work of two C3.ai DTI researchers, Ziad Obermeyer of the University of California, Berkeley, and Sendhil Mullainathan of the University of Chicago Booth School of Business.

In 2019, the two co-authored a research paper on bias in healthcare algorithms that was published in Science. That paper’s findings would inspire them to start Dandelion, along with two other colleagues.

The paper revealed how differences in access to healthcare services among Black and white patients could ultimately result in fewer Black patients being flagged by an algorithm that used overall healthcare costs as a proxy for which patients need extra care.

That’s because if you just consider the total cost of care – that the sickest patients would be the ones with the highest bills – the data will skew towards people who can afford to go to the doctor. The result was that only around half of the Black patients who should get extra services were identified.

Access can vary wildly “depending on where you live, who you are, the color of your skin, the language you speak,” Obermeyer told Forbes. In this case, white patients were more likely to go to clinics and get treatment or surgery and had higher costs, while Black patients were more likely to use the emergency room once their untreated conditions were spiraling out of control. The end result? “The bias just piles up.”

Dandelion is creating a massive, de-identified dataset from millions of patient records so that developers can build and test the performance of their algorithms across diverse types of patients. The founding team hopes they can help establish a framework for testing and validating healthcare AI “while regulators play catchup.”

Read the full Forbes story here.

Read the 2019 Science paper here.

Forbes photo via Dandelion Health