Hospitals use the algorithm -- from Optum, UnitedHealth Group's health-services arm -- to find patients with diabetes, heart disease and other chronic ailments who could benefit from having healthcare workers monitor their overall health, manage their prescriptions and juggle doctor visits.

According to the study published in the journal Science, the algorithm gave healthier white patients the same ranking as black patients who had one more chronic illness as well as poorer laboratory results and vital signs.

The reason the algorithm used cost to rank patients and researchers found health-care spending for black patients was less than for white patients with similar medical conditions.

For the study, data-science researchers looked at the assessments made by one hospital's use of the algorithm. The study didn't name the hospital.

The researchers focused on the algorithm's rankings of 6,079 patients who identified themselves as black in the hospital's records, and 43,539 who identified as white and didn't identify themselves as any other race or ethnicity.

Then the researchers assessed the health needs of the same set of patients using their medical records, laboratory results and vital signs, and developed a different algorithm.

Using that data, the researchers found that black patients were sicker than white patients who had a similar predicted cost. Among those rated the highest priority by the hospital's algorithm, black patients had 4.8 chronic diseases compared with 3.8 of the conditions among white patients.

The researchers found the number of black patients eligible for fast-track enrolment in the programme more than doubled by prioritising patients based on their number of chronic conditions, rather than ranking them based on cost.

To be fair, it was not the AI’s fault, and it was just duplicating an existing policy which involved treating black people differently from whites.