Step 2 · Fairness
Subgroup fairness audit
A model can look accurate overall while quietly failing one group of patients. So instead of one overall score, this audit reports how the risk model performs for each demographic group separately.
The key column is the true-positive rate: of the patients who really did go on to have a high illness burden, what share the model caught. Similar rates across groups mean the model misses people at similar rates regardless of who they are.
Held-out data only
Computed on patients the model never saw during training.
Uncertainty shown
The shaded bar is the 95% confidence interval; the dark tick is the estimate. Why
Small groups not hidden
Groups under 30 patients are listed but get no number, rather than a misleadingly precise one.
| GroupA demographic group (by age, sex, race or insurance) the audit reports on separately. | PatientsNumber of patients in this group in the held-out validation set. | True-positive rate · 95% CITrue-positive rate: of the patients in this group who really went on to have a high illness burden, the share the model flagged. Higher is better; big differences between groups mean the model misses some groups more often. 95% confidence interval: the range the true value plausibly lies in. Wide ranges mean small groups, so read the point estimate with care. | Flagged for review · 95% CIShare of this group the model would flag for care-management review. |
|---|---|---|---|
| asian | 16 | Too few patients (< 30)Fewer than 30 patients: too few to say anything reliable, so no number is shown rather than a misleadingly precise one. | |
| black | 16 | Too few patients (< 30)Fewer than 30 patients: too few to say anything reliable, so no number is shown rather than a misleadingly precise one. | |
| hawaiian | 2 | Too few patients (< 30)Fewer than 30 patients: too few to say anything reliable, so no number is shown rather than a misleadingly precise one. | |
| native | 1 | Too few patients (< 30)Fewer than 30 patients: too few to say anything reliable, so no number is shown rather than a misleadingly precise one. | |
| other | 4 | Too few patients (< 30)Fewer than 30 patients: too few to say anything reliable, so no number is shown rather than a misleadingly precise one. | |
| white | 194 | 50.0%34.8% – 65.2% | 33.0%26.8% – 39.9% |
Why not make every rate exactly equal?