Equity gaps in early-warning systems: who gets flagged, and who gets missed

This paper is about a mistake we made. We think publishing it is more useful than the alternative, and it is the reason our fairness reporting is now automatic rather than something an institution has to ask for.

In late 2024 an institutional researcher at a partner college asked a question we should have asked ourselves: of the students who actually did not persist, what fraction had we flagged — broken down by first-generation status? We did not have the report. When we built it, the answer was bad.

What we found

First-generation students had a higher observed non-persistence rate than continuing-generation students, and a lower flag rate. The gap was 11.2 points: for every 100 first-generation students who did not persist, we flagged about 11 fewer than we did among comparable continuing-generation students. Students in evening and part-time programmes were under-flagged by 8.7 points on the same measure.

No protected attribute was in the feature set. We had been careful about that, and it turned out to be beside the point.

The cause was a feature that looked neutral

LMS engagement features — logins, resource views, forum activity, time-on-task — carried roughly 40% of the model’s total feature importance. They are also recorded much less densely for students who do more of their studying offline, work during the day, or share a device. For those students the model saw sparse engagement data and interpreted absence as normal rather than as missing.

This is a mundane, well-documented failure mode and we walked straight into it. The feature was not a proxy for first-generation status in any deliberate sense; it was a proxy for having reliable individual internet access, which correlates with it.

Excluding protected attributes from a model does nothing to prevent this. What matters is whether your most informative features are equally informative for everyone.

What we changed

• Missingness became a feature. The model now distinguishes “low engagement” from “engagement not observed” instead of treating them alike.
• Per-programme thresholds. Flag thresholds are set within programme and mode of study rather than institution-wide, so an evening programme is not competing against a day programme for a fixed number of flags.
• Reweighting during training. Under-represented subgroups are upweighted so the loss function does not optimise away their errors.
• Fairness reporting became automatic. Every refit outputs flag rate, precision, recall and calibration by first-generation status, gender, programme, campus and entry route. A gap above 5 points blocks deployment and requires a human sign-off to override.
• Attribute gaps are declared. Several institutions do not record first-generation status at all. We report that we cannot audit that dimension rather than substituting a postcode proxy, which would create its own harm.

We gave up 2.1 points of overall precision. We think that is obviously the right trade, and we say so to institutions during model review, because the alternative is a system that is most accurate for the students who need it least.

Method

Limitations

We can only audit attributes institutions record, and coverage is patchy — three of eight contexts do not capture first-generation status, so the headline figure rests on five. Our gap definition compares flag rate to observed outcome rate, which itself reflects historical inequity; a model that perfectly reproduced an unjust status quo would score well on it. We regard this as a floor for fairness work rather than a sufficient test. And caste, religion and disability status are relevant in the Indian context and largely not recorded in usable form, so we cannot report on them at all.

The fairness reporting harness is open and runs against any binary classifier with subgroup labels. If you find a failure mode we have missed we would like to hear about it: team@eduplatter.com.

Related work

Gardner, Brooks and Baker [1] introduced slicing analysis for student models and is the direct methodological ancestor of our reporting. Baker and Hawn [2] survey algorithmic bias in education and name the missingness mechanism we ran into. Buolamwini and Gebru [4] established the disaggregated-evaluation pattern we follow. Barocas, Hardt and Narayanan [3] supply the framing for why excluding a protected attribute does not prevent disparate performance. Our contribution is narrower and more specific: a deployed system, the failure we shipped, the mechanism, and the precision we paid to fix it.

Artefacts

Everything below is published or available on request. A number nobody can reproduce is an advertisement, not a result. Real institutional records are never shareable under our processor obligations, so where that applies we release a simulated corpus that reproduces the qualitative finding.

References

1. Gardner, J., Brooks, C., & Baker, R. (2019). Evaluating the Fairness of Predictive Student Models Through Slicing Analysis. LAK.
2. Baker, R. S., & Hawn, A. (2022). Algorithmic Bias in Education. International Journal of Artificial Intelligence in Education.
3. Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and Machine Learning: Limitations and Opportunities.
4. Buolamwini, J., & Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. FAT*.
5. Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. (2021). “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. ACM CHI.
6. Ministry of Electronics and Information Technology, Government of India (2023). Digital Personal Data Protection Act, 2023.