If you are about to start a student analytics project and you only read one thing we publish, read this. It will change your sequencing and probably save you a quarter.
We held a prediction task fixed and varied two things independently: which model family, and how complete the data was. Six models, six completeness levels, five institutions, all thirty-six cells.
Model choice moved precision by 4.1 points across its whole range. Data completeness moved it by 21.3. A plain logistic regression on complete data beat gradient boosting on thin data by seventeen points.
Not all sources are worth the same
The more useful result was the per-source breakdown, adjusted for how long each source actually takes to integrate.
- Assessment submission timing: +8.9 points, two to three weeks. The single best thing you can do. Usually available from the LMS or examination system with modest work.
- Attendance, any granularity: +5.2 points, one to two weeks. Coarse and complete beats fine and partial — weekly for 98% of students outperformed daily for 60%.
- Fee status and holds: +2.6 points, one week. Cheap, quick, and it catches a failure mode that has nothing to do with academic ability.
- Card swipe and facility use: +0.8 points, three to six weeks. Heavily marketed, genuinely hard, almost worthless.
We had card-swipe integration on our roadmap for two quarters. This analysis is why it is not there any more.
Why coverage beats resolution
This surprised us and it matters. Missingness in institutional data is never random — it correlates with mode of study, programme and device access. Partial coverage therefore introduces a bias that extra resolution does not offset. It is the same mechanism that produced our equity failure, arriving from a different direction.
What to do with this
Sequence your integration by gain per week of effort, not by what sounds sophisticated. Get assessment timing, then attendance, then fee status. Stop there and reassess. Most institutions will have captured the large majority of available signal by that point, and can decide about the rest with real numbers rather than a vendor roadmap.
Full study: EPR-2025-02. The grid harness and a simulated corpus are published.
