We have now watched enough analytics deployments succeed and fail at close range to be reasonably confident about what separates them, and it is not the quality of the models.
The failing pattern looks like this. An institution buys analytics. Dashboards appear. Everyone agrees they are impressive. A committee reviews them monthly. Two years later retention is unchanged and the renewal conversation is uncomfortable for both sides.
The succeeding pattern looks less impressive and works. An advisor opens a list on Monday morning. It has eleven names on it. Each name has a reason attached and a suggested first action. She works through it, records what happened, and closes the items. Nobody demos this to a board.
Why the dashboard fails
A dashboard answers “how are we doing?”. That is a question for a committee, and committees meet monthly. The useful question is “who should I contact today, and what should I say?” — and that question has to be answered every day, for a specific person, by someone with forty minutes.
The gap between those two questions is where most of the value in student success analytics is lost. It is not a modelling gap. It is an operational one.
If your analytics output cannot be acted on by one named person before lunch, it is reporting, not analytics.
Three things the list needs
- A reason, not a score. “Risk 0.71” tells an advisor nothing she can open a conversation with. “Two missed assessments in a gateway course and attendance down 30% since week three” does.
- An owner. A queue that belongs to everyone belongs to nobody. Each item needs a name attached, and unclaimed items need to escalate rather than expire.
- A closing step. What was tried and what happened. This is the part everyone skips, and it is the reason most institutions cannot answer whether their interventions work.
The uncomfortable part
Building the list is the easy half. The hard half is that somebody has to decide who owns outreach, and that is an organisational decision an analytics vendor cannot make for you. We have paused implementations at this step, because deploying a queue nobody owns produces a more expensive version of the failing pattern above.
If you are evaluating analytics vendors, the question worth asking is not about model accuracy. It is: what does my advisor see at 9am on Monday, and who is accountable for the fact that she saw it?
This argument is the practical face of the efficacy finding in EPR-2026-03: initiatives with a single named owner were more than twice as likely to show a measurable effect.
