Who Marks the Markers?
Developing automated marking, from the 2017 prototype to the 2026 Auto Mark beta
Burt Verster and Matthew Smith · 2026 ASSA Convention, Cape Town
About the talk
At the 2017 ASSA Convention we presented Mark, a prototype that marked F102 scripts using text analytics. That prototype led to ThinkActuary, and nine years later to the Auto Mark beta.
This talk tells the story in between: why expert feedback is scarce, slow and expensive, why even experienced markers disagree, how we built an engine whose judgement can be inspected, and how we brought it to students with actuaries in control.
Five takeaways
- 1
Matching wording is not marking. Our 2017 prototype placed 78% of marks within one mark of the actuary on 23 pilot attempts, but similar words can hide wrong reasoning.
- 2
Senior markers disagree too, so agreement alone cannot define quality. The engine follows a clear, versioned method that anyone can inspect.
- 3
By February 2026 the engine agreed with actuaries on pass or fail for 71% of 101 scripts: enough to support markers, not to replace them.
- 4
In 2026 Session 1, actuaries reviewing in Grader accepted 91% of the engine's segment awards unchanged. The changes they made are how it keeps learning.
- 5
Auto Mark now returns feedback in minutes at a quarter of the cost in selected subjects. Next comes independent evaluation with ASSA, and turning better feedback into better learning.
The Agentic Actuary
Agents can now do actuarial work. The judgement and the responsibility stay with the actuary.