Beyond the Mark

Exams are good at measuring performance. They are less good at improving it. A student can understand exactly why they lost marks and still make the same mistake next time.

AI can now write longer, faster and more personal feedback. That is not the breakthrough. The breakthrough is connecting feedback to what the student does next, and measuring whether it worked.

We propose feedback systems that diagnose performance, guide deliberate practice and verify that learning transfers to unfamiliar problems. And we propose building them step by step, on evidence, rather than starting with an AI tutor.

An exam measures what a candidate can show on the day. Learning is what remains, and can be applied, later. An accurate mark does not produce learning on its own.

Take a student who scores 45% on a question. The examiner notes three problems: thin explanation, little use of the scenario, and missing financial consequences. The student reads the comments and understands them. Then what?

Today, the student must work that out alone: interpret the feedback, find the underlying weakness, choose what to study and judge whether it helped. That is hard, especially early on. The result is a gap between spotting a weakness and fixing it.

Black and Wiliam showed the value of using assessment evidence to improve later learning¹. Hattie and Timperley framed good feedback as answering three questions: where am I going, how am I going, and where to next². A mark answers the second. A marking schedule hints at the first. Almost nothing answers the third.

1. What feedback says matters more than how much it says

A meta-analysis of 435 studies and more than 61,000 participants found feedback helps on average, but some kinds help far more than others³. Explanatory feedback generally beats simply marking answers right or wrong, especially for higher-order skills⁴. Compare:

“You did not adequately explain the impact on profitability.”

“You identified the rise in claims costs but not its consequence. On similar questions, connect expected claims to premium adequacy, the loss ratio or underwriting profit.”

The first names a fault. The second teaches a reusable move. Optimise for instructional value, not length.

2. Feedback only works if it is used

Carless and Boud's work on feedback literacy shifts the focus from what is said to whether the learner can act on it⁵. So a good system does not stop at advice. It gives the student an immediate chance to apply it, and shows whether the attempt succeeded.

3. Practise the skill, not the answer

Retrieving knowledge strengthens memory more than re-reading it⁶. But repeating the same question mostly builds familiarity with that answer. Test the skill in a new context instead: a student who could not explain how claims severity hits profitability might next explain how longevity affects annuity liabilities. Retrieval can support transfer, depending on how practice is designed⁷.

4. Check that learning lasts

Doing well straight after help is not the same as learning⁸, and spaced practice aids retention⁹. So check three times, because each answers a different question:

• Straight away: was the correction understood?
• On a new question: does the skill transfer?
• After a delay: has it been retained?

5. Less is often more

Working memory is limited¹⁰, and actuarial questions are already demanding. Prioritise a few important improvements over an exhaustive list, and find the right number by testing.

6. Build independence

Beginners need structure, such as a simple frame for explanations:

Over time, support should fade. Instead of naming what is missing, the system can ask: “Which part of your answer would an examiner find insufficient?” The goal is not dependence on an AI tutor. It is candidates who need less help each time.

What this looks like for one student

A student keeps losing marks for not explaining financial consequences. They know the concepts; they just stop after naming the risk. The system flags this because it recurs, introduces Point → Why → Consequence, and sets a five-minute exercise.

If the student nails the point and the mechanism but drops the consequence, the next exercise targets only that step. Then comes a different scenario without the scaffold, and another chance a few days later. Over time, the evidence shows whether the weakness is persistent, improving or resolved.

This is personalisation based on what the student actually does, not on guesses about their learning style.

AI makes this affordable at scale: analysing answers against marking criteria, spotting patterns across attempts, explaining mistakes, setting targeted practice and judging what seems to work.

But capability is not effectiveness. A model can write convincing feedback that is wrong, misaligned with the marking schedule or unhelpful. Studies of AI feedback show promise alongside real variation in quality^11,12. So AI belongs inside an evidence-based, auditable system.

Marking must stay anchored to defensible criteria and professional standards, and feedback must never quietly change a mark. Subject experts set the benchmarks, review hard cases and judge whether feedback is educationally sound. AI brings scale. People set the standard.

Capable models will soon be everywhere. The lasting advantage will be knowing which interventions actually work, and that takes evidence models cannot supply on their own.

Building everything at once is risky. Each step should earn the next.

0. Measure today's feedback

Have experts rate a sample of existing feedback for accuracy, diagnosis, actionability, transferability and clarity. Accuracy is a gate, not a score that good presentation can offset.

Done when: we can reliably tell stronger feedback from weaker, and know our recurring defects.

1. Write better feedback

Test alternatives on the same marking evidence: fuller explanations, transferable rules, or a short diagnosis with a worked example. Use the benchmark and, where practical, randomised trials. Ratings are a start; better performance on new questions is the real test.

Done when: a new approach shows reliable quality gains and credible learning benefit.

2. Turn feedback into action

Link each key recommendation to practice that targets it: a past-paper question, a short structured answer or a five-minute retry. This closes the loop for the first time: we see not just the feedback, but what the student did with it.

Done when: targeted practice beats feedback alone.

3. Personalise over time

Track skills across attempts, such as scenario application, causal explanation, breadth and judgement. Profiles reflect demonstrated performance only, so the system can tell persistent weaknesses from one-offs.

Done when: personalised recommendations beat generic ones.

4. Adapt

Use the evidence to choose between strategies, such as a worked example first or straight into practice, and to set scaffolding and timing. Keep human oversight, and remember that association is not causation. Controlled experiments remain essential.

Done when: the system sustains better independent performance than simpler approaches, without losing accuracy, fairness or transparency.

It is easy to count feedback generated, speed of delivery and satisfaction. These are useful signals, but they are not evidence of learning. The question that matters is the third: can the student now do it on a new question, unaided, after a delay?

Where we can, we will randomise students between feedback approaches, while everyone still receives their marks. We will set outcomes in advance and use unseen questions on the targeted skill as the main measure. Small studies will be treated as feasibility work, and expert-rated quality gains will not be presented as proof of better marks.

Actuarial exams test far more than recall. Candidates must read unfamiliar situations, choose methods, reason in writing and explain implications: exactly the skills a model answer alone does not teach.

Two students lose the same marks on a pricing question. One does not understand claims inflation; the other does, but cannot explain its effect on premium adequacy. Same score, different problem, different fix. Good feedback must tell them apart.

The same reasoning (stating assumptions, interpreting results, explaining consequences) is the work of an actuary. So we hold a hypothesis, not yet a result: feedback that builds transferable exam reasoning may also build professional capability.

Every interaction can record what was attempted, what the marking found, what was recommended, whether the student acted, and what happened next. Over time that becomes evidence about what works, and for whom, and at what cost in student time.

That evidence must be trustworthy: protected student data, diagnoses traceable to their evidence, reviewable automated decisions, oversight of anything that affects marks, and checks for unfair differences between groups of students.

For examining bodies, this is a chance to extend assessment without weakening it, by keeping two functions distinct:

We start modestly. Today we give text feedback on marked answers. Rather than assume a smarter tutor will help, we will test it: measure our feedback, compare alternatives, link feedback to practice, track progress, and only then adapt.

At every stage, one question decides what we keep:

Our strength will come less from any one AI model and more from the quality of evidence linking assessment, intervention and learning.