Switch course ▾
Session 2. Comparing Groups Fairly: Progress, Not Just Marks
Objective. Use a simple value-added idea with start bands to compare classes by progress, recognise regression to the mean and mixed groups, and read small differences with care.
Final marks mostly tell you who started ahead. If you want to learn from your teaching and from your colleagues, you need to ask what each class added from its own starting point. This session gives you a plain paper method, with no software and no formulas beyond averages, to build an expected result for each class and read the difference as a question. It also shows two traps that fool experienced teachers: mixed groups and the rise of pupils who were picked because they scored lowest.
Key ideas
Attainment tells where, progress tells how far
- A June mark shows where a class is now. It does not show where it started. Two classes with the same mean can have travelled very different distances.
- Judging teachers or methods by final marks alone mostly rewards the classes that began strongest.
- Both views are useful: attainment for what pupils can do now, progress for what the teaching added.
Value-added in four lines, no heavy statistics
- START: sort pupils into bands by an earlier test (Low, Mid, High). EXPECTED: for each band, the average end mark of all pupils in the school or set in that band last year.
- ACTUAL: this class's real average. DIFFERENCE: actual minus expected. A class with many Low starters has a lower expected mark, so it is judged against a fair target.
- Read the difference as a question, not a verdict. It says 'look here', and the answer lies in pupils' work and lessons.
Traps that make groups look different
- Mixed groups: a class with more strong starters seems better even if every band did the same. Compare within bands, not only in total.
- Regression to the mean: pupils picked for very low scores tend to score higher next time even with no help, and very high scorers tend to drop. A catch-up group needs a comparison group.
- Context: language of instruction, new arrivals, attendance and a bad test day all push results around. Record them beside the figures.
How big must a difference be to matter?
- Pupil-equivalents: if one pupil's mark moves by 3, a class mean of 28 pupils moves by 3 divided by 28, about 0.1. A difference of 0.2 is smaller than one pupil changing a little.
- Be most cautious with small classes and with a single year. Trust a difference more when it repeats across terms or years and when pupils' work agrees.
- Use value-added to learn: 'What does the class that gained more do in the lesson that I could try?' Do not use it to rank colleagues.
Raw mean versus value-added
Raw mean
Where the class is now
- Easy to compute
- Rewards a strong start
- Good for what pupils can do
Value-added
What the class added
- Needs start bands
- Fairer for mixed groups
- A question, not a verdict
Building bands you can trust
Pick one earlier mark that every pupil has, such as the September test or last year's final mark. Sort pupils into three bands of roughly similar size, not by feeling but by cut-off marks written down in advance. Then use last year's pupils, or the whole parallel set, to find the average end mark of each band. That table is your 'expected' for the year. It will be wrong for any individual pupil, and that is fine: it is meant for groups of 20 or more.
Use at least two years of band tables when you can; a single year is itself noisy. Write the cut-offs and the years on the page, so a colleague can repeat your work. If your school is small, combine parallel classes before computing the bands, and keep all pupils, including those who joined late, with a note.
What a difference can and cannot tell you
A positive difference means pupils did better than similar pupils did in the past, not that the teacher caused it. It may reflect a strong routine, but also a gifted group, a lenient marker, an easier test or chance. A negative difference may reflect the opposite, plus hidden factors such as new arrivals or language. That is why the figure is a question: 'What do we see in the work of this class? What does the teacher of the class with the largest gain do in the lesson?'
Always attach pupil-equivalents (difference times class size, divided by a typical gain per pupil) and the number of years. A +0.75 mean difference in a class of 28 is 21 marks in total, roughly seven pupils each three marks above expectation; interesting, but only one year.
Examples from the classroom
Same start, different mean
Scene. In Tashkent three Grade 6 mathematics classes of 28 end with means of 12.5 (6A), 11.3 (6B) and 12.9 (6C) out of 20. Using last year's bands (Low 8, Mid 12, High 16), Malika works out expected means of 12.3, 10.6 and 13.4. The differences are +0.2, +0.75 and -0.5. The order of the classes reverses, because 6B began with 14 Low pupils and 6C with 14 High.
Insight. Judging by progress from the start band can reverse a ranking by final marks.
The catch-up that seemed to work
Scene. Dilnoza selects the eight lowest September scorers for four weeks of catch-up. Their mean rises from 4.75 to 7.25. Eight similar pupils who could not attend, because of the bus timetable, rise from 4.75 to 6.5. The honest claim is about 0.75 marks of extra gain, not 2.5, and with eight pupils she plans to repeat before telling the staff room.
Insight. A comparison group separates the effect of the help from regression to the mean and ordinary practice.
A mixed class in Bukhara
Scene. Jasur's Grade 8 class in Bukhara has 11 pupils who joined in the autumn from villages with a different curriculum. The class mean in the term test is below the parallel class. When Jasur splits by band, both his Mid and High pupils score the same as the parallel class's; only the Low band is lower. He takes the finding, not the total, to the team and asks for a bridging unit for new arrivals.
Insight. Splitting a total by band or group turns an excuse-sounding average into a specific, actionable finding.
Common pitfalls
- Defining bands after seeing the results, so that they flatter a class.
- Treating a value-added figure from one small class in one year as a measure of a teacher.
- Giving extra help only to the lowest scorers and judging it by their rise alone.
- Forgetting new arrivals, language and absence when reading a negative difference.
Expected, Actual, and a Catch-up Group
- ACTIVITY A (40 min). Open the Activity Kit: Table 4 (last year's average end mark for each start band) and Table 5 (three Grade 6 classes with their pupils per band and actual average per band). All sample data. Trios.
- Trios (10 min): rank the three classes by raw mean. Do not calculate anything else yet; write down who you think taught best and why.
- Trios (20 min): calculate for each class the expected mean from Table 4, the actual mean, and the difference. Rank again and compare the two rankings.
- Whole group (10 min): for the biggest difference, how many pupil-equivalents is it? What would you check in the lessons and pupils' work before saying anything to a colleague?
- ACTIVITY B (30 min). Table 6: eight pupils with the lowest September marks joined a four-week catch-up; eight similar pupils could not attend. Pairs (15 min): calculate each group's mean in September and October and the gains. What part of the gain may be due to the catch-up?
- Pairs (10 min): from your own anonymised results, choose a group picked for low or high scores and re-tested later. Write what a fair comparison group would have been.
- Alone (5 min): write your school-based task from the Kit.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- How did the ranking change when you used progress instead of the raw mean? What does that say about who often gets praised?
- Why can a catch-up group appear to work even if it does little? What would you add to your next catch-up plan?
- How would you explain a value-added figure to a colleague in two sentences, without any jargon?
Reflect
- Where in your school are final marks used to judge a class that began far behind or far ahead?
- Which earlier test could you use to make start bands in your subject?
- What would make colleagues feel safe when a value-added figure is shared?
Try it tomorrow
- Choose one earlier test and write down three band cut-offs for your subject and grade.
- Compute expected and actual means for your class and write the result as a question.
- Next time you select pupils for extra help, find a similar group that cannot attend and keep their marks.
Further reading and links
Compare classes by what they added from the same start, treat a gap as a question, and give every selected group a comparison group.