Switch course ▾
Session 1. Reading Results Across Classes, Years and Subjects
Objective. Read results across classes, years and subjects, decide which comparisons are fair, and allow for small numbers and natural variation before drawing any conclusion.
Your head of studies pins up a table, a colleague is praised or worried, and a decision follows. Yet the same numbers can look very different once you ask how many pupils sat the test, who was missing, where the groups started and whether the figure usually jumps around. This session gives you a short routine for deciding if a comparison across classes, years or subjects is fair, so that you protect both pupils and colleagues from conclusions the data cannot support.
Key ideas
Three ways of comparing
- Across classes: the same year, the same test, different groups. Across years: the same subject and grade, different pupils. Across subjects: the same pupils, different marking habits.
- Each comparison fails in its own way, so name which one you are making before you look at the numbers.
- Course 8.2 taught you to read one class's data. Here you widen the view and ask whether two sets of numbers may be put side by side at all.
Comparability checks
- Same test or same difficulty? Same marking key and the same marker? Same day, time and conditions (Monday morning is not Friday's last lesson)?
- How many pupils sat it, and who is missing? Absent pupils are often the ones who would have scored lowest, or the ones with the most going on at home.
- Did the groups start in the same place? Did the pupils change (new arrivals, transfers) between years?
Small numbers and natural variation
- In a group of 25, one pupil is 4 percentage points. A year-on-year jump of 8 points can be just two pupils.
- Results naturally wobble from year to year because cohorts differ and exam days differ. Look for a pattern over three or more years or terms, not one jump.
- Always print the number of pupils (n) beside every percentage. A reader should see at once how fragile it is.
Averages can hide, and subjects are not on one scale
- Split before you conclude: by gender, by language of instruction, by start band. A pattern in the whole group can reverse inside the parts (Simpson's paradox).
- Subjects mark differently. A 4 in literature and a 4 in physics do not mean the same. Compare each subject with its own history or with parallel classes, not with other subjects.
- Compare results, not colleagues. A table of teachers ranked by raw averages harms trust and teaches nothing.
Five checks before you compare
Missing pupils and the pupil-equivalent
A pass rate is a fraction, and fractions are only as good as their denominators. If 8 of 28 pupils did not sit the test, the percentage describes 20 children, not the class. Ask who is missing and why: illness is random, but 'sent home before the test' is not. A quick test of fragility is to recalculate with the absent pupils given a plausible result, for example half of them passing. If the ranking of classes changes, the original comparison was not safe.
Then express differences in pupils. In a group of 24, one pupil is about 4 percentage points; in a group of 60, about 1.7. A change of 8 points in the first group is two children. Writing 'n = 24 (1 pupil = 4 points)' beside a figure lets every reader judge it without a statistics course.
Reading several years, and splitting before concluding
Published research on school examination results shows that year-to-year swings are normal and are largely explained by chance in individual pupils' performance and by differences between cohorts, not by changes in teaching (see the Cambridge Assessment report). So draw at least three or four years on one line, with the range, before talking about a trend. A value that stays inside its usual range is not news; a value that leaves the range for two or three years in a row deserves attention.
Before concluding, split the group: by language of instruction, gender or start band. Sometimes the whole-school figure rises while every subgroup falls, because the mix of pupils changed. This is Simpson's paradox, and the cure is simple: look at the parts as well as the whole.
Examples from the classroom
The Monday test and the Friday test
Scene. In a Samarkand school three Grade 7 classes write the same fractions paper out of 20. 7A writes on Monday morning (24 of 30 pass, 80%), 7B on Friday's last lesson (21 of 28, 75%). Dilnoza, who teaches 7B, notices that her pupils were tired after a sports day. The head of studies had already put the two numbers side by side.
Insight. Same paper is not the same conditions; a 5-point gap that is under two pupils is no basis for a ranking.
Four years of algebra
Scene. A Fergana school's Grade 9 algebra pass rate is 64% (16 of 25), 77% (17 of 22), 63% (17 of 27) and 75% (18 of 24). Aziz, the head of mathematics, wants to announce that the new routine 'raised results by 12 points' because the last year is 12 points above the third. His deputy draws all four points on one sheet of graph paper.
Insight. A zigzag between 63% and 77% with about 24 pupils is the normal range; one jump is not a trend.
Subjects are not on one ruler
Scene. In Tashkent the class report for 8B shows mean marks of 3.4 in mathematics, 4.1 in Uzbek and 4.8 in physical education. A parent says, 'So my son's class is weak at maths.' Malika, the form teacher, takes last year's marks of the same class: 3.3, 4.2 and 4.8. She explains that the class is almost unchanged in every subject, and that maths is marked more strictly than PE.
Insight. Compare a subject with its own history or a parallel class, not with a different subject.
Common pitfalls
- Publishing a percentage without the number of pupils behind it.
- Reading a single-year jump or drop as a trend or as a teacher's merit or fault.
- Ranking subjects against each other by mean mark.
- Leaving out absent or moved pupils without saying so.
Fair or Not Fair? Classes, Subjects and Years
- ACTIVITY A (40 min). Open the Activity Kit: Table 1 (three Grade 7 classes on one fractions test, with notes) and Table 2 (class means in four subjects). Everything in the Kit is invented sample data. Colleagues in trios.
- Trios (15 min): for Table 1, fill the comparability checklist: same test, same conditions, who is missing, how many pupils. Then work out what the 7C pass rate would be if the absent pupils had passed at half the rate.
- Trios (10 min): for Table 2, decide which statements about the class are safe and which are not, and propose a fairer comparison for each subject.
- Whole group (5 min): share one comparison your trio would refuse to make, and one it would make with a footnote.
- ACTIVITY B (30 min). Table 3: a school's Grade 9 algebra pass rates over four years. Pairs (15 min): calculate the pass rate and what one pupil is worth each year, draw the four points on graph paper, and write the one sentence the maths department can honestly say.
- Pairs (10 min): bring your own anonymised results from two or more years. Choose one comparison you have made before and re-check it with the five checks. Write what you would now add as a footnote.
- Alone (5 min): write your school-based task from the Kit.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which of the five checks changed your reading of Table 1 the most?
- What did one pupil being worth about 4 points do to your reading of the four-year table?
- In your own school, which comparison is made most often without any of these checks?
Reflect
- Which comparison in your school has been used to praise or blame someone? Would it pass the five checks?
- What is the smallest group in your school for which you publish a percentage?
- Who needs to know the footnotes that go with a table: parents, the head, the team?
Try it tomorrow
- Pick one table your school already uses and add n and one footnote to it.
- Work out what one pupil is worth in each of your classes and write it on the class list.
- Draw your subject's pass rate for the last three or four years on one line and mark its range.
Further reading and links
Before you compare, check the test, the conditions, who is missing and how many pupils there are; expect wobble, and never read one jump as a trend.