Switch course ▾
Session 1. Reading Education Research: What Was Done, What Can Be Claimed
Objective. Read an education study or report with five questions, name its design, sample and measures, and match the claim to what the evidence can support.
You are handed reports and headlines far more often than you read studies. A methodist brings a leaflet, the district sends a summary, a newspaper praises a programme. Your job is rarely to be a researcher; it is to ask a few steady questions before your school acts. This lesson gives you those questions and a way to keep your own words as modest as the evidence behind them.
Key ideas
Every study answers one question
- Course 11.5 Classroom Research taught you to run a small study; this course teaches you to judge other people's. Start with the question: what did the authors want to know, and what was compared with what?
- Read in a smart order: title and abstract, then the methods, then the results table, then the limits. Read the discussion last, because it is where authors are most hopeful.
- A headline, a leaflet or a ministry summary is a claim about a study. Ask to see the study, or at least its design, before you repeat the claim in your school.
Designs in plain words
- Random-group trial: a lottery decides who gets the approach. Comparison without a lottery: two groups chosen some other way. Before and after: one group measured twice. Survey: many people asked once. Case study or interviews: a few people, in depth.
- No design is best for everything. A trial is strong for 'did it work here?', interviews are strong for 'how did teachers experience it and why?'. Ask whether the design matches the question.
- Your own 11.5 designs (before and after, two classes, rotation) sit on this same map, so you already know their typical weaknesses.
Who took part and what was measured
- Sample: how many, how chosen, where, and who dropped out. Volunteers, one city or one strong school give results that may not describe your pupils.
- A small sample is not useless, but its result is shaky; a large sample from the wrong group is solid but about someone else. Both matter.
- Measures: a test, a survey, teacher opinion or attendance? Who measured, when, and would a second marker agree? This is the validity and reliability you met in 11.5.
The claim ladder
- Four rungs: it DESCRIBES (what happened), it is LINKED with (moves together), it PROBABLY CAUSES (fair comparison, other explanations checked), it WORKS FOR WHOM (and where, and at what cost).
- Match the verb to the rung. 'Is associated with' is not 'improves'. A before-and-after result in one class rarely climbs above 'rose during'.
- Correlation example: pupils with more books at home often read better, but money, time and parents' schooling may sit behind both. Always ask what else could explain it.
Reading a study in five steps
Read the methods before the result
Many readers jump to the conclusion because it is short and confident. The methods section is where the truth is. Look for three plain facts: who was compared with whom, how the groups came to be formed, and what exactly was measured and when. If the groups formed by choice, ask who chose and why. If the measure was an opinion, remember that opinions and learning are different things.
Check the time frame too. A four-week reading study says little about what happens after a year. And ask who dropped out. If the weakest pupils left the study, the final average looks better than the real picture.
A link can still be useful
A correlation is not a cause, but it is not useless. If absence rises in the cotton-harvest weeks, that tells a leader where to look and when to offer support, even though the harvest alone does not explain every absence.
Use links to decide where to investigate or where to target help; do not use them to promise that changing one thing will change the other. Careful wording does this for you: 'is associated with', 'tends to', 'in these schools'. Words like 'causes' and 'proves' belong to the higher rungs of the claim ladder.
Examples from the classroom
The shift chart
Scene. Jasur in Fergana sees a chart: schools with a morning shift have higher Grade 9 marks than schools with two shifts. A colleague says, 'Cancel the second shift.' Jasur asks which schools are in each group. The morning-only schools are in the town centre and have more qualified staff. He asks the district for each school's marks from last year and finds the gap was there before.
Insight. Ask what else differs between groups before you act on a gap between them.
The volunteer mentors
Scene. Malika in Tashkent reads a report that a mentoring scheme improved teachers' confidence. Only teachers who applied took part, and confidence was measured by a form they filled in themselves. She notes that keen teachers may report high confidence with or without a scheme. She decides to suggest a comparison group of non-applicants next year, before the scheme is extended.
Insight. Volunteers and self-report limit what a result can say; name both before repeating the claim.
Twenty pupils, a national claim
Scene. Aziz in Samarkand finds an article based on interviews with twenty pupils in one Grade 8 class, saying 'pupils in Uzbekistan dislike group work'. The interviews are rich and honest. He decides to quote the article for what it shows about those twenty pupils and for the questions it suggests. He also writes down the question the article raises, so that a larger survey could test it later.
Insight. Qualitative work explains experiences well; do not stretch it to a whole country.
Common pitfalls
- Reading only the abstract and the discussion, where the claim is strongest.
- Treating a link between two things as proof that one causes the other.
- Dismissing a small or imperfect study entirely instead of saying what it can and cannot show.
- Repeating a headline in a staff meeting without opening the source.
Appraise Six Study Summaries
- Open the Activity Kit: six SAMPLE study summaries (invented for practice, not real studies), the appraisal card and the claim ladder. Trios (5 min): read the five reading questions on the card together.
- Trios (20 min): appraise summaries A, B and C on the card: design, sample issue, rung of the claim ladder, and one sentence that rewrites the headline so it matches the evidence.
- Two trios (10 min): swap and appraise D, E and F quickly, then compare your rungs with the other trio. Argue about any difference until you can name which fact decides it.
- Part B, pairs (10 min): bring one real claim from your school (a leaflet, a training handout, a ministry summary or a news item). Apply the five reading questions to its source, or to what you know of it, and list what you still need to see.
- Whole group (10 min): the facilitator reads the answer key for one summary. Which sample problem was easiest to miss?
- Alone (5 min): write your leadership task for the week from the kit and agree who in your school will see the one-page result.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which headline changed the most when you rewrote it to match the evidence?
- When is a link between two things still useful to a school leader, even if it is not a cause?
- Which of the claims you hear at staff meetings would you now ask to see a study for first?
Reflect
- Which report, leaflet or training claim in your school has been repeated more often than it has been checked?
- What would a colleague need to see before you changed your mind about a favourite practice?
- How can you help colleagues question a claim without making it feel like an attack?
Try it tomorrow
- Take one claim from this week and write its question, design, sample, measure and claim rung on a card.
- Replace one strong verb in your own report ('proves', 'improves') with a verb that fits the evidence.
- Ask a colleague 'how do we know?' about one practice, kindly, and write down the answer.
Further reading and links
Ask five questions of every study: what was asked, who took part, what was measured, which claim is made, and does it fit my pupils. Then match the verb to the rung of the claim ladder.