Switch course ▾
Session 3. Using Evidence Resources Responsibly
Objective. Read a Toolkit-style summary using its three measures (impact, evidence strength, cost), look beyond the average, and run a fit check before choosing an approach for a real problem in your school.
A ready-made summary like the Toolkit can save you months of reading, and it can also mislead if you only read the headline numbers. As a senior teacher you will be asked: 'What does the research say about this?'. A good answer is rarely a single number. This lesson shows you how to read below the headline, how to cost an approach honestly in minutes as well as money, and how to keep the Toolkit in its proper place: a starting point for a conversation about your pupils, not a shopping list.
Key ideas
What the Toolkit is
- The Teaching and Learning Toolkit, from the Education Endowment Foundation (EEF) in England, is a summary of summaries: for many approaches it brings together findings from many studies in one short page.
- It is a live resource that is updated as new research appears, so a figure you read last year may have moved a little.
- Much of the research behind it comes from countries unlike Uzbekistan. That does not make it useless, it makes the fit check essential.
Three measures, three different questions
- Impact: how much extra progress, on average, compared with similar pupils who did not use the approach. The Toolkit shows it as additional months of progress.
- Evidence strength (padlocks): how far to trust the impact number, based on how many good studies there are. It is not a measure of how big the benefit is.
- Cost: money and, for you, also minutes and training time. A high-impact approach that no school can afford is not a plan.
Look beyond the average
- An average hides variation: the same label (for example 'feedback' or 'small-group teaching') covers practices that work very differently. Read the strand's text, not only the headline numbers.
- How an approach is carried out matters. The J-PAL summary of Teaching at the Right Level says results came when it was implemented systematically; training teachers or supplying materials alone did not bring them.
- Ask: for which pupils, which subjects and which ages was the finding strongest? Is that my class?
A responsible way to use any resource
- Start from your problem, not from the ranking: 'My Grade 6 pupils finish tasks but cannot explain why', not 'what has the biggest number?'.
- Then: read the page; check fit (age, subject, language, class size, resources); count the cost in minutes and money; choose one approach to pilot small, then judge with your own evidence (session 4).
- Other good sources exist: practice guides, summaries from universities and ministries, and your own colleagues' trials. The Toolkit supports a conversation; it does not end it.
The three Toolkit measures
Impact
Average extra progress
- Shown in months of progress
- An average over many studies
- Does not show who gains most
Evidence strength
Trust in the number
- Shown as padlocks
- Based on how many good studies
- Not the size of the benefit
Cost
What it takes
- Money, plus your minutes
- Training and materials
- What you stop doing to fit it
Reading below the headline
Open a strand page and read the sentences, not only the box of ratings. Look for which pupils and subjects the findings come from, whether the strand covers several different practices under one name, and what the authors say about how to put it into practice. The same label can hide very different lessons. For example, 'feedback' can mean a mark at the top of a page, or a comment that tells a pupil what to do next, and these do not do the same thing.
Then compare the strand with your problem. If the strongest findings come from older pupils and your problem is in Grade 3, the approach may still help but the evidence is less direct, and that is worth saying aloud. Note the date: the Toolkit is updated as new studies appear, so cite it as current guidance, not as eternal truth.
Counting the true cost
The cost rating covers money, but your real cost also includes minutes and attention. Count preparation time, lesson time, training, printing and marking. Then count what you will stop doing to make room, because every new routine replaces another. An approach that costs ten minutes in every lesson for a class of 38 is a serious commitment even if it needs no purchase.
How well an approach is carried out matters as much as which one you choose. The EEF guidance on implementation describes it as a staged process: define the problem, choose a practice with good evidence, plan, prepare staff, support them and review. Treat a small pilot as the first stage of that process, not as a shortcut around it.
Examples from the classroom
Reading below the headline in Fergana
Scene. Malika wants to help Grade 3 pupils in Fergana who read words correctly but do not understand sentences. She finds a strand with a good headline. Reading the text, she sees that most studies involved older pupils and a different language. She still chooses to try a version for Grade 3 but writes that the evidence is less direct, plans a short pilot and asks the Grade 4 teachers to compare notes in the next unit.
Insight. Reading the text beneath the numbers shows how directly the evidence fits your pupils.
High impact or strong evidence?
Scene. Aziz compares two cheap approaches for improving Grade 7 writing in Bukhara. One has a higher impact figure but only two padlocks. The other has a moderate impact and four padlocks. Aziz asks which he can trust more. He plans to pilot the first approach in one class for three weeks and use the second as the steady base in all classes.
Insight. A high impact with weak evidence is worth a small pilot, not a big promise.
Printing the pages in Samarkand
Scene. The school in Samarkand has unreliable internet, so Jasur prints two strand pages and a cost sheet for the methodical association. They count the real cost of one approach: 30 minutes of shared planning a week for four teachers, a set of printed cards and 10 minutes per lesson. The head of studies sees that the approach costs time, not money, and agrees to protect the planning slot.
Insight. Counting minutes honestly turns a vague 'low cost' into a decision the school can plan for.
Common pitfalls
- Choosing the approach with the biggest impact number without reading the evidence strength or cost.
- Reading padlocks as the size of the benefit.
- Treating the Toolkit as a ranking of what to do in your school.
- Quoting figures from memory or from an old slide instead of opening the current page.
Choose an Approach for a Real Problem
- Trios (5 min). Open the Activity Kit: Part A has a school problem and six PRACTICE approach cards written for this course. Their ratings are invented for practice and are NOT real Toolkit figures.
- Read each card with the three measures. Fill the fit-check table for every card: matches the problem? evidence strength? cost in minutes and money? fits our school? what must we do to implement it (20 min).
- Choose ONE approach to pilot and ONE you would not choose. Write a two-sentence reason that uses all three measures (10 min).
- Second activity (15 min). If your group has printed pages from the real Toolkit or the EEF guidance, repeat steps 2 and 3 for one real strand. Offline fallback: use the practice cards only. Compare: what did the real text add that a headline number would miss?
- Whole group (10 min): which approach did most trios choose? Which was hardest to reject, and why?
Your facilitator will guide you through this activity in the session.
Questions to discuss
- When did a high impact number and a low evidence strength point in different directions?
- What did you count as cost that the card did not show?
- What would a head of studies need to see before agreeing to a school-wide change?
Reflect
- Which approach already used in your school was chosen because it was popular rather than because of evidence?
- What costs do you pay that never appear in a budget?
- Where could your school find local evidence if the Toolkit does not cover your question?
Try it tomorrow
- Open or print one strand that matches a real problem and underline the sentences on who was studied.
- Write the true cost of one approach in minutes per week and in money.
- Ask one colleague what they would want to know before trying it.
Further reading and links
Start from the problem, read all three measures, look beyond the average, check fit and cost in your terms, and pilot small.