Switch course ▾
Session 2. The Innovation Portfolio: Pilots and Evidence Gates
Objective. Build a one-page innovation portfolio for your school with owners, teacher-time costs and evidence gates, and make go, adjust, pause or stop decisions using criteria set in advance.
A school that tries everything learns nothing, because no project gets enough time, attention or honest measurement. A school that tries nothing cannot improve either. A portfolio is the leader's tool for finding the middle: a few well-chosen pilots, each with a written test it must pass before it grows. This session shows you how to see all projects on one page, count their real cost in teacher hours and make decisions that colleagues can trust because the rules were set before the results arrived.
Key ideas
Why a school needs a portfolio
- Many schools run separate projects: an app from a demo, a club, a policy review, a project week. Each looks small, but together they use the same teachers' time. A portfolio is a short list of all innovation efforts seen together: owner, stage, evidence so far, cost in teacher hours and the date of the next decision.
- Sort every line into three kinds: SUSTAIN (improve what already works), PILOT (a small test with a baseline and a measure, as in course 11.10) and EXPLORE (an idea you only read about and discuss). Most of the effort should go to sustaining and improving; pilots are few; explore costs almost nothing.
- Set a limit that you choose yourself, for example 'no more than two or three pilots at the same time and no teacher in more than one'. A limit is a leadership decision about what the school can learn from, not an insult to enthusiasm.
Evidence gates between stages
- A gate is a planned meeting where an idea must show something before it moves on and uses more time and money. Typical path: Idea, Gate 1 (is the problem real and the idea reasonable?), small pilot of 4 to 6 weeks, Gate 2 (did the result reach the target we set, and is it safe?), wider trial in a team or a grade for one term, Gate 3 (do other teachers get similar results at an acceptable workload and cost?), then routine use with a review date.
- Match the strength of evidence to the size of the decision. Nesta's Standards of Evidence, in plain words, climb from a logical reason, to data showing change, to a comparison with similar pupils, to an explanation of why it works, to the same result in other places. Moving one exercise needs little; buying licences for all teachers for three years needs much more.
- Reversible decisions can be made on thin evidence and checked quickly; hard-to-reverse decisions (a purchase, a timetable change, a contract) need the gate to be stricter. Say which kind each decision is before the meeting.
Writing a gate: criteria first, decision second
- Write the gate before the pilot starts: the target result, the baseline, one balancing measure (what must not get worse, such as workload, access for pupils without internet, or marks in other topics), the cost in hours and money, safeguards and who decides. A gate written after the results is a justification, not a gate.
- Four possible decisions: GO (move to the next stage), ADJUST (change one thing and repeat the test), PAUSE (the idea is fine but the school has no capacity now; set a date) and STOP (the result or the safety is not there). Record the reason in one line.
- Count the cost in teacher hours: number of teachers times hours a week times weeks. Then ask what the school will stop doing to make room. If the answer is 'nothing', the portfolio is a wish list.
Evidence gates between stages
Teacher time is the scarce resource
Money is often the first thing a project plan counts, but in most schools teachers' time is scarcer. A project that needs one hour a week from 36 teachers uses 36 hours, more than a full working week of one person, every week. These hours are rarely written down, so projects pile up unseen and the same few teachers carry them all.
Add up the hours for every line in the portfolio and compare the total with a limit you set for innovation work. Also count people: how many lines does each teacher appear in? Two lines is often a sensible maximum. When the total is too high, do not ask for heroics; decide what to pause, merge or stop, and write it on the stop list.
What a small pilot can and cannot show
A pilot with 28 or 30 pupils varies from week to week for many reasons, so a change of one or two pupils is weak evidence. The target in a gate should be clearly above ordinary variation: for example from 10 to at least 17 of 30, not from 10 to 11. If you can, compare with a similar class that did not try the change, and ask whether the same teacher teaching and counting could have tilted the result.
Match the next step to the strength of evidence. A good result in one class justifies a wider trial in two or three more classes, not a purchase for the whole school. A repeated result in different classes, with a believable reason and an acceptable workload, justifies routine use. Nesta's standards describe this ladder from a logical reason up to results repeated elsewhere.
Examples from the classroom
A stop list in Bukhara
Scene. Malika lists nine projects in her Bukhara school and finds 115 teacher hours a week against a limit of 36. She keeps the peer observation triads (12 hours) and the reading log (7 hours), reduces the e-diary from 54 hours to a pilot in two classes, pauses the radio club until January and merges parent reading evenings into the reading log work. The total falls to 48 hours.
Insight. A portfolio makes the total load visible, and a stop list turns a wish list into a plan.
Wider trial with mixed results in Tashkent
Scene. Error-analysis tickets reached the target in Grade 7A (10 to 19 of 30). In the wider trial 7B moved from 8 to 15 of 29 and 7C from 12 to 14 of 31, below the target of 17. The head of studies decides: continue in 7A and 7B, adjust in 7C. A visit shows that 7C's new teacher gave the ticket at the end of a rushed lesson, so the change is only the timing.
Insight. Gate decisions can be different for different classes, and a missed target is a question about conditions, not a verdict on the idea.
An app that met the target in Fergana
Scene. Aziz's Grade 9 homework app raised homework handed in from 22 to 30 of 32, above the target of 28. The balancing measure showed 11 of 32 pupils lack steady internet and used classmates' phones. At Gate 2 the committee chooses 'adjust': a printed alternative and school-hours time in the library, and a new test of 6 weeks before any further spread.
Insight. A gate with a balancing measure protects equity that a single target would miss.
Common pitfalls
- Writing gate criteria after seeing the results, so that any result can be called success.
- Counting only money and forgetting teacher hours and the number of lines each teacher carries.
- Moving from one successful class straight to a school-wide purchase.
- Treating stop as failure, so that nobody dares to propose it.
Build the School Portfolio
- Pairs (15 min): in Table A, a sample school of 36 teachers has nine projects. Sort each into sustain, pilot or explore, add up the weekly teacher hours, and compare with the limit of 36 hours a week the school set for innovation (sample). Say what you would stop, pause or merge.
- Pairs (15 min): in Table B, three sample pilots report results. Using the targets and balancing measures set in advance, decide go, adjust, pause or stop for each and write the reason in one line.
- Pairs (10 min): in Table C, match four sample decisions to the strength of evidence they need and say whether each is reversible.
- Part B, alone (25 min): in Table D write your own school portfolio on one page: up to six lines with kind, owner role, stage, evidence so far, weekly teacher hours, next gate date and decision. Add a stop list of two things the school will stop or reduce.
- Part B, trios (15 min): each presents the portfolio in 3 minutes. The others ask: Which line has no evidence? Which teacher is in too many lines? What would make you stop line 1? Revise two lines.
- Alone (5 min): write your leadership task from the Kit.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which sample project would you have found hardest to stop, and why?
- Which balancing measure changed a decision that the main result alone would have made differently?
- What will your school stop doing so that a good pilot has room?
Reflect
- How many separate innovation projects does your school run right now, and does anyone have the full list?
- When did your school last stop a project because the evidence was weak?
- Which teacher is quietly carrying three projects?
Try it tomorrow
- List every innovation project in your school on one page and add up the weekly teacher hours.
- Write Gate 2 criteria (target, baseline, balancing measure, cost, decision maker, date) for one current pilot.
- Choose two things your school will stop or reduce next term to make room for a pilot.
Further reading and links
One page, a few pilots, gates written in advance, and a stop list: that is how a school learns from innovation instead of collecting it.