Switch course ▾
Session 3. Observation Cycles at Scale: Time, Sampling and Calibration
Objective. Build a realistic term plan for observation cycles with an honest time budget, a mix of visit types and a calendar, and calibrate observers so that notes record the same facts.
Most leaders who want to see more lessons run out of hours before they run out of good intentions. A plan that looks fine in September collapses by week four, and the teachers who needed attention are the ones who get none. This lesson goes deeper into two hard questions: how much can you trust what a few visits show, and how do you protect a calendar so that the plan survives a real term, with exams, holidays and ordinary emergencies?
Key ideas
The arithmetic of a cycle
- A visit is not 15 minutes. One cycle costs: notice and agreeing the lens (5 to 10), the visit (15 to 45), writing up facts (10), the conversation (20), a short follow-up (10) and walking between rooms (5). A 15-minute development cycle is therefore about 65 to 70 minutes of a leader's day, a full-lesson cycle about 100 minutes.
- Write the budget before the plan: leaders x protected hours per week x teaching weeks = total hours; total hours divided by the cost of a cycle = how many cycles you can honestly do. A school that plans 80 visits with 40 hours will do 20 badly.
- Trade-off: breadth (everyone gets a little) against depth (a few get a lot). There is no right answer, only a stated one. Say which you chose and why.
A mix of visit types and a fair sample
- Use three kinds on purpose. Short drop-ins of 10 to 15 minutes with one agreed lens give many data points cheaply. Full-lesson visits are rarer and kept for a teacher's own request, a new teacher or a specific concern. Teacher-requested visits are always first in the queue. Learning walks (session 5) look at a school-wide question.
- A fair sample means more than one visit and more than one pair of eyes before any pattern is trusted. The Gates Foundation MET project found that observations were more reliable when more than one observer scored them, and that short observations by extra observers could add information cheaply, although some full-length observations were still important. Never trust one lesson.
- Decide who gets extra: new teachers, teachers who ask, teachers in subjects tied to the school's improvement priority. Say so openly. Hidden extra attention on one teacher feels like surveillance.
Calibration: agreeing on facts before opinions
- Two observers who watch the same lesson often disagree, not about what happened but about what it means. Calibration is a short routine: both watch or read the same lesson, each writes only facts, then each places the facts under the framework, then you compare and agree what an example of 'seen with most pupils' looks like.
- Big systems do this formally. The World Bank Teach tool trains observers and tests them by having them code practice videos and comparing with expert master codes. A school cannot copy that, but it can run paired visits and shared written descriptions twice a term and keep the anchor examples on one page.
- Calibrate before you compare teachers in any way, and say so. Even in pure development use it protects you from the reputation of being a 'strict' or an 'easy' observer.
Records, calendar and protected weeks
- Keep the record small: date, lens, three or four facts, the one agreed step, the follow-up date. One line each. Decide who holds it, how long, and that development records are not copied to appraisal files (see session 1).
- Publish a calendar with protected weeks: the first two weeks of a term, exam and assessment weeks, the days before and after Navruz or other breaks, and any week the school is in the middle of a big event. Share a simple sheet by initials so two leaders do not visit the same teacher in the same fortnight.
- Review the plan at the half-term point. If the hours are gone, cut visits, not the follow-up talk. A visit without a conversation is the most expensive nothing you can do.
One observation cycle
Why a few visits mislead, and what to do about it
A lesson is a sample of one teacher on one day with one class. Time of day, the topic, a previous bad break, a new seating plan: all change what you see. Two observers can also see the same lesson differently. That is why any pattern needs several visits on different days, ideally by more than one person, before it is treated as a pattern.
Reliable is not the same as right. Two calibrated observers can agree on what they see and still both look at the wrong thing. Keep the framework tied to pupils: check what you saw against pupils' work and against what pupils say (session 5). Write your notes as 'seen in this visit' or 'not seen in this visit', never 'this teacher does not...'. One visit can start a question; only several can support a statement.
Protecting the calendar so the plan survives a term
Plans fail at predictable points: the first weeks of term, exam periods, the days around a holiday such as Navruz, school events and staff illness. Mark these first, as protected weeks, and put visits in the remaining weeks. Build one 'catch-up' week near the end for cancelled visits. If a leader cancels a visit, rebook it within the week; teachers notice who cancels.
Make time real. Put the protected hours into the timetable with cover arranged, for example by combining two classes briefly. Keep one sheet, by initials, that all leaders can see, so nobody is visited twice in a fortnight while a colleague is never visited. At the half-term review, report to your head the number of cycles done and the number of conversations held; if the two numbers differ greatly, the plan has become paperwork.
Examples from the classroom
Fifty-six hours, honestly
Scene. The head of a school near Fergana has four leaders who each protect two hours a week for seven weeks. She writes 56 hours on the board, divides by 70 minutes per short cycle and gets 48 cycles at most. She plans 40 short cycles and 4 full-lesson cycles for new teachers, which uses about 53 hours, and keeps the rest for requests. She tells the department the numbers she will report: about 44 cycles and the same number of conversations.
Insight. Count the whole cycle, divide the hours you really have and promise only what the arithmetic allows.
Two deputies calibrate
Scene. In Samarkand, deputies Malika and Aziz each read a written description of a Grade 8 lesson and list facts. Malika wrote 11 facts, Aziz 9, and 7 were the same. Malika wrote 'weak questioning'; Aziz wrote 'six questions, five answered by the same three pupils'. They agree to keep only the second kind of line and write one anchor example: 'questioning seen with most pupils means at least half the class answers in 10 minutes'.
Insight. Calibration moves observers from opinions to counted facts and gives them a shared example.
Navruz and the catch-up week
Scene. A school in Bukhara in the March quarter marks the first two weeks after the holidays and the exam week as protected, and keeps the last week as a catch-up. A deputy's illness cancels five visits in week 5. Instead of dropping them, she rebooks four in the catch-up week and tells the fifth teacher why hers moved. All teachers see the calendar on the staff notice board from the start.
Insight. A protected calendar with one catch-up week lets a plan survive ordinary disruption.
Common pitfalls
- Promising a number of visits that the protected hours cannot cover.
- Treating the visit as the whole job and dropping the conversation when time runs short.
- Judging a teacher from one lesson or from the observer's mood on that day.
- Letting two leaders visit the same teacher in one fortnight because nobody keeps one sheet.
Our Term Plan: Budget, Mix, Calendar and a Calibration Check
- Part A (45 min). In pairs from the same school (or the same cluster) fill the budget table in the Kit with your real numbers: leaders, protected hours per week, teaching weeks, cycle costs. Compute how many cycles you can honestly do.
- Decide the mix: how many 15-minute development visits, full-lesson visits and teacher-requested visits. Who gets extra and why? Write one sentence that you could say to staff to explain the choice.
- Draw the calendar: mark protected weeks and the first week of every cycle. Test it for collisions: does any teacher get two leaders in one fortnight?
- Part B (30 min). Calibration. Each person reads lesson description B in the Kit alone for 5 minutes, lists facts only, and places each fact under one framework element (your one-page framework from session 2 or the sample). Then compare in pairs for 15 minutes.
- Agree one anchor example for the element where you differed most ('this is what seen with most pupils looks like'). Write it on one line. Last 5 minutes: write your leadership task from the Kit.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- How many cycles can you honestly do, and what did you cut or shorten to make the plan fit your hours?
- Where did you and your partner place the same fact under different elements, and what does that say about the wording of the framework?
- Which part of the plan will staff question first, and what will you say?
Reflect
- How many hours a week do you really protect for being in classrooms, and what do you give up to find them?
- Have you ever been judged as a 'strict' or 'easy' observer? What would calibration have changed?
- Which teacher in your school has not been visited by anyone for more than a year, and why?
Try it tomorrow
- Write your leaders' protected hours for the term, divide by the cycle cost and see how many cycles are real.
- Mark the protected weeks in your calendar and add one catch-up week before you promise anything.
- Pick one short lesson description and ask a colleague to list facts independently; compare how many match.
Further reading and links
Plan observation from the hours you really have, mix short and full visits, use more than one lesson and one observer before trusting a pattern, calibrate on facts and protect the calendar.