Switch course ▾
Session 4. Quality and Accountability Without a Compliance Culture, and Evaluating Impact
Objective. Distinguish genuine collaboration from contrived collegiality, set up a light peer-review quality frame, and plan an impact evaluation on Guskey's five levels with honest claims.
Leaders are asked for two things that pull in opposite directions: proof that the network is working, and a network that teachers actually own. Heavy proof tends to kill ownership, and no proof invites doubt. This session shows a middle way: ask for less paper that is read, check quality through colleagues rather than forms, and evaluate on several levels with claims small enough to be true, so that you can show a board something honest without wearing teachers out.
Key ideas
Compliance or commitment
- Hargreaves contrasts collaborative cultures with 'contrived collegiality': formal meetings imposed by administrators to carry out others' plans. His 2019 review notes that when a principal required already-collaborating teachers to keep minutes, they collaborated less. The same paperwork that proves a meeting happened can kill the meeting.
- Datnow (2011) adds nuance. In two districts that required structured data meetings, many became real peer learning, because leaders were strong, trust was built and capacity existed. Rigid protocols limited learning, and lower-capacity schools did not do as well. Structure helps when it supports trust and autonomy, and hurts when it replaces them.
- A practical test for any requirement you place on teams: who is the main reader of this document, and what will they do differently after reading it? If the answer is 'nobody' or 'only the inspector', remove it or merge it into something teams already write for themselves.
A light quality frame
- Quality needs a standard, but not a heavy one. Use five questions for every team: Is there a goal about pupils? Do we look at pupils' work or other evidence? Did a teacher's practice change? Did we see a change in pupils' work? Did we share what we learned beyond the team? Each answer is a level: starting, developing, embedding or leading.
- Use the frame first for self-review by the team, then for peer review between teams or schools in the cluster: a pair of visitors reads the team's evidence folder and attends 20 minutes of a meeting, and reports what they saw and one question. Reviewers are peers, results are shared as findings, and no scores are published or ranked.
- Accountability upward can be short: each team sends the school one half-page per term with its goal, one thing it tried, one thing it saw in pupils' work and one request for help. The school sends the district a one-page summary. That is enough to be answerable without turning teams into report writers.
Evaluating impact on five levels
- Guskey's five levels of professional development evaluation are: participants' reactions, participants' learning, organisation support and change, participants' use of new knowledge and skills, and pupil learning outcomes. They run from easy to hard to measure. Reactions are cheap and weak evidence; pupil learning is hard and most important.
- Plan evaluation backwards and early: decide the pupil outcome you hope for, then the practice that should change, then the school support needed. Collect a baseline before you start, for example five anonymous pupil answers to the same task, so that 'after' can be compared with something.
- Level 3 matters at system level: did the school protect the hours, did the district remove duplicate reports? Many networks fail here, not in the classroom. Report it honestly, because it is the part leaders can fix.
Honest data, modest claims
- Practice changes before pupil results do, so a term is rarely enough to see a change in pupils' marks. Report what you saw at each level, with the number of teachers and pupils it covers, and say what you cannot claim: you did not compare with a similar group, other things changed too.
- When a measure becomes a target, people start chasing the number. If teams are judged on test marks alone, they will teach to the test and stop looking at thinking. Use two or three kinds of evidence, including pupils' own explanations, and never rank teachers or teams by them.
- Protect people: pupil work is anonymised, results are reported for the group not for the individual teacher, and evidence is shown first to the people who produced it. The EEF guidance on effective professional development reminds leaders to build knowledge, motivate, develop techniques and embed practice, and to implement with care for context.
Five levels of evaluation (after Guskey)
Where structure helps and where it harms
Datnow's case studies are a useful correction to a simple message that all required meetings are bad. In districts with strong leaders and existing trust, required data meetings often grew into real peer learning. But rigid protocols limited what teachers could try, and schools with less capacity did not do as well. The practical lesson is to match structure to capacity: more structure for new teams, less for mature ones.
This gives you a rule of thumb for your own network. New teams get a simple meeting shape and a short termly note. Teams that have shown they look at pupils' work get more freedom to choose their own question and format, and a lighter touch from above. Release control as trust is earned.
Reading the five levels as a chain
Guskey's five levels work best as a chain, not a menu. Teachers' reactions and learning (levels 1 and 2) matter, but they only turn into pupils' gains if the organisation supports the change (level 3) and teachers actually use what they learned (level 4). When pupils' results do not move, the break in the chain is usually at level 3 or 4, not at level 5.
So your report should show where the chain holds and where it breaks. For example: reactions positive; hours protected in two of three schools; practice changed in five of six teams; pupils' reasoning visible in nine pieces of work from one class. That is a more useful and more honest picture than a single number, and it tells the board what to fix.
Examples from the classroom
The document that nobody read
Scene. A district in Namangan collects full weekly minutes from 40 PLC teams. An officer admits that in two years nobody has read more than a sample. The cluster proposes a half-page note per term: goal, one thing tried, one thing seen in pupils' work, one request. The officer reads all 40 notes in an afternoon, picks three themes and writes back. Teams report that they now know their notes have a reader.
Insight. A shorter document with a reader changes behaviour more than a longer one without.
Peer review without ranking
Scene. Two teachers from a Bukhara school visit a team in another school for 20 minutes after reading its evidence folder. They report three things they saw, one thing they would try, and one question. No score is given. The visited team says the question about what pupils did after the second task was more useful than any mark. In the following term, three teams ask to be visited.
Insight. Peer review works when it returns findings and questions, and when no ranking is published.
A modest claim
Scene. After one term, a Samarkand cluster has five anonymous pupil answers per team collected at the start and at the end on the same task. In 9 of 14 teams, more answers at the end include a reason. The leaders write: 'In nine teams we saw more reasons in written answers. We have no comparison group, and marks have not changed.' The board asks for another term of evidence rather than ending the work.
Insight. A claim that names its limits is believed and gives the work time.
Common pitfalls
- Asking for the same information in several documents for different readers.
- Using test marks alone as proof and ranking teams by them.
- Collecting evidence only after the work, with no baseline to compare with.
- Publishing the results of a peer review as scores.
Audit the Paperwork and Plan the Evaluation
- Open the Activity Kit: a SAMPLE list of ten documents asked of PLC teams, a SAMPLE self-review of six teams on the five questions, and the evaluation-plan table. In your own cluster, use the real list of documents teams must send. (5 min)
- Alone (12 min): for each of the ten documents write the main reader and what that reader does with it. Mark each Keep, Merge or Remove.
- Pairs (18 min): read the sample self-review. Decide what the cluster should do with the lowest question across teams, and who would be the peer reviewers. Write the three rules for peer review (what is read, who sees results, what is never done).
- Groups of four (20 min): fill the evaluation-plan table for the cluster on all five Guskey levels: what evidence, from whom, when, how cheap or costly, and the claim you could honestly make.
- Part B, the message in 20 minutes: write a five-line message to teams that removes or merges at least two documents and explains what replaces them and why. Read it to a partner who plays a doubtful team leader.
- Alone (10 min): write your leadership task from the Kit.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which document in your own cluster exists only to prove that something happened?
- What would peer reviewers need to read to judge a team fairly in 20 minutes?
- Which claim in your evaluation plan would be an overclaim, and how would you rephrase it?
Reflect
- What does your school ask teams to write that nobody reads?
- Where in your system is it safe to say 'this did not work'?
- Which of Guskey's levels do you currently measure, and which never?
Try it tomorrow
- Ask your teams which document they would stop writing tomorrow, and for whom they write it.
- Choose one team and read its evidence folder for 20 minutes, then give back one thing you saw and one question.
- Collect five anonymous pupil answers to one task now, as a baseline for later.
Further reading and links
Ask for less paper and more looking at pupils' work; judge teams by peer review and modest evidence on five levels, never by ranking.