Switch course ▾
Session 4. Risk, Vendors and Incident Response
Objective. Lead a group in scoring AI risks on a three-by-three matrix with controls, putting ten written due diligence questions to a vendor and spotting weak answers, and running a short practice drill of an incident (harmful output or data leak) with a named lead, a timeline and a no-blame review.
Risk work sounds technical, yet it is mostly honest conversation. What could go wrong, how likely is it really, how bad would it be for a child, and what would we do about it? This lesson goes beyond the matrix and the ten questions. It shows how to estimate likelihood from evidence rather than mood, how to choose controls that actually work, how to hold a firm but fair conversation with a vendor, and what to do in the first hour of an incident, when calm matters more than speed.
Key ideas
A risk matrix leaders can use
- Score each risk twice: how likely (1 rare, 2 possible, 3 likely) and how bad (1 minor and fixed in a day, 2 serious but recoverable, 3 severe or harming a child). Multiply: 1-2 low, 3-4 medium, 6-9 high.
- One override: a severe impact on a child is always high, even if unlikely. Do not let a pleasing number hide harm to a child.
- Then write the control (who does what to lower it) and re-score. A matrix is useful only when it ends in actions with owners. Keep the list to the ten risks that matter this term.
Ten questions for a vendor
- Course 14.6 taught weighted scoring of offers and contract terms. Here the focus is trust: who is the company, what data does it collect, is pupil data used to train its models, where is data stored and who can reach it, how long is it kept, and can you delete it?
- Also ask: which ages is it designed for and what child settings exist, does it work well in Uzbek and Russian on your own sample texts, what happens to our data if we leave, and how fast will you tell us about a security incident?
- Ask in writing and keep the answers. Vague words such as 'we take privacy seriously' are not answers. A refusal to answer is information too. Free offers deserve more questions, not fewer: if the service is free, find out what it collects.
Incidents will happen: be ready
- Two likely kinds in schools: harmful output (a tool shows inappropriate or dangerously wrong content to a pupil) and data leak (pupil names and phone numbers pasted into a tool, or a list sent to a wrong group chat).
- Six steps: Contain (stop the use, in the first hour) - Tell (a named lead and the head at once) - Assess (what, whose, how many, how harmful) - Notify (affected families, and authorities where the rules require it; confirm the rules with your education department) - Fix - Learn.
- Set your own targets, such as 'lead informed within one hour, families told the same day', and do not claim legal deadlines you have not checked.
Practise, log and learn without blame
- A plan nobody has practised fails on the day. Once a year run a 30-minute table drill: read a scenario, ask 'who does what now?', and time the answers. Put the contact list (lead, head, deputy, department) on one page on the office wall.
- Keep an incident log: date, what happened, tier of the tool, who was told, what was fixed. Keep evidence (a note of what appeared) without copying children's data further.
- After every incident hold a no-blame review within two weeks: what made the mistake easy? Punishing the teacher who reported makes the next report disappear. Update the policy or training the same month.
Reading a three-by-three score
Honest scores and controls that work
Scores shift with mood: after a scare everything is 'likely'. Anchor them in evidence: check the incident log and count how many teachers have ever pasted pupil data into a tool. A paper survey of ten colleagues, without names, moves a score from guess to fact. Score the risk today, add the control, then score the residual risk. A control without an owner and a date is a wish.
Choose controls from strongest to weakest: stop the use; redesign it so the risk disappears; add a technical limit such as an approved tool with a school account; train and remind; check afterwards. Training alone is weakest, because people forget under pressure. Where a child could be harmed, use at least one of the first three.
Questioning a vendor and the first hour
Send a vendor your questions in writing with a reply date, and ask every candidate in the same format. Ask what happens on the last day: how do we get our materials out and see our data deleted? Test with invented data, never real pupil lists. If an offer is free, ask what the company earns. A good vendor explains patiently; a poor one hurries you.
In the first hour of an incident, calm beats speed. The finder stops the use and tells the named lead, who writes down the time, what was seen and who knows. Keep only the notes needed, in a safe place, without sending children's data further. The head decides who tells families, with a short honest message: what happened, what is done, when the next news comes.
Examples from the classroom
A guess becomes a fact
Scene. Jasur, a deputy head near Bukhara, rates the risk that teachers paste pupil names into a chatbot as 'unlikely'. Dilnoza proposes a paper survey of twelve teachers, without names. Five say they have done it at least once, mostly for report comments. The score changes from 1 to 3 and the impact is serious, so the risk becomes high. The control is an approved tool with a school account and a short training with examples.
Insight. Evidence, even from a small paper survey, moves a risk score from guess to fact.
Test with invented data
Scene. Malika evaluates a quiz tool for Grade 5 in Samarkand. The vendor offers a demo with her real class list. She declines, and instead creates a list of ten invented pupils with invented names. After the trial she asks the vendor in writing how the invented data is deleted. The reply is vague, so she asks again. A second answer promises deletion in 30 days. She notes it in the register as a condition.
Insight. Invented test data lets you check a tool without exposing children.
The first hour
Scene. At 9:10 a Grade 7 pupil in Tashkent shows Aziz an unsuitable answer from an approved tool. Aziz stops the class use, writes the time, and phones the lead at 9:20. The lead tells the head at 9:35 and the vendor's contact receives a written report at 10:00. The head sends a short note to parents at 14:00. At the no-blame review, the team adds a filter setting and a one-line reporting form.
Insight. A rehearsed sequence and a calm tone make the first hour manageable.
Common pitfalls
- Scoring risks by mood after a scare, not from evidence.
- Writing controls without an owner, a date or a way to see that they work.
- Accepting a vendor's marketing phrases as answers to specific questions.
- Testing a tool with real pupil lists, or sending children's data on to colleagues during an incident.
Score, Question and Drill
- Open the Activity Kit: eight SAMPLE risks, a vendor's sample answers to eight questions and two incident scenarios from Zarafshon School (5 min).
- Pairs (12 min): score the eight risks (likelihood, impact, product, override) and write one control with an owner for each high risk.
- Pairs (10 min): read the vendor's answers. Mark each as clear, vague or missing, circle the red flags and write three follow-up questions.
- Part B, groups of four (20 min): drill scenario 1 or 2. Fill the incident timeline in the kit, with a named lead for each step and a one-line message to families. Then hold a three-minute no-blame review.
- Whole group (8 min): groups swap timelines and name one step that would fail in a real school (no phone number, head absent) and a fix.
- Offline fallback: everything is on paper and the drill is spoken aloud with a watch; no tool needs to be opened. Do not paste any real pupil list into a device during the drill.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which risk changed from high to medium after a control, and who owns that control?
- Which vendor answer would have been easiest to accept without noticing it was empty?
- In the drill, what happened in the first hour, and who would really have done it?
Reflect
- What is the likeliest AI incident in your school this year, and who would know first?
- Would a teacher who made a mistake report it to you today?
- Which free offer in your inbox deserves ten questions?
Try it tomorrow
- Run a ten-colleague paper survey on one risk and re-score it with the results.
- Ask one real supplier your ten questions in writing and set a date for the reply.
- Walk through the six incident steps with two colleagues, timing each one with a watch.
Further reading and links
Score risk twice and attach an owner to each control, put ten written questions to every vendor, and practise six incident steps so that the first hour is calm.