Switch course ▾
Session 3. Writing Good Items: Closed, Open and Competency-Based
Objective. Write clear closed items with plausible distractors, open-response items with a tested rubric, and a competency-based item with a necessary stimulus.
Item writing looks easy until a pupil answers correctly for the wrong reason or two markers give the same answer different marks. The craft is learnable and does not need special software. This lesson goes beyond the checklist: how closed items can test more than recall, how to mark open answers so that two teachers agree, and how to keep competency-based items short enough that you test the subject, not reading speed. You will test items with a colleague to find flaws you cannot see yourself.
Key ideas
Closed items: stem and options
- The stem should make sense on its own and pose one clear problem. Leave out material that is not needed; extra words test reading, not the subject. Write a question or a sentence to complete, not a blank in the middle.
- Options should be similar in length, grammar and detail, and not overlap. Avoid 'all of the above' and 'none of the above': pupils can reach them by partial knowledge. Avoid negatives; if one is needed, emphasise it (NOT).
- Vary the position of the key across the test, and check that no option is a giveaway because it is longest, most detailed or the only one that fits the grammar of the stem.
Distractors that come from real errors
- A good distractor is wrong in a way that real pupils go wrong. Take them from earlier wrong answers, exit tickets or common mistakes you know: adding instead of multiplying, confusing two terms, taking the first number in the text.
- Write the key first, then each distractor with its error in mind. If you cannot name the error behind a distractor, it is probably filler. Three good options are better than four with a silly one.
- Test the item blind: ask a colleague to solve it without the key. If they hesitate between two options, or choose a different key, the item has a flaw you cannot see.
Open-response items and rubrics
- Use open items when you need reasoning, explanation or a method. Ask a question that cannot be answered by one word, and tell pupils how many marks it carries and what is expected, for example 'Show your work and explain'.
- A rubric describes what an answer looks like at each score. Write from the top: describe the best answer first, then each lower level, covering the same elements. Use observable words ('states the loss as a number'), not vague ones ('good', 'sufficient').
- Test the rubric: two markers mark the same five answers, then compare. Each difference shows a descriptor that needs rewriting. Decide in advance how to treat a correct method with an arithmetic slip.
Competency-based items in practice
- Build a unit: one stimulus (a price list, a short text, a table of results) and two or three questions that rise from retrieve, to apply, to decide or justify. Large studies often present a stimulus with several questions; you can do the same on a small scale.
- Keep the reading load low. If a stimulus is long or uses rare words, you are testing reading as well as the subject. Use local, believable situations: market prices, a school garden, bus timetables, Navruz fair stalls.
- Check that the situation is necessary: remove it and see whether the question survives. Then check fairness: does every pupil know enough about the setting to answer? (See Session 5.)
The item-writing cycle
Closed items that test more than recall
Multiple-choice does not have to mean memory. Give a short situation, then ask for the best explanation, the next step or the valid conclusion, and make the options different lines of reasoning. For example, show a fair-test plan and ask what is wrong with it, with options that name different problems. Pupils must think; a pupil who memorised a definition cannot succeed.
Two cautions. First, closed items test recognising a good answer, not producing one, so keep open items for explanation. Second, a clever item is not a deep item: trick wording, close options and double negatives make pupils fail for reading reasons. Aim for a clear stem and distinct options whose differences rest on the idea you taught. If a colleague needs three reads to understand the stem, rewrite it.
Marking open answers so that two teachers agree
Most disagreement between markers comes from vague descriptors, not from carelessness. Before marking, pick three or four sample answers that clearly deserve different marks and use them as anchors. Write what each shows in observable words. Mark one question at a time across all scripts with the name covered, so that one pupil's earlier answers do not colour later ones.
Decide in advance what to do with common borderline cases: a correct method with an arithmetic slip (follow-through credit), a correct idea in unusual words, or a correct answer with no working. Write the decision into the rubric. Then double-mark five answers with a colleague and discuss each difference. Released items from large studies come with marking guidance and are good models of how an open answer can be scored by clear rules.
Examples from the classroom
A fair-test item with reasoning
Scene. Dilnoza, a Grade 7 science teacher in Samarkand, writes: 'Malika puts one tray of cotton seedlings in sunlight and another in a dark cupboard. She also gives the dark tray more water. Why is her conclusion about light not reliable?' The options name different problems. The key says the trays differ in water as well as light. A pupil who only memorised the word 'variable' cannot choose it.
Insight. A closed item can test reasoning when the options are different lines of thinking about a short situation.
Repairing a rubric
Scene. Two Grade 9 literature teachers in Fergana mark an essay answer with a rubric that says 'good explanation: 2 marks'. Their marks differ on three of five scripts. They rewrite it: 2 marks 'quotes a line and says what it shows'; 1 mark 'refers to the text without saying what it shows'; 0 marks 'no reference to the text'. On the next five scripts they agree on four.
Insight. Observable descriptors turn a matter of taste into a decision two markers can share.
Cutting the reading load
Scene. Jasur's Grade 6 word problem in Bukhara has a stimulus of about 120 words about a school bazaar. His Russian-medium colleague Sardor suggests cutting it to about 40 words and putting the prices in a small table. The same mathematics is tested, but a pupil who reads slowly now spends the time on the calculation instead of decoding long sentences.
Insight. Reduce the reading load to what the item needs, so the item tests the target skill.
Common pitfalls
- Using tricky wording or very close options and calling it a harder, better item.
- Writing a rubric of adjectives such as 'good' or 'excellent' that two markers read differently.
- Marking all questions of one pupil together, so that a strong first answer lifts the rest.
- Making a long stimulus without checking how many words a slow reader must handle.
Item Writing Clinic
- Trios (10 min): open the Kit. Use the item checklist to find the flaws in the three flawed closed items. Write each flaw in a few words.
- Trios (10 min): rewrite the three items. For the maths item, name the error behind each distractor in the distractor table. Then swap with another trio, who solve your items without the key (blind test).
- Pairs (15 min): write a 3-mark rubric for the Navruz pilaf task in the Kit. Then each of you marks the four sample answers alone. Compare marks and rewrite any descriptor that caused a difference.
- Second short activity, pairs (15 min): from your own unit, write a competency-based block: a short stimulus and three questions (one retrieve or know, one apply, one decide or justify). Swap and run two checks: 'delete the story' and 'how many words must a pupil read?'.
- Whole group (10 min): each pair reads one revised item aloud. The group names what changed and why the new version is better.
- Alone (10 min): write your school-based task from the Kit and choose the colleague who will blind-solve your items.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which flaw was hardest to see in your own items, and how did the blind test help?
- Where did two markers disagree about the pilaf answers, and what did you change in the rubric?
- Would your competency-based block still work if the situation were deleted? What does that tell you?
Reflect
- Which of my past closed items could be answered by a pupil who had not studied, using a clue?
- What errors do my pupils make most often in my subject, and are they in my distractors?
- How do I mark open answers today, and could another teacher reach the same mark?
Try it tomorrow
- Take one closed item from your last test and rewrite its options so each is a different line of reasoning.
- Choose one open item, write a 3-mark rubric in observable words and test it on two sample answers with a colleague.
- Cut a stimulus in one of your items by a third and check that the question still needs it.
Further reading and links
Write the key, then distractors from real errors; solve blind; give open items a rubric that two markers have tested.