Switch course ▾
Session 4. Difficulty, Discrimination and Improving Items
Objective. Calculate the difficulty index (p) and the discrimination index (D) from a small class set, read distractor counts, and decide whether to keep, revise or drop each item.
Item analysis is often taught with formulas and then forgotten. In a real school it is a short routine after a test: sort papers, count, decide. The aim is not statistics for their own sake but better tests next time and fairer marks this time. This lesson goes beyond the formulas: why p and D depend on each other, how to read distractor patterns, when a hard item is a teaching signal rather than a bad item, and how to keep a small bank of trusted items.
Key ideas
Two questions for every item
- After a test, most teachers look at total marks. Item analysis adds two questions for each item: how many pupils got it right (difficulty), and do pupils who did well overall get it right more often than those who did poorly (discrimination)?
- The answers do not judge pupils. They judge the item, and sometimes your teaching. An item that nobody gets wrong tells you little; an item that strong pupils get wrong more than weak pupils is a warning.
- You need no software. Paper, a pen and 20 minutes are enough for a class of 24 and a test of 20 items.
Calculating p and D by hand
- Sort the papers by total score. Take the upper group (the top third, or 8 papers in a class of 24) and the lower group (the bottom third, 8 papers). Groups must be equal in size. The middle papers are left out of D.
- p = number correct in the whole class / number of pupils. D = (correct in upper group - correct in lower group) / group size. Example: 7 upper and 2 lower correct in groups of 8 gives D = (7 - 2) / 8 = 0.63.
- A common starting guide is p between about 0.25 and 0.75 and D of at least 0.20. These are signals to look closer, not laws: an essential basic item may be easy on purpose, and a very hard item can still be a good stretch.
Distractors and decisions
- Count how many pupils in the upper and lower groups chose each option. A healthy distractor attracts some lower pupils and few upper ones. A distractor nobody chooses does no work. If more upper pupils choose a wrong option than the key, suspect a wrong key or an ambiguous option.
- Then decide: KEEP (works as intended), REVISE (fix wording, replace a dead distractor), or DROP (flawed and not worth saving). Write the reason beside each decision.
- If an item is miskeyed and you correct the key, rescore all papers and tell pupils what you changed and why. Quietly dropping marks damages trust.
Item problem or teaching problem?
- A hard item may be well written and expose a gap in teaching. If the key is right, the wording is clear and strong pupils also miss it, reteach the idea. If the wording or key is wrong, fix the item.
- With 20 to 30 pupils, one or two papers can move D by 0.10 or more. Treat the numbers as a signal, look at pupils' work too, and trust patterns across several items or several tests more than one item once.
- Keep a simple item card for items worth reusing: wording, key, date, p, D and any change. Over a few terms you build a small, trusted bank. These class numbers describe your test and your pupils only; they are not comparable with scores from other tests.
The paper item-analysis routine
What p and D can and cannot say
Difficulty and discrimination are linked. An item that almost everyone gets right cannot separate pupils, so its D is low even if the item is fine; the same is true of an item almost everyone misses. D can be highest for items of middle difficulty. So a low D on an easy item is no defect if it checks an essential basic skill.
Also remember what class numbers are. With 24 pupils, one paper moves D by about 0.13. One university guide suggests looking hard at items whose D falls below about -0.10, and another gives 0.20 as a minimum for D. Use these as prompts to check the wording, key and distractors, never as automatic deletion rules. Patterns across two or three tests are more trustworthy than one result.
Reading distractor patterns and keeping a bank
Four patterns cover most cases. (1) The key is chosen mostly by the upper group and the wrong options by the lower group: healthy. (2) One wrong option is chosen by nobody: replace it with a real misconception. (3) Many upper pupils choose one wrong option: suspect a wrong key or an option that is also correct. (4) Choices are spread evenly across all options in both groups: pupils are guessing, so the idea was not learned or the stem is unclear.
Keep a card for each item worth reusing, with wording, key, date, p, D and changes. After two or three terms you hold a small bank you trust more than a new draft. Store it carefully; if pupils see the items in advance, it loses its value. Share cards with a colleague who teaches the same grade.
Examples from the classroom
A test that was too easy to learn from
Scene. Sardor, a Grade 8 maths teacher in Tashkent, finds that 14 of his 20 items have p above 0.85 and low D. His tests make pupils feel good but give him little information. He keeps six easy items for essential skills, replaces six with apply items and adds two reasoning items. Next test, the marks spread out and he can see which pupils need what.
Insight. A test of only easy items rarely discriminates; deliberate variety gives more information about learning.
The second correct answer
Scene. Gulnora, a Grade 7 science teacher in Samarkand, asks: 'Which material is a good conductor of electricity? A) copper wire B) steel spoon C) wooden ruler D) plastic cup.' Item analysis shows many upper pupils chose B. Steel conducts too, so there are two correct options. She rewrites the stem as 'best conductor in the table' with values, rescoring the original item as both A and B accepted.
Insight. When strong pupils choose a wrong option, check whether it is also right before you question the pupils.
A hard item that was really a teaching signal
Scene. Aziz's Grade 6 geography item in Fergana asks pupils to read a bar chart; only 5 of 24 get it right, including few strong pupils. The wording is clear and the key is right. He realises he never practised charts. He teaches two short lessons on reading charts and gives a parallel item two weeks later, which more than half the class now answers correctly.
Insight. A low p with a good item is often a teaching problem; reteach instead of dropping the item.
Common pitfalls
- Treating the thresholds 0.25-0.75 and 0.20 as laws and deleting every item outside them.
- Calculating D with groups of different sizes or with the middle pupils included.
- Changing marks after a test without telling pupils what changed and why.
- Reusing items in public view so that they lose value, instead of keeping them secure.
Item Analysis Lab
- Pairs (10 min): open the Kit. A class of 24 took a 5-item test; the groups are upper 8, middle 8, lower 8. Calculate the difficulty index p for each item.
- Pairs (10 min): calculate the discrimination index D for each item. Mark any item with p outside 0.25-0.75 or D below 0.20.
- Pairs (15 min): read the distractor counts for items 2, 4 and 5. For each item write one decision (keep, revise or drop) and the reason in one sentence.
- Whole group (10 min): compare decisions. For item 4, role-play the conversation in which the teacher tells the class that a key was wrong. Practise a calm, honest sentence.
- Second short activity, pairs (15 min): practise the paper tally with your own last test (or the test you will give). Sort the papers into three piles, tally one item, and write an item card for it. Plan when you will do this for the rest.
- Alone (10 min): write your school-based task from the Kit and agree the test you will analyse with a colleague.
Your facilitator will guide you through this activity in the session.
Questions to discuss
- Which item looked fine from the total marks but was flagged by D?
- In item 4, what would you say to the class, and what would you do with the marks?
- When would you accept an easy item (p above 0.75) in a test on purpose?
Reflect
- Have I ever blamed pupils for a result that was really caused by an item?
- Which items from my past tests would I want in a small trusted bank?
- What do I tell pupils when I change a mark after a test, and when?
Try it tomorrow
- After your next test, sort the papers in three piles and calculate p and D for the five items you are least sure about.
- Write an item card for one item worth keeping and one note on a change you will make.
- Choose one low-p item where the key is right and plan a short reteach before you test it again.
Further reading and links
Count p and D by hand, read distractors, decide keep, revise or drop, and treat small-class numbers as signals.