Grand Challenges 2026
Grand Challenges in Comparative Judgement
Wednesday 13th and Thursday 14th May 2026
Burleigh Court Conference Centre, Loughborough University
Funded by the Engineering and Physical Sciences Research Council.
Day 1 Agenda
Wednesday 13th May 2026, 9:30am to 5:00pm
9:15
Arrival refreshments.
9:30
Welcome and introduction (Ian Jones, Marie-Josée Bisson, Rowland Seymour).
9:45
Ben Lauderdale, University College London.
Comparing comparative judgements of public opinion made by citizens, MPs, and LLMs.
10:00
Safa Elsheikh, Loughborough University.
Comparative judgement in medical imaging.
10:15
Tuya Sa, Loughborough University.
Comparative judgement and mathematical ugliness.
10:30
Mid-morning refreshments.
11:00
Matthew Inglis, Loughborough University.
Issues with eliciting intuitions.
11:15
Dave Sirl, University of Nottingham and DAMSL Ltd.
Can we wring more information out of our comparative judgement data?
11:30
Lapo Santi, University College Dublin.
The Bradley-Terry stochastic block model.
11:45
Robin Hankin, University of Stirling.
Generalizing Plackett-Luce to accommodate draws: the hyper2 package as applied to chess, the Scottish Premier League, and the Eurovision Song Contest.
12:00
Lunch.
13:00
Sabina Sloman, University of Manchester.
Well-specified models & well-designed experiments go hand in hand.
13:15
Joe Marsh, University of Birmingham.
Bayesian experimental design for comparative judgement.
13:30
Depy Makri, Loughborough University.
Multiple-criteria comparative judgement and mathematics question qualities.
13:45
Tom Bramley, Ofqual.
Comparative judgement in educational assessment.
14:00
Andy Gray, Bath Spa University.
Bayesian active learning for multi-criteria comparative judgement.
14:15
Anna Brown, University of Kent.
Measuring individual differences with ipsative questions.
14:30
Afternoon refreshments.
15:00
Beth Woollacott, University of Nottingham.
Comparative judgement in qualitative research.
15:15
Chris Wheadon, No More Marking Ltd.
Validating AI decisions.
15:30
Serena Rossi, Loughborough University.
Open science: from principles to practice and future directions.
15:45
George Kinnear, University of Edinburgh.
Comparative judgement as a learning activity.
16:00
Marie Hoffelinck, University of Liège.
Making comparative judgements: a learning perspective on the judges’ experience.
16:15
Charlotte Norridge, University of Birmingham.
Heterogeneity in comparative judgement: a Bayesian Bradley-Terry framework.
16:30
Whole group discussion.
17:00
Event finish.
18:45
Dinner.
Day 2 Agenda
Thursday 14th May 2026, 9:30am to 4pm-ish
9:15
Arrival refreshments.
9:30
Welcome back and overview of the day (Marie-Josée Bisson).
9:45
CJ-RAVE results (Joe Marsh).
9:55
Online tools for question refinement (Rowland Seymour).
10:00
Question refinement 1
Working in pre-assigned mixed-discipline groups.
10:30
Mid-morning refreshments.
10:45
Question refinement 2
Working in pre-assigned mixed-discipline groups.
12:00
Whole group discussion.
12:30
Lunch.
13:30
Question refinement 3
Working in pre-assigned thematic groups.
14:30
Afternoon refreshments.
15:00
Whole group discussion.
15:45
Close and next steps (Rowland Seymour).
Conveners
- Ian Jones, Loughborough University
- Marie-Josée Bisson, De Montfort University
- Rowland Seymour, University of Birmingham