FairGrade: Auditing Bias in AI-Assisted Grading in Multilingual Swiss Education
How fair is AI-assisted grading? We study bias in multimodal LLM grading of handwritten exams across French- and German-speaking Switzerland.
Duration: January 2027 – December 2027
Education levels: Upper secondary level, tertiary level
Topics: AI – Artificial Intelligence, Digital Ethics
Keywords: AI grading; fairness; bias; multilingual education; LLMs
Initial Situation
AI-assisted grading is spreading fast. In the OECD’s 2024 TALIS survey of 280,000 teachers, about one in three already use AI in their work, and a quarter of those use it to grade student work. Multimodal AI models can now even read handwriting directly, further simplifying the processing of handwritten exams.
But fairness concerns are mounting. Studies show AI graders can reward longer or more fluent answers over correct ones, and can be swayed by writing style rather than content. Essays in “non-native” English have been marked down by up to 15%, and regional dialect patterns trigger negative judgments from language models. Handwriting itself is known to correlate with gender, ethnicity, and other identity traits, raising the question of whether it could also skew AI grades.
Switzerland’s multilingual education system makes this study urgent: students write in French, German, and regional dialects, with varying fluency and handwriting styles. No study has yet examined how these factors interact to bias AI grading of handwritten Swiss exams, or how AI grading bias compares to human grading bias. Institutions need evidence to better evaluate the risks posed by AI automated grading and determine appropriate guardrails and policies.
Objectives
We aim to: (1) measure how much handwriting, language of submission (French/German), and language fluency affect AI-assigned grades; (2) compare bias patterns between human and AI graders on handwritten STEM exams and longer written assignments; (3) identify mitigation strategies that could reduce bias; and (4) build a reusable benchmark for evaluating fairness in automated grading. The project translates these findings into practical guidance for institutions and educators considering AI-assisted grading.
Method
We build a benchmark of handwritten French and German exams: authentic Physics exams from EPFL (200+ per language), a controlled study where participants write semantically equivalent answers with varied handwriting, fluency, and demographic name cues, and a smaller set of written business assignments.
Human evaluators grade everything with standard rubrics. We then have multimodal LLMs grade the same material and compare AI and human grades across demographic and linguistic conditions to isolate bias, before testing mitigation strategies such as anonymization and rubric-constrained prompting.
Planned Translation
Findings feed directly into six practitioner workshops (in German and French) with educators from EPFL, BFH, and BeLEARN network schools, where we present results and co-develop guidance with teachers regarding AI grading tools. Outputs include an internal benchmark institutions can reuse to test AI grading systems, a comparative bias report across AI models, and practical guidelines on bias in AI grading systems.
Short term, the project gives Swiss educators concrete evidence on bias in AI-assisted grading, letting schools and universities make informed adoption decisions rather than guessing. Longer term, it establishes reusable evaluation methodology and benchmarks that later studies can extend to more cantons, subjects, and assessment types. We measure impact through workshop uptake and educator feedback, and through adoption of the benchmark and guidelines by participating institutions.
