AI Graders Also Go Off Script When Scoring Criteria Change... Soongsil Undergraduates and Research Team Expose AI Flaws
- Input
- 2026-11-18 16:59:07
- Updated
- 2026-11-18 16:59:07

[Financial News] A South Korean university research team has identified hidden errors and biases that arise when artificial intelligence (AI) grades tests or handles tasks in place of humans. The achievement was driven by undergraduates as well as graduate students.
Soongsil University announced on the 18th that eight papers by Professor Chanjun Park’s team in the School of Software (Natural Language Processing Laboratory) had been accepted to EACL 2027, Europe’s most prestigious international conference in natural language processing. EACL draws researchers from around the world, and its review process is rigorous: this year, the acceptance rate was just 13.5%.
The most notable aspect of this achievement is that students led the research. Students affiliated with the lab served as first authors, leading the research, on five of the eight accepted papers. Among them, researchers Lee Si-hyeon and Geum Ji-u demonstrated their ability to get first-author papers accepted at an international conference while still undergraduate researchers who had not yet graduated.
The team closely examined the “reliability” of large language models (LLMs), which are widely used in businesses and everyday life. One notable study exposed flaws in “AI graders.” Researcher Kim Do-yun found that even when given the same answer and criteria, AI produces different evaluation results depending on how the scoring scale is set (for example, 0–5 points or 0.3–0.7 points). Researcher Kim Chan-u identified a bias in which AI is swayed by the size of the numbers rather than the meaning of the evaluation criteria, assigning inappropriate scores when 5 is the highest score versus when 1 is the highest.
The research led by undergraduates also offered a penetrating analysis of how AI works. Researcher Lee Si-hyeon examined how AI responds when given conflicting instructions at the same time, such as “answer carefully” and “answer helpfully.” The study found that AI tended to conceal the conflict from users and secretly choose just one instruction, even when it recognized that the two instructions clashed. Researcher Geum Ji-u identified how AI’s core internal functions are preserved rather than destroyed during “optimization,” the process of reducing an AI model’s size to increase its speed.
The team was also recognized for research across eight topics, including a timing mismatch that occurs when the latest AI systems that understand text, images and sounds simultaneously (omni models) process information internally, studied by researcher Jo Seong-hyeon, as well as machine translation bias and multilingual processing technologies.
Professor Chanjun Park said, “This is the result of students pursuing the questions they raised themselves to the very end and growing together,” adding, “We will continue our research into reliable AI so that our findings can be applied in real-world industry settings.”
[email protected] Kim Man-gi Reporter