Monday, November 16, 2026

Shinhan Bank Goes Beyond AI Responses to Verify How Tasks Are Performed

Input
2026-11-16 14:35:58
Updated
2026-11-16 14:35:58
View of Shinhan Bank in Jung District, Seoul. Photo courtesy of Shinhan Bank.

[Financial News] Shinhan Bank presented research findings aimed at enhancing the safety and reliability of financial AI at an international academic conference.
According to the financial industry on the 16th, Shinhan Bank presented two research papers on financial AI at a workshop held on the 15th at the International Conference on AI in Finance (ICAIF) 2026 in Milan, Italy. ICAIF is an international academic conference where researchers and industry professionals share the latest research and examples of industry applications focused on the convergence of finance and AI technology.
At the conference, Shinhan Bank introduced a technology for verifying AI agents’ performance of banking tasks, using the ‘K-BankEval’ benchmark framework for evaluating large language models (LLMs) suited to banking tasks and ‘Agent Foundry,’ its internal AI agent evaluation and verification platform.
‘K-BankEval’ is an LLM evaluation framework comprising a total of 5,000 questions and six evaluation areas, designed to reflect the characteristics of banking tasks such as lending, deposits and foreign exchange.
The evaluation areas include banking expertise; financial document retrieval and evidence-based responses (RAG); calling tools required for tasks; calculations and reasoning related to banking; response safety from the perspective of financial consumer protection; and AI agents’ plans for carrying out tasks.
A comparative evaluation of six LLMs by Shinhan Bank found that even models with strong general-purpose AI performance differed in their ability to carry out actual banking tasks. Based on this, the bank established an evaluation foundation that can help select LLMs suited to specific task requirements and identify each model’s weaknesses.
The bank also established a framework for verifying not only AI agents’ final answers but also the process by which they carry out tasks. Shinhan Bank used its internal evaluation and verification platform, ‘Agent Foundry,’ to enable inspection of the execution process of AI agents used in banking.
Specifically, task requirements are broken down into verification cases and evaluation criteria, after which the AI agents’ answers and tool execution records are analyzed. This makes it possible to check whether responses meet work standards, whether the necessary tools were used appropriately and whether the task execution process was efficient.
This verification framework can be used to systematically determine whether requirements have been met and what areas need further improvement before AI agents are introduced into actual banking tasks.
The research also presented a method for verifying required checks for each task by linking them to actual execution records. It outlined ways to improve the evaluation framework, including distinguishing cases with insufficient grounds for judgment from cases that pass normally. Shinhan Bank plans to use the verification technology to improve the accuracy and stability of AI-based task processing.
A Shinhan Bank official said, “To use AI in the financial sector, it is important to verify not only that it performs well but also that it is suited to actual tasks and that those tasks are carried out in accordance with standards,” adding, “We will continue to advance AI evaluation and verification technologies tailored to the financial environment and build a financial AI environment that customers and employees can use with confidence.”

[email protected] Seo Ji-yoon Reporter