Detecting AI manipulation in video, audio and text... Police upgrade detection and analysis systems

[Financial News] The police are moving to establish a detection system that analyzes not only video but also audio and text to combat crimes involving the misuse of artificial intelligence (AI), including deepfakes. Their plan is to strengthen investigative capabilities so they can trace the criminal process from creation to distribution, rather than simply identify content manipulated with AI.
The National Office of Investigation of the Korean National Police Agency announced on the 15th that it had drawn up and would implement a “Basic Plan for Enhancing the Detection and Analysis System for Illegal AI-Manipulated Content.”
The plan was drawn up in response to the growing range of crimes involving AI, from deepfake sexual crimes to defamation, investment fraud and voice phishing. The police plan to improve technology for detecting manipulated content and their ability to analyze the process behind crimes, while also securing specialized personnel and refining analysis procedures.
The police will introduce “multimodal” technology that analyzes audio and text alongside video, improving on existing deepfake detection methods centered on video and images. The approach will determine whether content has been manipulated by examining factors such as whether a person’s lip movements in a video match the audio and whether the audio differs from the subtitles.
The police have been using a deepfake detection system since 2024 and are improving its performance by applying training data tailored to the characteristics of Koreans. In addition, they plan to secure multimodal detection technology through a research and development project worth 9.1 billion won over three years, from 2025 to 2027. Once the research and development is complete, they will also pursue the development of a system that investigative agencies, including the police, can use jointly.
The police will also strengthen digital forensics technology to verify how AI-manipulated content is created and distributed. They will develop analytical techniques to establish how crimes were carried out, including by examining program execution histories and command logs from major generative AI services and identifying programs used to manipulate content.
The police have already developed analytical techniques for major generative AI services and are distributing related manuals. Going forward, they plan to develop automated tools to analyze the entire process, from the creation of illegal manipulated content to its distribution, and extract evidence. To this end, they are conducting an AI forensics technology research project worth 10 billion won over four years, from this year through 2029.
The police will also reorganize their specialized workforce and organizational operating system. They plan to assign and operate AI specialist analysts responsible for analyzing manipulated content at police agencies across the country’s cities and provinces, and expand specialized training for field investigators and analysts.
They will also build datasets suited to the domestic environment and establish standard procedures for detection and analysis. They plan to operate a cooperation system involving experts from the private sector, government and academia to share the latest technology trends and analysis cases, and to pursue international-standard certification in AI forensics.
Through a total of eight detailed tasks—four in technology and systems and four in personnel and organization—the police aim to strengthen their ability to respond to crimes involving the misuse of AI.
Acting Commissioner General of the Korean National Police Agency Kim Jong-cheol emphasized, “As methods of committing crimes using AI are changing rapidly, the police response must change as well. The police will not only quickly identify fakes created with AI, but also develop advanced investigative capabilities to effectively trace how they were made and distributed, so that the public can use AI technology with confidence.”
[email protected] Kim Ye-ji Reporter