Real-time interpretation captures emotions conveyed in the voice...DeepL unveils voice AI model “Aure”

[Financial News] Language AI company DeepL has unveiled a model that interprets not only voices but also emotions and tone in real time.
DeepL said on the 11th (local time) that it would integrate its new AI model Aure, which has these capabilities, into its real-time interpretation software DeepL Voice.
Aure analyzes the other person’s voice in real time. The company said this takes communication to a new level by reflecting in the interpreted voice nonverbal cues such as intonation, emphasis, emotion and speaking rate, which change as a conversation progresses. DeepL’s existing voice interpretation technology had recognized speakers’ vocal characteristics based only on fixed voice samples.
To make this possible, DeepL combined techniques that separately train on a speaker’s vocal characteristics, prosody and spoken content with reinforcement learning.
It also adopted a method that directly converts the audio itself, instead of transcribing speech into text, translating it and then converting it back into speech.
It removes background noise and reverberation and minimizes distortions in intonation that can occur when speech is reproduced in another language.
DeepL also made the system suitable for use in security-conscious businesses by not storing meeting transcripts or translations generated using its voice interpretation feature and by deleting data from memory when a session ends.
In an interview with Yonhap News Agency about the new model Aure, CEO Jarek Kutylowski emphasized that it was “an important step toward a stage where a speaker’s unique characteristics are faithfully preserved even when the language changes.” He also said, “The voice is a key element of identity and plays an important role in expressing emotions and building relationships.”
At launch, Aure supports eight languages, including Korean, English, German, Spanish, French, Italian, Japanese and Portuguese. DeepL plans to expand the languages it supports over time. 

[email protected] Lee Seok-woo, International Affairs Correspondent Reporter