Encountering a film or series in a foreign language might lead to a few choices for viewers. One can activate subtitles to keep track of the dialogue. Alternatively, there’s the option to use an audio track in a preferred language, referred to as a “dub.” This involves talented voice actors re-recording lines to capture the essence of the original performance.

However, this process can be costly and time-intensive, often deterring studios from providing a dub for every title. But what if every show or movie came with this option?

On Wednesday, Amazon unveiled its “AI-assisted dubbing” feature for Prime Video, which allows users to select AI-generated dubs in either English or Latin American Spanish for a limited selection of titles. Currently, Amazon has indicated that this feature is available for 12 titles, only explicitly naming three: El Cid: La Leyenda, Mi Mamá Lora, and Long Lost. The company assures that “localization experts” collaborate with AI to maintain quality control, which presumably means that human oversight is involved to minimize errors caused by the technology’s limitations.

With access to Prime Video, curiosity led to a firsthand exploration of these AI-generated dubs, comparing their effectiveness against traditional dubbing methods. While unable to locate Mi Mamá Lora, both El Cid: La Leyenda and Long Lost were readily available. To experiment with this feature, the language settings must be changed to “AI beta” in compatible titles.

El Cid: La Leyenda

Several documentaries under the title El Cid exist on Prime Video, yet only this particular one fits the AI dubbing category. It’s a 63-minute documentary delving into Rodrigo Díaz de Vivar, often known as El Cid. My intention was not so much to learn about this historic figure as to evaluate how well an AI system would manage English dubbing for a Spanish-language production.

The outcomes are intriguing, to say the least. The majority of the spoken content consists of voiceover narration, which lands anywhere from mediocre to poor. Recognizing the AI voice leads to detecting its bland and sometimes odd tones. Typical of AI-generated speech, the voice changes pitch and cadence with mixed results; certain words are either slurred or artificially slurred, giving the impression of a tipsy narrator. Yet, surprisingly, the result surpassed my negative expectations. While it’s not akin to enjoying a documentary read by an engaging human voice, it did evoke a modest amount of emotion.

Nevertheless, inconsistencies plague the AI dubbing of the interview subjects in the documentary. The voices often sound stiff, detached, and lacking emotional range, reminiscent of older voiceover technologies rather than contemporary AI advancements.

Despite these shortcomings, Amazon’s approach does feature varied voices, making it easier to differentiate speakers and avoiding a monotonous experience for the audience.

Long Lost

Long Lost presents a more complex evaluation: this film, created in English, features an AI dub in Spanish. Without Spanish proficiency, recognizing the quirks associated with AI becomes challenging.

Nevertheless, experiencing Long Lost provided insight into how AI translates performances by actors. Surprisingly, these AI-generated voices deliver their lines with a degree of emotion that defies expectations. Although it’s clear that the system is not perfect—several lines can come off as abrupt or overly loud—there are moments when the delivery feels natural enough that one might not immediately suspect AI involvement.

During heightened scenes, the AI endeavored to convey emotion; when a character shouted, the pitch of the voice would rise. While not flawless, it is a fascinating display of what current AI capabilities can accomplish.

One amusing instance occurs when characters participate in a “Chubby Bunny” challenge, resulting in the AI sounding oddly muffled—an entertaining quirk that noticeably enhances authenticity. At other moments when characters have their mouths filled with marshmallows, the AI voice sounds completely normal.

An unexpected moment arises when a character shifts their speech to French, causing the AI dubbing to momentarily cut out, reverting to the original audio. This abrupt change highlights a stark contrast between the sterile sound of AI output and the more organic quality of the film’s natural audio.

Future exploration of this feature is desired on a film created in Spanish with an AI-generated English dub; greater language comprehension would likely make it easier to assess the AI’s accuracy and voice quality.

Is AI the Future of Dubbing?

It’s unlikely that watching an entire film or series with an AI dub would be a pleasurable experience. A fondness for human-done dubbing is already limited, making robotic-sounding replacements less desirable.

Despite this, the technology demonstrates promise. At times, the AI voices convey enough realism to convey essential information, particularly in a voiceover context typical of documentaries. Moreover, the incorporation of contextual elements, like altered speech during certain scenes, suggests a level of innovation.

It is essential for voice actors to remain employed and compensated for their creative contributions, as audiences deserve high-quality dubbing for their viewing experiences. However, AI dubbing might serve as a practical solution in circumstances where traditional dubbing seems unfeasible. If studios aren’t inclined to invest in dubbing options, the availability of an AI option could still enhance accessibility, especially for visually impaired viewers who rely on audio content.

Yet, it’s naive to assume that studios will limit this technology to specific applications, as the risk of replacing human voices entirely for financial gains is imminent. Thus, it might be wise to halt further developments in AI dubbing now. The novelty is evident, but the quality does not justify replacing skilled voice actors.