Quick Buzz Feed

Disney Patents Real-Time AI Technology That Allows Meet and Greet Characters Talk to Guests - WDW News Today

Gary Lloyd | Nov 03,26 | 01:44 EST

Entertainment

Disney has patented an AI technology that could allow characters to speak and respond to guests with real-time dialogue.

AI Character Speech Titled “Artificial Intelligence Based Character-Specific Speech Generation,” the patent was first filed in May 2024 but was granted and published now. In most cases, Disney’s full-body “fur” characters like Mickey and Minnie do not speak during meet and greets. Disney has experimented with having these characters recite pre-recorded phrases along with moving mouths. For five years, Mickey spoke during his Magic Kingdom meet and greet. But this ended in 2018. Mickey and other characters continue to “speak” in stage shows with similarly pre-recorded dialogue. As Disney states in the patent, this pre-recorded speech lacks spontaneity and immersiveness. With the AI technology described, they hope to create more realistic experiences with dynamic dialogue responsive to the emotions of the performer and guest. The character costume would be equipped with sensors and microphones to detect what the guests are saying and doing. An inward-facing “selfie” camera and/or eye tracker would read the facial expression of the performer. A database would store aspects of the character’s voice, for example their unique phonation, pitch, loudness, rate, dialect, accent, rhythm, inflection, etc. It would also feature elements of their personality, archetype (such as hero, villain, or sidekick), favorite words, and catchphrases. A Large Language Model (LLM) would take in the information from the microphones, sensors, and cameras, access the database, and instantly write a response to the guest in the character’s style. The AI would then recite the dialogue in the character’s voice, which would play through a speaker hidden in the costume. Actuators would adjust the expression of the character’s face to match the dialogue. Optionally, the database could include a history of the guest’s past interactions with that character, but would not store personally identifiable information like age, gender, race, ethnicity, etc. Previously, Disney filed two patents regarding AI characters that could form emotional bonds with guests using their memories. Guests can currently meet talking Dug at Disney’s Animal Kingdom. He has pre-recorded phrases like the old Mickey meet and greet, but his mouth does not move, which is in line with his portrayal in Up. Though the patent has been granted, this does not guarantee Disney will introduce the technology to Disney Parks and we don’t have a timeline for when this could potentially happen.

AI Character Speech

Titled “Artificial Intelligence Based Character-Specific Speech Generation,” the patent was first filed in May 2024 but was granted and published now. In most cases, Disney’s full-body “fur” characters like Mickey and Minnie do not speak during meet and greets. Disney has experimented with having these characters recite pre-recorded phrases along with moving mouths. For five years, Mickey spoke during his Magic Kingdom meet and greet. But this ended in 2018. Mickey and other characters continue to “speak” in stage shows with similarly pre-recorded dialogue. As Disney states in the patent, this pre-recorded speech lacks spontaneity and immersiveness. With the AI technology described, they hope to create more realistic experiences with dynamic dialogue responsive to the emotions of the performer and guest. The character costume would be equipped with sensors and microphones to detect what the guests are saying and doing. An inward-facing “selfie” camera and/or eye tracker would read the facial expression of the performer. A database would store aspects of the character’s voice, for example their unique phonation, pitch, loudness, rate, dialect, accent, rhythm, inflection, etc. It would also feature elements of their personality, archetype (such as hero, villain, or sidekick), favorite words, and catchphrases. A Large Language Model (LLM) would take in the information from the microphones, sensors, and cameras, access the database, and instantly write a response to the guest in the character’s style. The AI would then recite the dialogue in the character’s voice, which would play through a speaker hidden in the costume. Actuators would adjust the expression of the character’s face to match the dialogue. Optionally, the database could include a history of the guest’s past interactions with that character, but would not store personally identifiable information like age, gender, race, ethnicity, etc. Previously, Disney filed two patents regarding AI characters that could form emotional bonds with guests using their memories. Guests can currently meet talking Dug at Disney’s Animal Kingdom. He has pre-recorded phrases like the old Mickey meet and greet, but his mouth does not move, which is in line with his portrayal in Up. Though the patent has been granted, this does not guarantee Disney will introduce the technology to Disney Parks and we don’t have a timeline for when this could potentially happen.