The next generation of video games could be filled with AI-generated text, speech, characters, and background art.
What’s new: Nvidia announced a system that enables players to converse directly with in-game characters. Meanwhile, game developers are using generative AI to produce media assets, The New York Times reported.
How it works: Tech companies are providing software that generates game assets either in production or on the fly. Some large game studios are developing their own tools.
- At Computex 2023 in Taipei, Nvidia showed off a suite of tools called Avatar Cloud Engine (ACE). In the demo, a human player speaks to a game character that replies in real time with information that drives further gameplay. ACE interpreted the player, generated the character's words and voice, and drove the animation. Nvidia developed the software in collaboration with Convai.
- The startup Scenario offers a text-to-image generator with a specialized user interface for fine-tuning on a developer’s assets. Didimo offers a text-to-3D generator that outputs editable, animation-ready character models in developer-friendly formats.
- Blizzard Entertainment, producer of the popular Diablo, Overwatch, and World of Warcraft franchises, trained an image generator on assets from its own games. Developers use it to generate concept art for characters and environments.
- Ubisoft, whose titles include Assassin’s Creed and Far Cry, built a dialogue generator. Writers use it to create dialogue for in-game characters. Given a prompt like, “I used to be an adventurer like you,” the model generates variations such as “I remember when I was young and strong,” and “I was once the greatest explorer in the world.”
Behind the news: Gamers, too, are using generative AI to modify their favorite games. For instance, modders have used voice cloning to vocalize lines for the main character of “The Elder Scrolls V: Skyrim,” who otherwise is silent.
Why it matters: Generative AI tools can streamline video game production, which is bound to appeal to developers who aim to cut both costs and timelines. More exciting, it can supercharge their ability to explore art styles, characters, dialog, and other creative features that may not be practical in a conventional production pipeline.
We’re thinking: Given the high cost of media production, game development is ripe for disruption by generative AI. While we worry that some artists and writers may lose work, we expect that automating production will also create jobs. Big players are already using the technology to build more elaborate virtual worlds, and many smaller studios will benefit from lower production costs.