Artificial intelligence (AI) is poised to accelerate the creation and development of the metaverse, with multiple companies including Nvidia, Google, and OpenAI creating AI-powered 3D tools.
The public is already familiar with chatbots and AI-image generators, both of which can be leveraged to speed up development cycles for the metaverse. Text-to-3D generators, among other type of 3D generators, accelerate that process still further, and empower enthusiasts to channel their creative urges in new arenas.
The filling the metaverse problem
In the earliest days of the internet most users passively consumed the world wide web as they surfed from site to site. The emergence of social media empowered users to share their own content, and the internet experienced a cambrian explosion of information.
Filling the metaverse means learning the lessons of the early internet all over again argues Rev Lebaredian, VP for omniverse and simulation technology at Nvidia. This time, however, ordinary people will need greater levels of assistance to create the content.
“We’re able to fill the internet with interesting stuff because everybody is capable of taking a picture, recording a video, or writing words,” said Lebaredian on January 27. “If we are going to create a 3D internet, then you absolutely have to have the people who are participating in it creating content as well, and the only hope we have of making that happen is if AI can help us.”
Nvidia holds the belief that AI and the metaverse are intrinsically linked, and that both rely heavily upon each other.
“Just as AI is crucial to the expansion of the metaverse, so too is the metaverse to the expansion of AI,” explains Gavriel State, director of the system software team at NVIDIA.
“Digital twins in the metaverse are providing physically accurate virtual environments that allow developers to simulate and test AI,” says State before concluding, “Generative AI is key to scaling these digital twins and virtual environments that will usher in a new era of AI and the metaverse.”
The here and now
Sean Ellul, co-founder of the 3D development studio Metaverse Architects, is a big believer in transformative potential of AI-generated content.
In a recent interview with TIME Ellul revealed how his own firm has incorporated ChatGPT into a multitude of everyday tasks including brainstorming creative ideas, writing code and composing texts for decks and emails.
“I can’t stress how helpful this has been to us,” says Ellul who has gone all-in on AI, using it as regularly as most internet users use Google.
Ellul also takes the process one step further by prompting MidJourney to create floor plans for buildings which he can then use a template for 3D models. It’s a process which would undoubtedly be a little too complex for the average person, but it’s a powerful time-saving technique for Ellul and his team.
Case in point, the following floor plan was created from the Ellul prompt: “create a floor plan sketch with measurements and dimensions of a house plan built on one floor including two bedrooms three bathrooms kitchen and living and a garden. Include furnishing.”
Ellul also uses MidJourney to help him create virtual non-player characters (NPCs). In one example Ellul used 9 different female faces to create a composite image with MidJourney. The resulting image produced a fresh-faced nordic female with blonde hair.
Ellul then returns to ChatGPT to find out who this young woman is. Ellul prompts ChatGPT to “generate a backstory for a 24-year-old woman from a tiny village in a fantasy setting who works at an inn.”
ChatGPT suggests that the woman is named Elspeth, and that she, “worked as a farmhand, a blacksmith’s apprentice, and even as a barmaid in a bustling city town. But no matter where she went, she always felt like something was missing.”
Tools to make it happen
The challenge for AI is to transform a character such as Elspeth from a 2D image and chatbot-inspired biography into a fully realized character in three dimensions
The field is not short of competitors who are creating tools to make that dream reality.
In November Nvidia announced Magic3D, a high-resolution text-to-3D image generator. According to Nvidia the tool is capable of transforming a text prompt into a fully-realized 3D model in 40 minutes.
In December OpenAI released its own 3D image generator called Point-E. Google also has its own text-to-3D tool, DreamFusion.
At the moment there remain limits to what these tools can accomplish on their own with significant human intervention and expertise in the fields of 3D modeling. The question, therefore, is how long until that is no longer true.
According to Nvidia’s Lebaredian a future in which every person can create their own virtual worlds with the help of AI assistants is not too far away.
“I would say ten years from now, I’m fairly certain that most people are going to be able to create high quality 3D content just by talking to the computer,” says Lebaredian. “And I’m bullish that it’s going to happen much earlier.”
and then