Despite declarations of its demise, Meta’s new photorealistic avatars prove that the metaverse is not dead after all. In a ‘Metaverse interview’ on the Lex Fridman podcast, Mark Zuckerberg demonstrated Meta’s new lifelike Codec virtual reality avatars.
Developed by Meta’s metaverse arm, Reality Labs, since 2019, the avatars use AI scanning technology to build 3D models of the user’s face. The Codec avatars can capture subtle facial expressions used by humans to communicate, according to Fridman.
They give off an “incredible” sense of presence and intimacy. “This is really the most incredible thing I’ve ever seen. It felt like we were talking in person,” he said.
Zuckerberg’s metaverse comes to life
For last week’s podcast, Fridman and Zuckerberg used Meta Quest Pro VR headsets to interact with each other in a virtual space, or metaverse, represented by ultra-realistic 3D avatars of themselves. The tech relies on noise-canceling headphones to boost immersion.
“This gets to the core of what the vision around augmented reality is, of delivering a sense of presence no matter where you are in the world,” Zuckerberg, the Meta founder and CEO, said in the interview.
“Most communication is not actually through words, but with expressions—we tried to capture that with the more cartoon-designed avatars that we have, but there’s obviously a certain realism that comes with delivering the photorealistic experience,” he added.
Also read: Meta’s AI-Powered Ray-Ban Glasses Cause Stir on Social Media
While countless obituaries have been written about the metaverse, Zuckerberg’s new Codec avatars show that it is alive and well. Unlike Meta’s previous cartoonish, legless avatars that were widely criticized, the new avatars look so realistic they can be mistaken for real people.
They mark a significant step forward in the development of the metaverse and will likely be an important tool for people to interact in a much more immersive and natural way.
Here's my conversation with Mark Zuckerberg, his 3rd time on the podcast, but this time we talked in the Metaverse as photorealistic avatars. This was one of the most incredible experiences of my life. It really felt like we were talking in-person, but we were miles apart 🤯 It's… pic.twitter.com/Nu8a3iYWm0
— Lex Fridman (@lexfridman) September 28, 2023
During the interview, Fridman was so stunned by the realism that he was often left speechless. “This is just really incredible. I don’t know how to describe it with words. It really feels like we’re in the same room,” said the computer scientist.
Zuckerberg suggested that users would be able to customize their Codec avatars to be more expressive. He hopes people will use the photorealistic avatars for remote-work meetings, gaming, and social interactions to feel a greater sense of presence and connection.
Fridman said the technology represents the future of “meaningful” human communication on the Internet. However, Mark Zuckerberg explained that Meta is still working to make the scanning process easier, and the product will not be available for the next few years.
Zuckerberg in the metaverse
Meta’s realistic avatars lauded on social media
Meta’s early avatars for the metaverse were criticized for lacking realism and being cartoon-like. Last year, Zuckerberg’s selfie of his metaverse avatar behind a virtual version of the Eiffel Tower was mocked as archaic. The Meta CEO claimed the avatar was “basic.”
Horizon Worlds, Meta’s early gateway into the metaverse that allows people to connect online using avatars, has struggled to grow its user base. The platform had about 200,000 monthly users at the end of 2022, according to the Wall Street Journal.
Now, the company is combining AI with virtual and augmented reality to create a metaverse that is more real and immersive. Zuckerberg told Lex Fridman that the photorealistic avatars could help drive interest in Horizon Worlds and the metaverse in general.
“People were blown away by the experience, but we do get some feedback that people are saying that the avatars don’t feel as realistic in this environment,” he said. “I think something like this could make a very big difference.”
On social media, many people were impressed. Posting on X (formerly Twitter), designer and entrepreneur Jelle Prins said:
“Nine minutes into the Lex/Mark metaverse podcast, I completely forgot I was observing avatars.”
VoodooPC founder Rahul Sood said: “People bashing Mark for his push into the’metaverse.’ Apple calls it ‘spatial computing.’ This is actually super innovative and compelling. At the end of the day, Zuck has led technology shifts in a number of areas.”
The Metaverse has upgraded pic.twitter.com/QT1LAkjQGB
— Dexerto (@Dexerto) September 28, 2023
and then