Metaverse January 19, 2023

WiFi New Solution For Metaverse Body Tracking

Published

4 months ago

January 19, 2023

WiFi New Solution For Metaverse Body Tracking

You can run, but you can't hide. From wifi.

Full body tracking in the metaverse may now be possible using only WiFi signals, according to new a study by three Carnegie Mellon University researchers.

Jiaqi Geng, Dong Huang and Fernando De la Torre found that “common WiFi antennas [or 1D sensors] can be used as the sole source of active sensing to track fine human movements in a room.”

Also read: SHIB The Metaverse at Texas Film Festival: Here’s What You Can Expect

Mapping the human body

The researchers developed what they are calling a “deep neural network” that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. They used three WiFi transmitters and three receiver antennas to assess several people.

Results of the 13-page study, published in December, show that “our model can estimate the dense pose of multiple subjects by utilizing WiFi signals as the only input.”

“This paves the way for low-cost, broadly accessible, and privacy-preserving algorithms for human sensing. We believe that WiFi signals can serve as a ubiquitous substitute for RGB images for human sensing in certain instances,” it said.

UV coordinates are used in the process of projecting a 2-dimensional image, or 2D, onto the surface of a 3D model to create texture mapping. Dense pose is designed to map all human pixels of an RGB image to a 3D surface of the human body, according to experts.

RGB stands for the primary colors red, green and blue. It typically refers to three hues of light that can be mixed together to create different colors. A combination of the three colors is the standard method of producing color images on TVs, computer monitors, and smartphones.

Expanding human tracking research

The first row illustrates the hardware setup. The second and third rows are the clips of amplitude and phase of the input WiFi signal. The fourth row contains the dense pose estimation of the algorithm from only the WiFi signal.

The Carnegie Mellon University study expands on the use of WiFi signals in combination with so-called “deep learning architectures”, commonly used in computer vision, to estimate dense human pose correspondence.

There is already a great deal of progress made in this area using 2D and 3D sensors over the last few years, fueled by applications in autonomous driving and augmented reality (AR). It includes uses linked to RGB sensors and radars.

However, traditional sensors are criticized for being too pricey for the ordinary everyday user. For example, a simple radar detector costs between $200 – $600. They are blamed for using too much power and have been flagged as a threat to user privacy in non-public spaces.

A lot of people would not install the tech say in their home bathrooms and other such private areas. For RGB cameras, narrow field of view and poor lighting conditions, such as glare and darkness, can have a severe impact on camera-based approaches, said the study.

It also mentioned occlusion as another obstacle that prevents the camera-based model from generating reasonable pose predictions in images. This is “especially worrisome for indoors scenarios, where furniture typically occludes people,” it adds.

WiFi body tracking protects user privacy, say researchers

According to the researchers, illumination and occlusion have little effect on WiFi-based solutions used for interior monitoring. The equipment required for this sort of tech is cheaper and can protect user privacy, they claimed.

“Compared to video or lidar, WiFi offers a better privacy-preserving signal,” said Fernando De La Torre, a co-author of the study who researches Computer Vision and Machine Learning at Carnegie Mellon University. He was speaking to AiBreakfast in a recent interview.

“This is critical in applications such as monitoring the well-being of elderly people at home (e.g., detecting falls, computing the amount of social interaction, or detecting potential health concerns), where many users will feel uncomfortable using cameras,” he said, adding:

“WiFi adds an additional layer of anonymity because it cannot be immediately interpreted by humans and it can only recover shape information rather than texture.”

Body tracking in the metaverse

Tracking your body movements in the metaverse is a big deal. It helps to make an otherwise virtual experience feel real. Companies such as Sony have been at the forefront of trying to bring this sense of reality to the metaverse.

In November 2022, the Japanese electronics company launched “Mocopi“, a device that allows users to translate their body movements onto a metaverse avatar. The unit is made up of six motion-tracking bands worn on your hands, feet, back and head.

Worth around $350, Mocopi is a play on the phrase “mocap”, meaning motion picture. It is designed to “track your body to create videos or operate avatars in real time with metaverse apps like VRChat.” It offers tools that let users “import motion data into 3D animation apps.”

“Normally, video production using motion capture requires dedicated equipment and operators,” said Sony in a statement at the time.

“By utilizing our proprietary algorithm, ‘Mocopi’ realizes highly accurate motion measurement with a small number of sensors, freeing VTubers [virtual YouTubers] and creators involved in movie and animation production from time and place constraints.”

Downside of WiFi human tracking

The Carnegie Mellon University WiFi-based full body tracking tool could hold some promise for future use in the metaverse. However, the current version is “trained for our scenario and specific hardware,” said Fernando De La Torre.

There is also a few downsides to the tech. Positioning for the four separate devices should be impeccable. It may also interfere with other WiFi networks.

“It is important to keep in mind that a number of variables, like the position and orientation of WiFi devices, the presence of objects, and the movement of people and things in the environment, can alter the WiFi signals,” De La Torre explained.

“Therefore, more research is required to make the approach robust to these factors, before releasing the code/model.”

Related Topics:AR Carnegie Mellon University Dong Huang Fernando De la Torre Jiaqi Geng metaverse Mocap Mocopi RGB images Sony WiFi Body tracking

Jeffrey Gogo Image credits: Shutterstock, CC images, Midjourney, Unsplash.

Up Next

People Who Struggle With Identity Find the Metaverse a Safer Place to Be Themselves

Don't Miss

Stability AI Sued for Allegedly Stealing Getty Images Photos

AI

Metaverse Experiences Must Be Worth Having, Says Stephenson

Published

5 hours ago

June 2, 2023

Robert Knight

Metaverse Creator Stephenson Opens Up On It 30 Years Later

The success of the metaverse depends on the ability of developers to build quality experiences that “millions or billions” want to have. To do that the sector must attract, find, and financially incentivize the very top talent from related industries.

This is the verdict of Neal Stephenson, the man credited with coining the word metaverse in his 1992 novel Snow Crash.

Source the best developers

Famed author and futurist Neal Stephenson says the metaverse must find and attract the most talented people to make the sector a success. Stephenson’s comments came during an appearance at AWE USA 2023 convention on Wednesday.

“If we’re going to have a metaverse that’s being used all the time by millions or billions of people, then there have to be experiences in the metaverse that are worth having,” Stephenson said.

“That seems like an obvious statement but for me, there’s a glaring and frustrating lack of support for the kinds of people who make those experiences,” added the author. “Right now the skill set that is needed to create the metaverse is basically what you see in the game industry. People who know how to use game engines and how to create the assets that feed into those game engines. Those people by and large have jobs and other things they could be doing.

“We need to create the economic basis for [developers] to get rewarded if they succeed in creating metaverse experiences that a lot of people enjoy.”

Stephenson cited a number of ways that developers may be rewarded, but his personal vision is for a tokenized metaverse owned and controlled by its citizens.

In June last year, Stephenson announced Lamina1, a layer 1 blockchain infrastructure and open metaverse company. Stephenson co-founded the “batteries-included blockchain” with Peter Vessenes, reflecting their vision for an incentivized metaverse that, according to its website, could “empower a new generation of interconnected, immersive experiences across gaming, entertainment, fashion, music, and beyond.”

Metaverse Experiences Must Be Worth Having, Says Stephenson

Seeing double: Ori Inbar and Ori Inbar introduce Neal Stephenson

A tale of two metaverses

Ori Inbar, the CEO of AWE, hosted the conversation with Stephenson on what marked the opening of the 14th annual AWE convention. The latest event is running from May 31 to June 2 in Santa Clara, California. Those who can’t attend in person are invited to participate online.

In an entertaining introduction, a virtual facsimile of Inbar initially addressed conference attendees, only for the real Inbar to interrupt and reveal the entire monologue was written by ChatGPT.

Inbar then asserted that AI makes “original voices… even more valuable than before.”

Once Inbar sat down with Stephenson the pair discussed just how far technology is developing across multiple fields. Inbar asked Stephenson where he believed the metaverse was headed; to something dystopian or something utopian.

“I think it would be very early and premature to make announcements today about whether it is going in one particular direction,” said Stephenson.

To Stephenson, both the positives and the negatives of the metaverse can co-exist within just one reality – something he explored in his 1992 novel.

“Our initial exposure to the metaverse is a kind of very vast market, a lowest common denominator to include … the worst of television,” said Stephenson said as he described the surface level of the metaverse. “But later on, as we get farther into the book, we see that people have used it to make beautiful works of art. There are some people … who lavished a lot of time and attention on making homes in the metaverse that are exquisite works of art, both visually and in this sonic environment.”

That ambition of a deeper metaverse is worth striving for, especially as a driver of its long-term success.

Featured

Mark Zuckerberg Unveils $500 Meta Quest 3 VR Headset

Published

7 hours ago

June 2, 2023

Jeffrey Gogo

Mark Zuckerberg Unveils $500 Meta Quest 3 VR Headset

Mark Zuckerberg has announced the new Meta Quest 3 virtual reality (VR) and mixed reality (MR) headset. The new version of the Quest will feature a higher-resolution display, a faster processor, and improved tracking, the Meta founder and CEO claimed.

In a blog post on Thursday, Meta revealed the headset also features a new design that is 40% thinner and more comfortable to wear compared to Quest 2, its predecessor. Quest 3 will be released later this year in over 20 countries, including the U.S., Canada, UK and Australia.

The 128GB headset starts at a price of $499.99 and users have the option to pay for more storage space. However, a lot more detail about the headset was left unannounced, including the full specifications of the chipset that will power it.

Zuckerberg said more information about the new Quest would be revealed at Meta’s “Connect” conference on Sept. 27. “Quest 3 will be the best way to experience MR and virtual reality in a standalone device. It’ll be compatible with Quest 2’s entire library…” he wrote on Instagram.

Virtual reality market heats up

The launch of Meta’s Quest 3 comes at a time when the virtual reality market is starting to heat up. Apple is reportedly working on its own headset that is expected to be launched on Monday during the company’s Worldwide Developers Conference.

Also read: 3 Top Headsets to Watch: Samsung XR Glasses, Meta Quest Pro, Apple Reality Pro

The headset is rumored to be similar to the Meta Quest 3 in terms of its focus on graphics and comfort. According to industry reports, it will be used for gaming and social features. Other companies such as Sony and Magic Leap, are also developing new virtual reality products.

Earlier this week, PC maker Lenovo announced that its enterprise-focused ThinkReality VRX headset had gone on sale with a starting price of $1,299.

Meta did not discuss Apple’s imminent headset launch. But the timing of Mark Zuckerberg’s reveal suggests it may have been scheduled to beat Apple to market. For a long time, Meta, formerly Facebook, has looked to VR and augmented reality as the next big thing in tech.

It is all part of Zuckerberg’s grand plan to build the metaverse, a virtual world where people can work, play, and socialize. Quest 3 will be a key part of that strategy. The headset is the “first” that can provide both virtual reality and mixed reality experiences in a single device.

This means users can fully immerse themselves in virtual worlds, or they can overlay virtual objects onto the real world.

“Ultimately, our vision is to…[go] beyond the rigid classifications of ‘virtual reality’ and ‘mixed reality’ to deliver truly next-gen experiences that let you effortlessly blend the physical and virtual worlds,” said Mark Rankin, Meta’s vice president of virtual reality.

“Meta Reality gives you both the deep, immersive magic of VR and the freedom and delight of making your physical world more fun and useful with MR. We’re excited to see what developers and creators can build on the Quest Platform…”

The Quest 3 upgrade

According to Meta’s blog post, the new version of the Quest is a significant upgrade over its predecessor, the Meta Quest 2. It features a higher-resolution display than Quest 2, meaning that images and videos will look sharper and more detailed.

The Quest 3 is powered by a Qualcomm Snapdragon processor, which Meta claims to be significantly faster and more powerful than the processor in Quest 2. Meta said the new chipset will allow for more immersive and realistic virtual realty experiences.

With a battery life of up to three hours, the headset comes with new controllers to help with how people can use it more effectively and efficiently. The new system is fully capable of AR gaming and users can still access over 500 VR games, apps, and experiences from Quest 2.

Meta said the Quest 2, which has been well received in the market since launch in 2020, will see its price drop to $299.99 from $400 starting June 4. The product will also receive software updates to boost performance and speed.

“Quest 2 remains our most affordable entry point to VR and [Quest] Pro is optimized for work use cases, with face and eye tracking for more authentic self-expression in meetings,” the firm said in its announcement.

“We’ll continue to sell Quest 2 and Pro alongside Quest 3, and we’ll keep shipping new software updates to make the Quest experience even better.”

Business

Metaverse Gaming Market Expected to Reach $119.2 Billion by 2028

Published

1 day ago

June 1, 2023

Umesh Dahal

Metaverse Gaming Market Expected to Reach $119.2 Billion by 2028

The metaverse gaming market is estimated to encompass $22.7 billion in 2023 and projected to reach $119.2 billion by 2028, according to a recent report from ReportLinker.

The metaverse has been a hot cake in the tech industry in recent years and was boosted by Mark Zuckerberg’s decision to change Facebook’s name to Meta. However, the market has been limping towards AI, which stole the spotlight from virtual reality.

“The global metaverse in gaming market size is estimated at USD 22.7 billion in 2023 and is projected to reach USD 119.2 billion by 2028 at a Compound Annual Growth Rate (CAGR) of 39.3%,” stated the report.

The growth of the metaverse in the gaming market is expected to be fuelled by several significant factors, including the dynamic and evolving landscape of adjacent technology markets “such as extended reality (XR),” which encompasses “virtual reality (VR),” augmented reality (AR), and mixed reality (MR).

Expectation of rapid growth

In 2021, the gaming industry experienced rapid growth, with billions of people playing video games globally and generating over $193 billion in revenue.

Gaming companies quickly became early adopters in exploring the potential of the metaverse. Looking ahead to 2023, it was projected that the metaverse will continue to reshape the gaming landscape.

A survey from last year shows that about 52% of U.S. gamers believe the metaverse will change the game industry.

“According to the survey, just over half (52%) of gamers believe the metaverse will change the video game industry and a plurality (41%) think that the metaverse will have a positive impact on the industry (vs. 25% who disagree),” reads the survey report of Globant and polling firm YouGov.

Moreover, 40% say the buzz around “metaverse gaming is warranted,” though nearly “one-third (30%) were undecided” on that subject.

Who are the big players?

The metaverse is not just a single platform, virtual experience, or game; it is an entire world recreated to provide an immersive experience. Gaming is one of the many experiences in the metaverse that is powered by AI, VR, and AR.

Whenever it comes to gaming, some of the giant games like Fornite, Unreal, and Roblox come to mind.

And those are expected to be significant players in the metaverse due to their existing influence and capabilities in the gaming industry, as a report from 2022 states.

“As gaming platforms like Fortnight gain functionality and evolve into technologically advanced social meeting places, it becomes more likely that a functioning Metaverse, with an independent economy, systems, and processes is in our future,” reads the report.

Fortnite has transformed into a social meeting place, offering interactive events and branded experiences.

Unreal’s powerful engine enables immersive media experiences, while Roblox’s user-generated content and virtual currency have attracted millions of users. These factors position them to thrive in the evolving metaverse landscape.

Europe expecting significant growth

Europe is expected to witness significant growth in the gaming metaverse market, with the second-highest CAGR during the calculation period.

The UK, Germany, and France lead the way in technology investment, while Russia and Spain are also adopting new display technologies.

“The substantial growth of the virtual world immersive interactive gaming industry in Europe is a crucial driver for the gaming metaverse market in this region,” stated the report.

The immersive interactive gaming industry in Europe, along with the demand for AR, VR, and MR technologies in the entertainment sector, serves as a driving force, indicated ReportLinker.

@riseofaitech are now entering the Metaverse and the Play2Earn gaming space which, according to a 2021 Bloomberg report, is valued at $500bn with huge growth predictions. @sanboy23 @Iamchike2 @nainiydd #NFT #Crypto #P2E $AITECH pic.twitter.com/uMzyD2aSqL

— 오라클🔺 (@hoonjitw) June 1, 2023

Initiatives such as the European Association for Virtual Reality and Augmented Reality (EuroVR) and projects like Augmented Heritage and International Augmented Med (I AM) contribute to market growth.

With increasing startups in extended reality, particularly in Sweden, Europe is poised for increased growth in the gaming metaverse markets.