Nvidia has made another attempt to give flat images more depth. The company has shifted its focus to editing after transforming 2D photographs into 3D scenes, models, and films.
The approach, dubbed 3D MoMa, could provide game developers with an easy way to change pictures and scenes. Photogrammetry, which captures measurements from images, is commonly used for this.
Through inverse rendering, 3D MoMa speeds up the process. By analyzing still photographs, this method employs AI to estimate a scene’s physical properties, such as geometry and illumination. After that, the images are reassembled in a realistic 3D format.
“The NVIDIA 3D MoMa rendering pipeline uses the machinery of modern AI and the raw computational horsepower of NVIDIA GPUs to quickly produce 3D objects that creators can import, edit, and extend without limitation in existing tools by formulating every piece of the inverse rendering problem as a GPU-accelerated differentiable component,” Lubeke said.
3D MoMa creates items as triangle meshes, a format that’s easy to alter using common software. On a single NVIDIA Tensor Core GPU, the models are built in under an hour
The Computer Vision and Pattern Recognition Conference (CVPR) in New Orleans this week featured 3D MoMa. Nvidia engineers utilized the method to graphically represent jazz in honor to the musical genre’s roots. Hundreds of photographs of trumpets, trombones, saxophones, drums, and clarinets were originally gathered. The photos were then rebuilt into 3D representations via 3D MoMa. After that, the instruments were reworked and given new materials. For example, a cheap plastic trumpet was converted into a magnificent gold trumpet. From the brass instruments reflecting brightly to the drum skins absorbing light, the company claims that all of the instruments reacted to light like they would in the real world. Finally, an animated scene was created using the 3D objects.
Although 3D MoMa is still in the works, Nvidia believes it will allow game developers and other designers to swiftly change 3D objects and then place them in any virtual setting.
and then