Bringing Real Objects into VR: Sony’s Patent for 3D-Scanning Technology

Bringing Real Objects into VR: Sony’s Patent for 3D-Scanning Technology

Person wearing virtual reality headset

Photo by JESHOOTS.COM (Unsplash)

Sally Yoon is an IPilogue Writer and a 2L JD Candidate at Osgoode Hall Law School.


Playing games in your room is one thing, but what about bringing your room into your game? Sony allegedly updated a patent for virtual reality (“VR”) technology that can scan real objects into virtual reality. As per GameRant, Sony appears to finally be moving forward with the patent process despite issues and resubmissions following their initial filing on June 23, 2021.

The technology appears to stem from structure from motion (SfM) photogrammetry, which uses “overlapping images to create a 3D point cloud of an object or landscape.” SfM photogrammetry is a concept that has already been widely used in the video game industry to add greater immersive elements. One of the first games that might pop into mind would be Microsoft Flight Simulator, which exhibits photogrammetry cities through a collaboration with Bing Maps. Using SfM photogrammetry in the game, players can virtually experience flying over photorealistic cities around the world.

Airplane flying over city

Photo from Fanbyte

The technology in Sony’s recent patent aims to bring this technique closer to home, literally. The 3D-scanning technology would enable users to scan real-life items into a VR space and interact with them, with the requirement that the user has a 360-degree view of the object for a successful scan. It’s currently unclear what Sony intends to do with this technology and whether it will eventually have any connection with the new PlayStation VR2, which is anticipated to release sometime this year. However, it does promise a greater level of user customization within the virtual reality space as well as signal interesting developments for both gaming and collaborations with other companies’ existing VR projects.

Technical drawings

Photo from GameRant

Over the years, VR technology has increasingly shown its potential to create extravagant, otherworldly environments, but also simulate the real world. VR has been a space for various subjects, from religious matters as well as history education. You may have stumbled across the viral video of a man using a VR headset for a week in 2019 where he engaged in various activities such as meditation and movie screenings. On top of already prominent developments from Meta (formerly known as Facebook) and Microsoft, and their investments in virtual reality, Sony’s patent has interesting future implications for a more immersive metaverse.

The 3D-scanning technology is also reminiscent of the company’s infamous “banana as a PS5 controller” patent last year, where the company introduced technology that would enable non-luminous objects (mostly simple, household objects) to be used as controllers. These patents suggest the company's desire to close the gap between players' real and virtual lives. After all, regardless of how impressive it is to replicate historical events or fly through picture-perfect cities, no space will ever be more realistic than your living room.