Virtual Reality News Roundup: VR Flakes April 15, 2023
Welcome to this week's issue of the VR Flakes Newsletter. Our goal is to deliver the best virtual reality news from the week, all in one place. Get this delivered to your inbox every Saturday along with a surprise inside. Subscribe below on mobile or on the right if you’re on a desktop.
This week's VR Flakes Newsletter is packed with exciting news from the virtual reality world. We've got exciting updates on new software, interesting projects, and how the industry is evolving. So keep reading to find out more about these exciting developments
Meta introduces the Segment Anything Model
The future of augmented reality just got a lot more exciting with the introduction of Meta's new Segment Anything Model. This technology aims to revolutionize computer-vision-based object segmentation, which is the ability for computers to understand the difference between individual objects in an image or video. With this capability, an AR system can have a more comprehensive understanding of the world around the user, making the overall experience much more useful.
Object segmentation is the process of identifying and separating objects in an image or video. With the help of AI, this process can be automated, making it possible to identify and isolate objects in real time. Having a reliable and fast segmentation system that can identify each object in the room around the user will be critical to unlocking tons of AR use cases.
One of the biggest challenges for object segmentation is having a large enough set of images for the systems to learn from. Meta's Segment Anything Model addresses this issue by providing a general segmentation model that can identify any object in any image or video, even for objects and image types that it didn't see during training. SAM allows for both automatic and interactive segmentation, giving users control over what the system is attempting to identify at any given moment.
Part of SAM's impressive abilities come from its training data, which contains a massive 10 million images and 1 billion identified object shapes. This is far more comprehensive than contemporary datasets, according to Meta, giving SAM much more experience in the learning process and enabling it to segment a broad range of objects.
The release of this massive training dataset will accelerate research into an image and video understanding. Meta hopes that the SAM model can be used as a component in larger systems, enabling versatile applications in areas like AR, content creation, scientific domains, and general AI systems. It's exciting to see how this technology will shape the future of AR and computer vision.
RizzGPT: real-time charisma powered by AR
The future is here, and it's full of innovative technological solutions such as RizzGPT. Developed by a group of student researchers from Stanford University, RizzGPT is the real-time Charisma as a Service (CaaS) that combines augmented reality (AR) and artificial intelligence (AI) technology to generate natural responses in real time.
The RizzGPT device is a pair of glasses that listens to your conversations using OpenAI's automatic speech recognition tool, Whisper. The device then generates responses using OpenAI's ChatGPT-4 chatbot software. These responses are layered over the user's real-world environment using Brilliant Labs' Brilliant Monocle, a hackable AR lens that can be clipped onto any pair of consumer glasses.
The result is a communication assistance device that can deliver knowledge on virtually any topic up until 2021 at a moment's notice. It's like having a knowledgeable friend in your ear at all times. As Bryan Hau-Ping Chiang, one of the members of the student research team, pointed out, the system could prove useful in a number of high-stress scenarios.
RizzGPT is a unique technological solution that has the potential to revolutionize the way we communicate and access information. It's exciting to see the potential of AR and AI technology combined in such an innovative way.
Instant VR: John Carmack’s vision
In the newest episode of Meta CTO Andrew Bosworth's podcast, John Carmack, a VR expert, and independent AI researcher shared his technical and strategic insights on virtual reality. Carmack had a decade-long run helping shape the direction of VR development at Oculus and advising executives at Facebook before formally leaving Meta in December to focus on developing Artificial General Intelligence. However, he still refers to himself as part of the effort, and Bosworth jokingly refers to him as a "permanent member" of the team.
During the podcast, Carmack discussed Meta's shortcomings in VR and provided advice to Bosworth. They covered new ground from previously published discussions, including more detail on the decision to shut down Echo VR and a balanced look at how the main features of Star Trek's Holodeck may be achievable in technology's future. Carmack even provided a breakdown of exactly what Neural Radiance Fields could offer VR, making the entire podcast worth a listen.
Carmack's vision of Instant VR is one of his most significant contributions to the VR world. It's the idea of using VR as a window into the real world, providing a seamless way to interact with others and our surroundings. The technology used for Instant VR will require high-resolution cameras and advanced software to process the images in real-time accurately. While the idea of Instant VR seems far off from our current technology, Carmack believes that it is a possibility in the near future.
John Carmack's expertise in VR and AI has been an invaluable asset to the development of VR technology. His continued involvement and advice will undoubtedly shape future advancements in VR technology and bring us closer to achieving his vision of Instant VR.
Smart contact lenses maker raises funds for Micro-LED Displays for XR
Smart contact lens maker, Mojo Vision, recently raised $22.4 million in a Series A investment round. The funds will be used to develop and commercialize micro-LED display technology for consumer, enterprise, and government applications. This comes after the company had to put its smart contact lenses on hold and downsize its workforce by 75%. Before this round, Mojo Vision had already amassed $205 million in outside investment, with its most recent in January 2022 bringing in $45 million.
Mojo Vision's new focus is on displays for AR/VR, automotive, light field, large format displays, and others that require high-performance micro-LED displays. The company is developing its own High-Performance Quantum Dot (HPQD) technology to make a "very small, very bright, very efficient RGB pixel". The company is boasting a number of advances in its proprietary technology, including dynamic displays with up to 28,000 ppi, efficient blue micro-LED devices at sub-μm scale, high-efficiency quantum dot ink for red and green, high brightness at 1M+ nits, and a display system that incorporates an optimized CMOS backplane, wafer-to-wafer bonding, and custom micro-lens optics.
Mojo Vision's prototype smart contacts made use of its own in-house displays, which at the time included a monochrome display capable of over 14,000 pixels per inch (ppi). Mojo Vision's new CEO, Dr. Nikhil Balram, is said to bring semiconductor and display technology expertise to the company.
Preventing wildfires with VR technology
Preventing wildfires is a crucial endeavor, and now there is a new tool in the fight against them: VR technology. Immersive technology is being used to train pilots thanks to companies like Intellic Technologies. This is just one example of how VR technology can be used to prevent and fight wildfires.
A new partnership between Becker Support Services and Anchor Flank & Pinch has formed a company called Intellic Technologies. This new venture is dedicated to providing various organizations with safe and efficient wildland fire training using immersive technology. They are using wildland fire simulation software and other training services to create a new way to train pilots.
The FS3D virtual wildland training hardware and software is a crucial part of this technology. It provides pilots with hyper-realistic training scenarios featuring VR and mixed reality technology, photo-realistic imagery, and scientific fire growth modeling. According to Intellic Technologies, the FS3D wildland fire training simulator is designed for aerial and ground support companies, government agencies, and U.S. ally wildfire organizations.
Thanks to these developments in VR technology, we may be able to prevent more wildfires in the future. By providing pilots with realistic training scenarios, they will be better equipped to handle challenging fire situations. This new technology from Intellic Technologies may be a game-changer in the fight against wildfires.
The new 3D display from Sony doesn't need AR/VR gear
Sony's latest addition to its Spatial Reality Display lineup, the ELF-SR2, is set to launch next month. The 27-inch 4K display promises an immersive 3D experience without the need for additional hardware like augmented reality glasses or a virtual reality headset. The device features an improved vision sensor and several image quality-enhancing technologies, making it suitable for use by architects, engineers, and software or game developers, among others.
What's more, the ELF-SR2 comes with facial tracking technology and a wide viewing angle to ensure consistent 3D content viewing from different vantage points. It is compatible with Unity and Unreal Engine SDKs and supports development using Open GL and DirectX11/12, with OpenXR support coming later this year.
To explore compatible applications and relevant use cases and studies, Sony has launched the new Spatial Reality Display App Select website. Users can also easily display their 3D projects using the Spatial Reality Display Player app.
The device is designed to be easily transportable and is compatible with several PCs. It retails for $5,000 and is available in the United States and Canada from May. Interested individuals can check it out in person at the upcoming NAB Show in Las Vegas, Nevada, or the Augmented World Expo (AWE) in Santa Clara, California, from May 31 to June 2. With this new device, Sony is taking a step forward in delivering an immersive and convenient 3D experience.
Introducing the Seamless Walk: A genius carpet that harnesses the power of pressure sensors and machine learning tech!
The future of virtual reality is looking brighter and more innovative every day. One exciting development comes in the form of the Seamless Walk system, which could potentially revolutionize the way we move in VR. This experimental foot-based movement system eliminates the need for additional wearables and body-tracking camera technologies, instead utilizing pressure sensors and machine learning technology embedded in an intelligent carpet. Developed by a research team based out of South Korea’s Gwangju Institute of Science and Technology, the Seamless Walk system offers a more natural in-headset experience while also being scalable and inexpensive.
The technology works by tracking the movement of your feet in real-time, measuring foot pressure inputs that are then fed into a machine learning model. This model estimates the angle of your body and your general movement speed, translating the data into realistic virtual movements. The research team tested the technology with a group of 80 participants and found that the Seamless Walk system "outperforms" existing VR locomotion systems.
In addition to gaming, the Seamless Walk system has potential applications in healthcare as well. The technology offers a new way for patients to receive physical therapy in a virtual environment, allowing for more accurate tracking of movements and progress. Moving forward, the research team plans to continue expanding the system with additional features and use cases, propelling virtual reality technology to new heights
Catch CS 2 matches in VR soon!
Catch CS 2 matches in VR soon! That's right, Counter-Strike fans, get ready to experience the iconic video game series in an entirely new way. Virtex, an ambitious virtual event platform, has partnered with Skybox Technologies to bring Valve's immensely-popular multiplayer first-person shooter series to Virtex Stadiums. This includes Counter-Strike: Global Offensive as well as the upcoming sequel, Counter-Strike 2.
Virtex Stadium promises to immerse fans in their favorite live esports events like never before. Powered by VR technology, custom 3D arenas create an immersive experience that puts users in the stands or directly on the battlefield where they can inspect every play from virtually any angle.
The partnership with Skybox Technologies means that later this year, certain users will be able to view select Counter-Strike events in VR. They can even access the lobby area, complete with a lounge and main hall, where users can chat with other esports fans, play games, and search for hidden interactions. Virtex has worked closely with Skybox to integrate maps, gameplay features, and map control options to offer the most flexible experience possible.
There's no official release date for Virtex Stadium yet, but interested users can join the waitlist for a chance to participate in the closed beta. Virtex hopes to kick things off later this year with specific Counter-Strike events. Get your VR headset or Windows PC ready for an immersive esports experience like no other.
Join the closed-Beta here.
VR game new releases and updates you should check out this weekend
1. Kill It With Fire VR
Released: April 13, 2023
Looking for a new way to deal with your fear of spiders? Look no further than the latest virtual reality game from Casey Donnellan Games. Kill It With Fire puts players in a first-person perspective as they embark on the ultimate spider-killing spree.
This unique experience is sure to delight gamers and arachnophobes alike, providing an exciting and therapeutic way to conquer your fear of spiders. With realistic graphics and immersive gameplay, you'll feel as though you're actually in the room with these creepy crawlies.
And the best part? You can do it all without any risk of harm. Let your inner exterminator run wild and take control of your fear with Kill It With Fire VR.
2. Call of the Sea VR
Released: April 13, 2023
Virtual reality enthusiasts have a new game to look forward to with Call of the Sea VR. Developed by Out of the Blue and published by Raw Fury, Call of the Sea VR relies on puzzle-solving over combat. It transports players back to the 1930s, telling the story of Norah Everhart, whose family suffers from a mysterious illness. You play as Norah, who is tasked with investigating her husband Harry’s disappearance on a South Pacific island near Otaheite (Tahiti) after he vanished while searching for a cure. Armed only with his photo and island coordinates to work with, Norah sets off on an “otherworldly tale of mystery and love.”
Initially launched for PC, Xbox, and PlayStation, Call of the Sea was first released in 2020. It was well-received by critics and players alike, earning a BAFTA nomination for Debut Game in 2021, and winning Best Immersive Game at the 2022 Raindance Immersive Awards. The VR adaptation of the game doesn’t appear to deviate much from the original flatscreen release, beyond expected additions like motion controls and 360-degree immersive environments. If you’re a fan of exploring immersive worlds and solving puzzles, make sure to add Call of the Sea VR to your must-play list.
3. Eye of the Temple
In development for Quest
Released by indie developer Rune Skovbo Johansen on SteamVR headsets in 2021, Eye of the Temple is a VR puzzle game unlike any other. The game's innovative locomotion style lets you explore a massive temple complex with your own two feet, as you solve puzzles and navigate moving platforms of all shapes and sizes.
What sets Eye of the Temple apart from other VR puzzle games is the game's use of physical space. Unlike other games where you move using a joystick, Eye of the Temple requires players to physically walk and jump within a 2x2m area to progress through the game.
This style of locomotion adds a level of immersion that is rarely seen in VR games, as players must use their own bodies to solve puzzles and traverse the temple complex.
Eye of the Temple has received critical acclaim for its unique take on VR gameplay and has become a fan favorite among puzzle game enthusiasts. If you're looking for a VR game that will truly transport you to another world, Eye of the Temple is definitely worth checking out.
4. Star Wars: Jedi Outcast
Team Beef's new mod
Star Wars Jedi Knight II: Jedi Outcast is getting a major upgrade thanks to a new mod that introduces motion-controlled lightsabers and gesture-based Force abilities. Thanks to a new port from Team Beef, players can now experience the iconic 2002 first-and-third-person adventure game in VR on Meta Quest and Pico 4 headsets. Built using OpenXR, the JK XR – Jedi Knight II: Outcast VR Port faithfully recreates the original Force-driven experience, complete with full motion-controlled lightsabers, gesture-based saber throws, collision-based laser deflections, and gestured-based Force abilities. Players can even access a range of new fully-modeled VR weapons to take down their enemies in style.
In addition to the fully-modeled VR weapons, the game also features VR weapon/Force wheels and gesture-based use/interact options. Players can use a variety of different gesture-based Force actions, including push, pull, and grab, to interact with the game world and take down any enemies that stand in their way. The game also has in-game menu tutorial videos and a cheat menu (hidden in the Datapad) to help players get the most out of their experience.
To make the game even more accessible for players, there is even a companion app available for streamlined mod download. Overall, this new mod is a must-have for any Star Wars fan looking to immerse themselves even further into the action-packed world of Jedi Outcast.
That's a wrap! This week's VR Flakes were a blast, and we hope they left you buzzing with excitement. Now it's your turn to join the conversation! What's your take on the latest VR news and updates? Share your ideas and feedback with us, and let's keep the VR family thriving. Get ready for even more mind-blowing VR content next week. Until then, stay curious and keep exploring! Ciao for now VR folks!
