Applying AI to augmented and virtual reality is not a novel concept, but its application has had certain limitations. However, the Stable Diffusion image generation software boils down the algorithm to be usable on consumer-level hardware and is released under the Creative ML OpenRAIL-M license. This means that developers can not only create and launch programs using this technology without having to rent vast amounts of server silicon, but they are also free to profit from their creations.
ScottieFoxTTV (opens in new tab) is one of the creators who has been showing off his algorithmic creations in VR on Twitter. Says he, "I was woken up in the middle of the night to conceptualize this project." As one of the creators myself, I understand that the muse enjoys throwing fits at ridiculous hours.
What they have brought him is a fusion of Stable Diffusion VR and the TouchDesigner (opens in new tab) app building engine, the result of which he calls "real-time immersive potential spaces." It may sound like hippie nonsense to some, but latent space is a concept that is captivating the world right now.
At a basic level, in this context, the term describes the swelling potential that artificial intelligence brings to augmented reality, drawing ideas from the vastness of the unknown. While an interesting concept, this is for a later feature. Right now, I am interested in how Stable Diffusion VR can make any consumer GPU (even the recent RTX 4090 (open in new tab)) work so well in real time without turning it into a burning puddle It is.
"By diffusing small fragments into the environment, you save resources," Scotty explains. The small clips are sent from the engine to be diffused. When they are ready, they are queued back into the projection". The blue boxes in the image here show the portion of the image that the algorithm processes at a time. This is a more efficient method that works in real time.
If you have ever used an online image generator, you understand that it can take a minute to create a single image. And while not quite reaching the level of photorealism that may someday be achieved, the videos that Scotty posts are utterly breathtaking.
Flying fish in the living room, ever-changing interior design ideas, lush forests and nightscapes evolving before our eyes: now that AI can be projected onto our physical world in real time, there is great potential for its use in the gaming space.
David Holz, CEO of Midjourney, has spoken of the possibility of gaming one day becoming a "dream" (open in new tab), and it certainly feels like we are moving in that direction at a rapid pace. But an important next step is navigating the minefield of copyright and data protection issues (opens in new tab) that arise around datasets trained by algorithms like Stable Diffusion.
Comments