For as long as I can remember, photorealism has been a strongly sought after milestone in the game. I guess it is one of those things that we humans feel we must achieve. A research team at Intel Intelligent Systems Labs has developed an AI that remixes the art style of games into something closer to photorealism.

Called "photorealism enhancement," it uses convolutional neural networks to analyze frames and modify them to appear more true to reality.

According to researchers Stefan R. Richter, Hassan Abu Alhaija, and Vladlen Koltun, "a perfect test case" GTA 5. Check out the results in the video below.

The results are a bit mixed. From a gamer's perspective, it looks gray, green, and kind of bleak; it has the feel of an early 2000s FPS, with saturation sucked out of the picture. From a research perspective, however, this is clearly a huge success, and I think that's what we should focus on here.

ML techniques are a way to graft photorealism onto an unrendered image. This requires fully raytraced scenes and high workloads that only major animation studios can handle on a frame-by-frame basis.

Researchers can also mix and match results by using different training data sets. While the "cityscapes" dataset shown in the video may look rather bleak, the scene highlighted by the "Mapillary Vistas" dataset is much brighter and more colorful.

Clearly, there is still work to be done to get this technology off the ground and make it actually useful to gamers, or any media for that matter, and Intel Labs researchers believe that porting this extension to a game engine could easily yield deeper, more realistic results. They believe that it would be possible to easily achieve deeper and more realistic results.

Nevertheless, to get close to playable frame times, the process needs to be sped up through optimization. Today, it takes 0.5 seconds on an RTX 3090 to infer photorealism enhancement. Ouch.