This is a cool approach to improving the speed of training and rendering but seems to come with some limitations when it comes to representing reflections.

It reminds me of some of the experiments and approaches used for the custom rendering approach used by the game Dreams:

https://www.mediamolecule.com/blog/article/alex_at_umbra_ign...

I don’t understand how this works but it’s very cool. I would to be able to feed in a few photos of something like a property viewing or a birthday party and it just churns out photorealistic 3D scenes like this.

Makes me wonder about its potential for use in forensics.

There are many different "reverse rendering" algorithms being worked on. In essence, they take 2D photos as inputs, and try to figure out the 3D scene that they could have been taken from.

It's an absolutely fascinating area of computer graphics research and huge progress has been made over just the last few years.

Most approaches I've seen have just throw computer power at the problem (neural rendering), use brute-force traditional approaches like various point-cloud fitting algorithms, or use a clever trick like NVIDIA's differentiable 3D hash table.

This approach is in some ways the most naive, but that's what makes it so fast and so good! They literally just fit blurry blobs to the input data. The blobs can be rotated and stretched, so they can be used to model long thin lines or large flat surfaces. That's... pretty much it! There's some cleverness with using blobs of different sizes and splitting/merging them for efficiency, etc... but the point is that this simple method blows most of the others out of the water.

Your description of neural rendering techniques reminds me of a video (https://www.youtube.com/watch?v=6aXx6RA1IK4) where someone used a genetic algorithm to represent videos in Geometry Dash levels.