VR is especially unforgiving of "fake" detailing, you need as much detail as possible in the actual geometry to really sell it. That's the opposite how these models currently work, they output goopy low-res geometry and approximate most of the detailing with textures, which would be immediately register as fake with stereoscopic depth perception.
Yup. I'm doing a VR project, urban environment. Haven't really found a good enough solution for 3D reconstruction from images.
Yes, there is gaussian splatting, NeRF and derivatives, but their outputs _really don't look good_. It's also necessary to have the surface remeshed if you go through that route, and then you need to retexture it.
Crazy thing being able to see things up to scale and so close up :)
Not meta VR, but one of my favorite things to do in gran turismo 7 with my PSVR2 is just “sit” in the cars and look around the cabins. The level of detail the devs put in is on another level.
I find it much easier to remesh and deal with textures with a crappy 3d reconstruction vs working with 2d images only. I also shoot HDRI and photos for PBR. I find sculpting tools super useful for VR, but yeah its still an Art even with all the AI help.
Given the scale it wouldn't be wise to render them directly. There's also the issue of being able to record in real life without changes happening while doing so.
I should've have clarified it, but yes I was talking about the extracted surface geometry.
I would push back on this a bit. The best games I've played on my VR headset are not "realistic". "Fake" is more about consistency I guess, if everything around me in VR looks equal amount of "goopy" then I don't think it will feel weird.
Doom 2 in gzdoom VR mode is amazingly convincing for how janky the graphics are. Even with the enemies as 2D sprites it’s quite compelling, so it’s my go-to rebuttal for all the “VR games need amazing graphics” sentiment. Good low-detail artwork is fine.
We need more of the stable diffusion tool set in 3d AI creation. Upscaling does incredible things in SD adding in all kinds of detail and bringing 512x512 to 4k. Inpainting to redo weird arms or deformities. Controlnets like outlines, depth, pose etc to do variations on an existing model.
The real fun begins when rigging gets automated. Then full AI scene generation of all the models… then add agency… then the trip never ends.
this is why I love half life alyx. it just gets so much detail in VR space in a way that no other game ever has that makes for a truly immersive experience.
I'd liken it to the trend from 5-10 years ago for every game to have randomly generated levels.
It does't feel like an expansive world - it's the same few basic building blocks combined in every possible combination. It doesn't feel intentional or interesting.
I think displacement maps are often made by starting with high detailed models and converting some of the smaller details to normal, bump, reflection? maps etc.
correct. the process is called texture baking. however, displacement uses displacement maps, which are grayscale, essentially depth maps. displacement is not very useful for realtime rendering because then the displacement has to be calculated in realtime. displacement is mostly used in offline rendering. realtime rendering just uses normal maps.
It absolutely does, I don't quite get comments here. All VR games use normal assets with all their normal (map) shenanigans. It works the same as on desktop.
Sure if you look at something super close the "illusion" breaks, but it has nothing to do with VR, same thing happens in flat screen games.
Maybe people mistakenly think that most standalone Quest games don't have those
maps because they don't work? Well it's not the case. The standalone games (especially on Q2 vs Q3) have just very low performance budget. You strip out what you can to make your game render in 90 fps for each eye (each eye uses a different camera perspective so each frame scene has to be rendered twice).
Funny, pretty much everything on my Meta Quest seems to be "fake" detailing, without much detail in the actual geometry.
I mean, yes it's obvious because the GPU is only so powerful. The difference against my Xbox is night-and-day.
But even if VR is unforgiving of it, it's simply what we've got, at least on affordable devices. These models seem to be perfectly fine for current mainstream VR. Maybe Apple Vision is better, I don't know.
