"Ray Tracing" has always been an overhyped or misunderstood technology, at least in my experience. Because of the impressive lighting effects it's famous for producing, people view it as some "holy grail" of superior computer graphics technology that we just need to optimize a bit for use in games.
As Carmack described, those highly realistic ray traced renders come at a price: billions of ray calculations. You don't get all that pixel-perfect refraction, etc. for free (and probably never will - at least not with ray tracing.) He also explains that for most people, this (e.g. pixel perfect refraction) really doesn't matter, when rasterization techniques exist that achieve extremely high quality approximations at a fraction of the computational cost.
Conversely though, ray tracing and related concepts (ray casting) are not at all without value. Many modern video games today actually use a sort of hybrid rasterization / ray casting approach. Macrostructure objects are rasterized the normal way, while smaller high resolution detail patterns (like a brick wall, or stones in the ground) are "added" to surfaces via per-pixel displacement mapping, which is ray casting at its core. This is one of the few cases where you can take advantage of the "log2" efficiency Carmack mentioned -- in a highly specialized implementation without a huge time constant.
Pretty much agree, the only thing that we might disagree on is the 'billions' number, while I expect good ray tracing to trace billions of rays, if transistors are cheap enough this becomes more interesting. The typical HDMI 1080p display is 2 megapixels (or megatexels) if you have a rendering engine with 2,073,600 cores each of which is looking at a billion ray 'view' based on where it sits in the scene, its easier to set up the scene and light it. That presumes that you can do those cores like DRAM instead of like current processors of course. My point is that cheap transistors keep amazing me and whenever I say "Oh that will never happen" some fool goes and shows me I'm wrong.
I'm trying to imagine how complex a 'core' would be that computed the incident rays on a single pixel. Then figuring out how big that is in a 18nm process technology and then trying to see if I can fit 2M on a reasonable size die. My head exploded sadly.
Throwing more ( or finer etched) silicon at an algorithm will not help the algorithm, compared to a superior algorithm. For almost all scenes you get better results with rasterization, the two exceptions I can think of is scenes where the geometry of light rays is not flat, so for example involving lenses or black holes, or where there is a lot more optical complexity than you can reasonably use. ( A forest where each of the leafs is an object, you will never be able to notice that the advantage of this over an forest where the trees in the background are just textures.)
> Throwing more ( or finer etched) silicon at an algorithm will not help the algorithm, compared to a superior algorithm
Actually, it will when the complexity classes of the algorithms differ. Which is true in this case -- the time taken to ray trace a scene rises as the log(n) of the size of that scene, while the time taken to rasterize rises linearly with regards to the scene.
There exists a threshold of computing power/scene complexity above which raytracing beats rasterization in speed. However, like Carmack pointed out, the constant factors are massive, so this won't be reached in the near future, if ever.
The reason the constant factors are so huge is that complexity of raytracing rises linearly with the count of pixels to be drawn, while in rasterization a lot of the work can be shared by neighbouring pixels.
Intriguingly, this means that if you can reduce the pixel counts, you can vastly improve the value of raytracing.
Notably, if you can do eye tracking and rendering in less than 15ms, you can reach the same visual quality as a full-screen, high-resolution render by rendering only the areas you are actually looking at in high resolution, and rendering the rest of the scene at a progressively lower resolution farther from the focus point. The cone of high-precision vision is surprisingly small, something like tens of pixels when looking at a screen from a normal viewing distance. If you did this, you should be able to cut the amount of rays you need to send by at least two orders of magnitude, which would bring raytracing to real-time quality on modern hardware.
Does your last paragraph have any implications for VR? Could something like the Oculus Rift benefit from raytracing technology? Or have I misunderstood what you are saying?
I only skimmed the paper referenced below [1]. But what I understand at the moment is, that you essentially measure the area of the screen where high quality is important (using an eye tracker). And with this you can allocate your resources better. The downside is, that the rendering looks rather strange for anyone who looks on it without proper alignment of the high quality region. So VR goggles seem to be a good candidate for the technology, if they include an eye tracker, because they guarantee a single viewer and provide a frame to attach the eye tracker.
The idea is that raytracing can degrade gracefully, so there's a sliding scale between rendering time and rendering quality for every pixel. And yes, this means that, theoretically, the computer could naturally degrade pixels that you're not currently looking at.
I suspect this would look highly unnatural in practice, though, as even static scenes would flicker and change, especially with reflections and any surface or technique that uses random sampling methods (which is virtually every algorithm that is both fast and looks good).
> I suspect this would look highly unnatural in practice
In pair tests test subjects were unable to see a difference between the foveated image and the normally rendered one. The flicker can be removed by blurring the image to the point where flicker no longer exists. Because your brain is used to a very low sampling rate outside the fovea, it actually helps in hiding the artifacts, because they occur naturally in your vision.
Yes. But the "constant factor" will be very large, even larger than for traditional ray tracing, for some global illumination algorithms, even when you chose to only render a small part of the scene. For example photon mapping requires a pre-pass where you trace photons from the light source onto the scene, that means that reducing the pixel count doesn't decrease the time for the pre-pass. http://en.wikipedia.org/wiki/Photon_mapping
Yes, as I pointed out in the last sentence, perhaps I should have put a big-O somewhere into it. Do you have a link on the eye tracking/rendering combo? That technique sounds rather fascinating.
When CPUs have millions of cores and cost next to nothing to produce, the superior algorithm may be the one which scales linearly with parallel hardware. He mentions the problem of the log2 vs linear with rasterization, but that's just moving the log2 problem to hardware. Ray casting may still win out due to it's simplicity of throwing more hardware at it.
Yes, but it perhaps more instructive to think of a curved mirror. In case of a raster engine, you need to calculate a texture for every triangle of the mirror. ( Which is essentially the same as rendering the scene in the first place.) On the other hand for a ray tracing engine you just need to reflect one ray on the mirror surface for every pixel the mirror occupies in the final rendering. So compared to the same scene with a window instead of the mirror, you just need to calculate one additional reflection instead of running the entire rendering pipeline again.
Yes, I was essentially using one ray per pixel as shorthand for constant amount of rays per pixel. (As in 'I did not think about that point.') One needs more than one to avoid creating artifacts like aliasing ( see [1] for a very nice example), or depending on how 'naive' or complex the algorithm is for stuff like shadows or caustics. But for nice scenes (which do not create artifacts) with mirrors and lenses, one ray per pixel would work fine.
I think people rather think of ray-tracing as much better motivated by physics ( and intuition) compared to all the complicated matrices in raster engines. Instead of thinking it is superior because of some demos.
Physically Based Rendering - often GPU accelerated - is the new thing -- check out Octane Renderer, Lux-Render, Indigo, the Cycles renderer in Blender.
These renderers start at the light source and stochastically generate and follow photons. Like a digital camera they suffer from noise :-)
"These renderers start at the light source and stochastically generate and follow photons."
Yes, that's similar to ray tracing, but going 'backwards'
Rasterizing is certainly faster, requiring more work in the 'intelligence' of scene assembly.
With raytracing (or physics rendering) you have lights, objects 'naturally interacting' with each other so shadows, reflection, radiosity comes ""automatically"" (still hard to do)
With rasterizing you have triangles with different colors, and it's up to you to paint them accordingly.
"Ray Tracing" has always been an overhyped or misunderstood technology, at least in my experience. Because of the impressive lighting effects it's famous for producing, people view it as some "holy grail" of superior computer graphics technology that we just need to optimize a bit for use in games.
As Carmack described, those highly realistic ray traced renders come at a price: billions of ray calculations. You don't get all that pixel-perfect refraction, etc. for free (and probably never will - at least not with ray tracing.) He also explains that for most people, this (e.g. pixel perfect refraction) really doesn't matter, when rasterization techniques exist that achieve extremely high quality approximations at a fraction of the computational cost.
Conversely though, ray tracing and related concepts (ray casting) are not at all without value. Many modern video games today actually use a sort of hybrid rasterization / ray casting approach. Macrostructure objects are rasterized the normal way, while smaller high resolution detail patterns (like a brick wall, or stones in the ground) are "added" to surfaces via per-pixel displacement mapping, which is ray casting at its core. This is one of the few cases where you can take advantage of the "log2" efficiency Carmack mentioned -- in a highly specialized implementation without a huge time constant.