> Like most would agree that Ray-Tracing is the "real deal" relative to shadow mapping, but with how far shadow mapping has come can it really be considered a "hack" still?
Even ray tracing isn't the real deal and isn't trying to closely simulate reality. You need to use hacks in ray tracing for soft vs sharp shadows, for light patterns on glass objects like wine glasses (caustics), to get illuminated surfaces to spread their colour to other objects (colour bleeding) and more.
Phonton mapping gets closer but is even more expensive:
Phonton mapping is trying to model the path of all photons being emitted from light sources back to the camera vs ray tracing that works backwards by tracing rays out of the camera back to the light sources i.e. ray tracing is an optimisation.
You seem to think that ray tracing as a very limited thing. I guess it stems from the fact that the nomenclature has become a bit murky in recent years. Ray tracing often is used to refer to the initial algorithm developed by Whitted. This has pretty much all these limitations you mention, but it is an obsolete method. What usually gets mixed in with ray tracing in marketing and non-expert discussions is also path tracing and its various variations. These methods are capable of solving the rendering equation exactly for a huge range of scenarios. Unfortunately, some variants deal with certain situations better than with others and there is no overall best algorithm. But you can most certainly trace absolutely perfect soft shadows or complex caustics with these methods. Color spills are a walk in the park even for simple unidirectional path tracing. The main limiting factor right now is that these are all Monte Carlo methods and need to draw high numbers of samples for each pixel to converge. This is why fast and accurate denoisers are such a huge deal at the moment. They are another hack, a last one that ideally fades away as the number of rays per second increases. Beyond that, it "just" comes down to modelling surfaces and materials accurately enough to get a photorealistic image.
As you mention photon mapping: it has pretty severe limitations, notably around the handling of specular paths and the biased artifacts created by the filter kernel (loght leaks, certain SDS-type paths aren't covered). There are some improvements like (A)PPM which converge to unbiased solutions with infinite runtime. At that point, path tracing based methods become more attractive w.r.t. runtime, memory requirements and implementation complexity.
> You seem to think that ray tracing as a very limited thing.
My point was mainly that ray tracing is still a hack like polygon based rasterization is, that's a combination of clever tricks to get something to look good enough with modest resources.
I'd advise you to be more precise in your wording. The terminology is a bit tricky. Path tracing (which I assume you're actually referring to) as a an algorithm isn't a hack at all. It's a direct Monte Carlo solution to the rendering equation which has a pretty straight, albeit long, derivation from Maxwell's equations. Ray tracing hardware (that is, hardware accelerated ray-geometry intersection computation) is still quite constrained computationally. That's why we see little actual path tracing in real time renderers at this time and more of the slightly more correct rasterization based hacks that include ray intersection queries instead of screen space/cubemap based guestimates. Calling them ray tracing is technically somewhat misleading in my opinion.
You got that backwards, photon mapping is a hack in the same sense as shadow mapping while ray trasing is the real deal (although implementations will likely resort to hacks for perfromance reasons). Soft shadows are simply a matter of making light sources have a non-zero volume and while tracing rays from the camera instead of the lights is an optimization, it is a valid one that (given infinite rays) produces the correct result and is not a hack.
Even ray tracing isn't the real deal and isn't trying to closely simulate reality. You need to use hacks in ray tracing for soft vs sharp shadows, for light patterns on glass objects like wine glasses (caustics), to get illuminated surfaces to spread their colour to other objects (colour bleeding) and more.
Phonton mapping gets closer but is even more expensive:
https://en.wikipedia.org/wiki/Photon_mapping
Phonton mapping is trying to model the path of all photons being emitted from light sources back to the camera vs ray tracing that works backwards by tracing rays out of the camera back to the light sources i.e. ray tracing is an optimisation.