And that's exactly the kind of stuff Google is trying to accelerate. One tangential but highly relevant example is their work on predicting molecular properties [0]. They were basically able to use neural networks to speed up the simulator by 300,000x, taking something that would take hours to less than a second. As you mention, this entirely changes the way you work and iterate. Here's a video of Jeff Dean talking more about it [1].
How can something like this even conceptually make sense? I'm sorry, but can someone explain to me how a neural network could approximate physically-deduced laws of nature (as had-coded into the classical simulator as I imagine) "without any distinguishable error"?
This is the biggest bs I have ever heard, since an molecular physics simulator that is beat by a neural network can only just be unoptimized as hell.
[0] https://ai.googleblog.com/2017/04/predicting-properties-of-m...
[1] https://www.youtube.com/watch?v=rP8CGyDbxBY&t=16m43s