This really rings true for me. I've also found that looking for several "known" properties within a complex model's behavior to be an effective way of rooting out subtle bugs. If the model isn't keying in on the obvious, it hardly has a chance of keying in on subtle unknown relationships. I've even gone so far as to optimize hyperparameters based off context specific properties, e.g., hunting for the model that has the most coherent output behavior with regard to a range of inputs (assuming that outputs are expected to have continuous behavior).
https://sproutling.ai/blog/harvest-simulations?jm
https://sproutling.ai/blog/growth-simulations?jm