It’s not anecdotal. We used Anchorage for some time before dropping it for this very reason. There’s a compiler warning available to you to let you know if any line or function takes more than a certain amount of time to compile. Our autolayout code that used Anchorage became the #1 source of warnings when we enabled it.
I've been using my operators for a long time without a problem. I think the secret is the fact that I'm using custom operators that have no other overloads, which avoids the combinatorial type system explosion during evaluation. The operators I use are ≈, ≤, and ≥ (which can all be typed on a US English keyboard using the option key). I admit there are overloads on the normal arithmetic operators for doing constants/multipliers too, but those haven't ended up as a problem (it's quite possible that the use of the custom operators constrains the expression enough that the arithmetic ones aren't a problem).
For context, I've had the options on to warn on expensive expressions for a long time and we use a lot of these operators everywhere and I think only once did we ever have an autolayout line flagged, and that was due to a complicated expression used to come up with the constant value.