> The compiler is forced to make a static choice while hardware can choose based on dynamic effects that the compiler will never be able to predict.
At the limit of complete generality the Halting Problem guarantees your statement is correct. But it also appears to be extremely difficult to do this at runtime without side-effects that change global state such that information leaks. The real cost for speculative execution may be that it can't be done in a purely-functional manner.
It is true that we currently cannot hide the microarchitectural effects of unsuccessful speculation, as exposed by transient execution attacks like Spectre and cousins.
So what?
Most workloads / customers are much more about performance than security. We can trivially remove such attacks by switching off all speculation (branch prediction, caches, OOO, prefetching). Indeed you can buy CPUs like that, and they are used in environments where safety is of extreme importance. The cost of this is an orders-of-magnitude loss of performance
Most workloads (e.g. Netflix streaming, Snapchat filters, online advertising, protein folding, computer games, Instagram, chat apps) are simply not security sensitive enough to care.
Building a competitive general purposes CPU costs a lot (probably > $1Billion end-to-end), and who would buy a CPU that is safe against Spectre but 3 orders of magnitude slower than the competition? (Not to mention that there are many more dangerous vulnerabulities, from Rowhammer to Intel' Management Engine, to rubber hose ...)
At the limit of complete generality the Halting Problem guarantees your statement is correct. But it also appears to be extremely difficult to do this at runtime without side-effects that change global state such that information leaks. The real cost for speculative execution may be that it can't be done in a purely-functional manner.