I'm not a mathematician at all (mechanical engineer), but to me, "exact" sounds like "deterministic" as an opposition to stochastic.
I though optimal conveyed the idea of "literally the best possible solution but you're still in the presence of a fully random system here".
Which might be the wrong interpretation, but hopefully it explains why some people (who aren't necessarily familiar with rigorous mathematics) use optimal.
I do see your point. But if you're talking about a probability or probability distribution, it can still be an exact solution to a model. For example, if I throw two standard dice, what is the probability of throwing two sixes? The answer is 1/36. To me, it sounds odd to describe 1/36 as the "optimal" solution to that problem, even though it's stochastic. Even "exact" solution is a bit odd, I'll concede, but a lot less so. "The solution" or "the answer", with no more qualification needed, sounds best to me.
It's an estimator in this case. A fixed number is an estimator too, it's just not going to be desirable in most cases. But single numbers are absolutely and unquestionably also valid estimators.
In any case, what I'm trying to get at there is that in estimator theory there is a concept of optimality for an estimator over a distribution.
Sure, a single number is a trivial example of a procedure/formula.
But an estimator estimates an unknown parameter from data (or in such a trivial estimator possibly without data) - and I believe this is central to the confusion.
I though optimal conveyed the idea of "literally the best possible solution but you're still in the presence of a fully random system here".
Which might be the wrong interpretation, but hopefully it explains why some people (who aren't necessarily familiar with rigorous mathematics) use optimal.