This is trivially avoided by adding a setup password. Additionally, the system could display a hash of the keystore at boot - AIUI, Purism has worked on something like that.
OK, so we're talking about users who have unlocked the system themselves and enrolled a user key onto it, but aren't going to notice if it changes unexpectedly? Suuure.
