Are you sure that's how Matrix E2EE works? In my experience, to access encrypted chats after logging in for the first time I have to verify the new session by performing a synchronous prompt-verify sequence from an existing session, and I don't remember setting any encryption passphrase at all.
What gives? Are we just using different defaults or clients or something?
Disclaimer: I'm not an expert in this stuff, just a 3rd party client developer who is interested in the topic.
AFAIK, there are two ways to validate a new device in Matrix.
It sounds like you're describing the interactive device validation procedure. After that, your two devices trust each other, and the old device can "cross sign" the public key of your new device, so others can trust it too.
Personally for me, even with cross-signing, I was still getting a significant number of messages that failed to decrypt. Usually I think the problem was that one of my devices wasn't around to receive the original version of some key. And for whatever reason, even with cross-signing, it wasn't getting the keys from my other devices. For example, maybe my other devices were offline at the time.
The fix for me was to go the passphrase route. Now, regardless of what device I'm on, the passphrase gets me access to all of my secrets via the encrypted blob on the server.
What gives? Are we just using different defaults or clients or something?