If you actually dig into Matrix.org, you'll see that the goal is for it to work offline, peer-to-peer, ideal for use in areas of natural disaster, and cities that become battlefields.
P2p over IP is useless and failing to protect against DPI.
Also natural disaster or war tend to impact infrastructure and cutting communication.
Have a look at meshtastic real p2p chat out of band (Lora based) and ciphered. Cheap devices with the right firmware and you can setup fairly an infrastructure and chat p2p
If you don’t own infrastructure you are in the hand of the owner.
https://matrix.org/blog/2020/06/02/introducing-p-2-p-matrix
https://matrix.org/blog/2021/05/06/introducing-the-pinecone-...