The one issue for the longest time was that networks would use self signed certs and often different certs per server in the network, so it was hard to have any kind of trust. I did see a few using Let's Encrypt in recent years but moved over to Matrix these days
