Standard tls between server and client helps against any outside threat. The only threat from which e2ee helps better than tls comes from the server admin.
Now, do you REALLY need end to end encryption to protect yourself from a person running your team chat server?