For what it’s worth, the thing I find most exciting about Spaces is that they provide a decentralised hierarchical namespace with decentralised access controls for every room (ie pubsub topic) in Matrix. So it’s like we’ve sprouted an openly federated global hierarchical filing system for freeform realtime data streams of all flavours - where people can go crazy defining their own trees, applying their own curation ideals; perhaps we’ll even see a single global tree emerge (although the implementation may need some more optimisation first).
It’s like a multiplayer hybrid of DMOZ and USENET and the read/write Web all rolled together. Once we start storing more interesting data streams than instant messages in it (eg forums, email, bulletin boards, DOMs, scene graphs, ticker data, IOT sensor data...) it really gets interesting :)
Plus you can use it to organise your own rooms and have Discord style communities or Slack style workspaces, but that’s the boring obvious bit ;)
I would love to migrate my family to Element (and also friends, and eventually also recommend it to employers if they're ever choosing something other than Teams) but can't until it's reliable as webmail. I also wrote this comment:
I get that E2E means it cannot be as simple as email, and I can handle the extra training that's needed, but not for figuring out all these known issues and attempting to prevent people hitting them, or having to handle the situation if they do.
So there is a finite number of active contributors to Element, and we have to prioritise what to work on. While E2EE is not perfect (the UX needs another major iteration and we are still hunting rare edge cases where messages cannot be decrypted) in practice it's usable day to day.
We consciously chose to prioritise building out Spaces over the last few months over E2EE UX as otherwise there's a risk of Discord becoming the de-facto home for open source projects, much as Slack started to be a few years ago - which would be catastrophic for open standards-based communication like Matrix.
Meanwhile, the cryptography team chose to focus primarily on implementing next-generation encryption (IETF's MLS) on Matrix rather than polishing the current behaviour - given MLS should both radically improve scalability, but also fix the majority of the edge conditions which are problematic for today's E2EE, or at least entirely switch bugs in the existing implementation for entirely different failure modes in the new implementation. We showed off MLS over Matrix last week (https://youtu.be/xn0fzyimycs?t=248), and we're now finishing the decentralisation component of it (https://matrix.uhoreg.ca/mls/ordering.html).
Eitherway, now that the Spaces beta is out the door, we're catching up on other UX issues, including E2EE. We also have more folks being paid fulltime by Element to work on encryption (amongst other stuff) starting in July. Talking of which, if anyone wants to get paid to make this happen sooner, Element is hiring at https://apply.workable.com/elementio.
Thanks for the response, that sounds good and makes sense, I guess I'll continue waiting...
My family regularly does things like forget which browser they last used, or re-install windows, or otherwise mysteriously have all browser settings disappear, etc - all things that seem to be related to those key-backup issues.
I'll just say; both of these seem like buckets of good priorities. I too wish some aspects of the onboarding were simpler, but I also am very very excited about Spaces, and have a number of communities I'm stoked to onboard once that feature launches.
Kudos to you and the team and I've been really enjoying tracking your progress over the last ~4 years.
- cross-signing (so users verify their own devices themselves, and you verify users only once by verifying their public key, regardless of how many times they add or update devices)
- and key backup (so moving between devices doesn't need manual polling for other devices' keys the first time for decrypting messages)
would be a great step forward. Those are there, but they are disabled by default yet, which is a disservice.
cross-signing & key backup are enabled by default, and have been for a year or so? you have to explicitly opt out by repeatedly hitting the 'skip' button. however, the UX is certainly more complicated than it should be, and we're due to rework it in the coming months.
Woops. I may be dealing with friends & family that joined prior to that. I wonder if one could get prompted again for enabling cross-signing & backup once you open a new client (I don't think it happens right now). That would help migrating everyone to enabling it.
I'd love to see those encryption usability issues fixed, but I do want to say that e2ee on Matrix is less bad than on anything comparable (like OMEMO on XMPP). I like XMPP because of its speed, but you really can't combine OMEMO and multi-device unless you just don't mind having parts of your message history unreadable on some devices. The key backup and session validation in Matrix/Element usually solves this issue.
Unfortunately, the complexity means that even other Matrix clients that support e2ee don't support key backup and session validation, which means you either stick to Element on every platform, or live with the same history issues as on XMPP.
Boring and obvious, but this is probably what most people would want. Now that we've seen how good and usable the Discord model is, it's pretty much the baseline now for public chat rooms.
To build an open source decentralised Discord alternative you need a tool-box of primitives - that’s what Matrix is. Then clients like Element (https://element.io) sit on top to fill in the rest.
We don’t think the model for Spaces is overcomplicated. In fact it’s simpler than Discord (no role-based permissions; just power levels).
Is there any documentation detailing power levels, or that decision was made to only offer power levels but not role-based permissions? FWIW, the reason most major platforms offer role-based permissions is because enterprises demand it, and it’s become the de facto primitive for authorization. User type based permissions also make sense, but are often used in conjunction with (not in lieu of) roles.
Because the decentralised ACL implementation (state resolution) relies on power levels as a simple comparator for determining merge conflict resolution. State resolution is the academically novel bit of Matrix and hard to get right, and while we can swap it out for role based ACLs in future, this is orthogonal to Spaces. Meanwhile it honestly hasn’t come up as a requirement, even from enterprisey folks, given we push power levels as hard as possible.
I should also add that there are at least two matrix spec change proposals flying around to add role based ACLs, and one of the main reasons for switching spaces to be built on rooms is that they will automatically inherit the ACL model when it lands :)
Solving conflict resolution should be doable - it's just that it's literally the most critical and important bit of the entire Matrix ecosystem, and any changes have to be done very deliberately and carefully, ideally with formal proofs that it's not going to break stuff. It's the equivalent of switching out Git's merge algorithm (https://github.com/git/git/blob/master/Documentation/technic...) or switching out for a new kernel scheduler implementation. Any bugs or thinkos there mean that the whole fabric falls apart, hence wanting to consider it separately to Spaces.
It’s funny you mention that — I just started such a project (three hosting modes: centralized, federated, fully decentralized) as an educational series to teach software development on multiple levels. It’s very early (read: non functional), and I’m still figuring out the best way to teach and build simultaneously, but constant weekly progress is being made.
Educational site (I’m backporting the previous sessions to the site, session 7 is the first one hosted right now): https://www.codewolfpack.com
Wohoo! Thanks to Spaces, I feel comfortable using Cactus comments for blog posts (e.g: https://viccuad.me/blog/enabling-cactus-comments), as I could create a private space containing all the rooms for each blog post comment chain.
To me it shows the snowballing effect of features.
Also, Matrix's Spaces (even in Beta) seem more flexible and thought out than the second best implementation I know, in Slack.
Matrix’s threading model has been designed to support Zulip (we’ve chatted a bunch to Tim about linking them up) - the only catch is that it’s not implemented in any clients other than Cerulean so far. Element should add support soon now Spaces are out though.
What's the Matrix analog for Zulip? My understanding of Zulip is that you have an instance, streams in that instance, and topics in that stream.
Maybe in Matrix the hierarchy would be implemented by nested spaces? A space for the stream and one for each topic? That would probably work if not for the critical Zulip feature of being able to move messages between topics (and between streams?) to better organize the conversation.
Maybe one Zulip instance is a giant Matrix room and the threading model that's landing soon is used to implement the relationships? I know the upcoming threading model can dynamically change event relationships. But then maybe it would be better to avoid giving single rooms that much traffic.
There are probably lots of ways to implement that functionality. I'm curious as to what you were thinking.
What is the E2EE story for spaces? Besides space names and member lists which I assume are visible to the server like normal rooms, what other information is also visible to the home server? Are rooms/subspaces inside spaces also visible?
Spaces are identical to rooms, which means that state events (key value data like name, membership, avatar, etc) are currently not e2ee, but conversation data would be e2ee if there was any. Encrypting human visible payloads for state events is on the roadmap however, but encrypting the metadata would be hard. For that, we are working on P2P Matrix to stop the metadata pooling anywhere.
It's also early beta, though (although we're committed to getting Matrix hybrid client-server & p2p in the medium/long term). https://matrix.org/blog/2021/05/06/introducing-the-pinecone-... has more on the latest developments on P2P (and public beta builds for iOS & Android) from a few weeks ago :)
Pretty exciting stuff, still trying to wrap my head around some of the technical possibilities outside of chat. Just to be clear, a room can be contained in multiple spaces, right? Or even, if I'm understanding it correctly, in multiple places in a hierarchy in the same space?
The Element blog post talks about setting up admin spaces and community spaces. So if rooms aren't exclusive to a single space I could build custom, private spaces that pull rooms from multiple different communities I'm a part of.
That's a really cool feature that Discord doesn't have right now.
There are lots of different things you can do with Spaces. But yes, you can build your own Space out of arbitrary rooms you don't own without anyone's permission.
Then there are what you might call "canonical" spaces, where space A labels space B as its child, and B labels A as its parent. Then you know that the relationship between the two is mutual, and clients can take that into account.
But you're free to create your own Spaces that are the parents of other rooms, where those rooms/spaces are unaware of the Space you created.
Once this becomes widespread, I can actually start using Matrix as a real Discord / Slack alternative (communities just weren't cutting it, unfortunately).
Once the audio chat feature is seamlessly integrated, I even consider ditching my old mumble server for it. Or not. Probably not. Our group just needs audio, with the occasional picture flood, until they get it out of their system. Since some (me include) won't use Discord due to privacy concerns, I still hope for a good and reliable self hosted alternative. A more modern user interface sure would be nice.
Great to hear! I'd love to be able to recommend Matrix/Element as a Discord alternative.
However, actually competing with Discord will require a tonne of hard work in Element. The little things like automatic voice activation, echo cancellation and volume normalization are the reasons Discord are as fantastic as it is, so Element will need to be excellent with those to compete.
The first cut will be using Jitsi as the engine, albeit with more appropriate UI, which gives you AEC (echo cancellation) from WebRTC and some level of AGC (normalisation). But we have plans to go far beyond that, and are very aware it’s hard work. However, pre-Matrix, the core team professionally built VoIP stacks, so we have some experience here (and our own WebRTC implementation should we need it :)
The main reason to consider something different to Jitsi is to directly use Matrix for decentralised e2ee signalling to manage the media streams, and allow hybrid SFU and MCU models (like hangouts or zoom) rather than pure SFU like Jitsi. We do like Jitsi though and already contribute directly and indirectly - Jitsi’s E2EE is derived from Matrix, and the Matrix community just contributed a tonne of a11y fixes to it that just landed. But we’d still like a fully decentralised Matrix-native group call solution eventually.
If you do AGC, you'll already be more usable than Discord, who apparently refuse to implement this. Whenever we're voice chatting in my group, it's extremely annoying that one person is quiet as a whisper and the other is AT 200% VOLUME AND CLIPPING.
The risk on AGC is that unless combined with good voice activation you can end up amplifying background noise and then deafening people when they start speaking. But yup, we'd definitely want to do this.
Oh, sorry, you meant input normalization, of course. Please please please also add output normalization, so all people in a room sound at the same volume!
I think it depends on which perspective your inputs/outputs are coming from ;) The way for all people in a room to sound at the same volume is to normalise the audio you capture from them. If you normalise the mixed signal you play back to the other users in the room, it's too late (at least if they speak over the top of each other), plus it'd be too hard as you'd be constantly yoyoing around.
So I understand why the spaces api is similar to rooms from a developers perspective, but why do they share the same # sigil as rooms and not take the old + community sigil?
Under the hood, spaces are rooms - and we made the decision to keep using a # sigil to refer to them, given the act of joining and being invited and having multiple aliases etc is the same as a room. When you say “hey, come join #community:matrix.org” does it really matter if the thing you are joining is a room or a set of rooms?
That said, it’s subjective, and we went back and forth on it. My original proposal had a twigil of #+wherever:example.com, which in retrospect was daft. I think it was a member of the community who proposed just using plain # aliases - hard to tell given https://github.com/matrix-org/matrix-doc/pull/1772 is so heavily commented that it crashes if you expand the comments :/
This is really cool tech. I'm using it in my "Discord meets Minecraft" app (jel.app), it's basically exactly the primitive missing to really have a multi-tenant story for these kinds of chat systems.
I looked at this post and the user facing blog post, but I’m not sure if I’ve understood this. Is it just a way to organize different rooms together for easy access (like Telegram’s folders, which is just for each person, but here these can also be shared with others)? If yes, then can someone please point to similar implementations (preferably with screenshots) on other platforms for comparison and where the similarities and differences lie?
At Element we have a bunch of customers doing so, but the one I can speak to most accurately is Element itself, as a company. Our internal space view looks like this, for instance: https://matrix-client.matrix.org/_matrix/media/r0/download/m.... As you can see, it's pretty enterprisey - we even have HR rooms ;P
A good and robust bot api and useful smart widgets based on url are a must. I've had chats go from discord to matrix and then simply die after a few days.
Not sure what problems you're seeing with the Matrix API; it's lightyears ahead of Slack & Discord in my experience, but I guess i may be getting high on my own supply...
> As many know, over the years we've experimented with how to let users locate and curate sets of users and rooms in Matrix. Back in Nov 2017 we added 'groups' (aka 'communities') as a custom mechanism for this - introducing identifiers beginning with a + symbol to represent sets of rooms and users, like +matrix:matrix.org.
We already have a universal notation for identifying resources on the Web. Why not use that?
Do the Matrix clients still phone-home to some centralized server (Vector, I think it's called) as well as to matrix.org on every launch?
That's what kept me from switching last time; to me a federation client that's going to phone home to a central server is no better from a privacy standpoint than using Signal (which isn't federated but is also end to end encrypted and phones home to its centralized server), and Signal's UX is better thus far.
If that can get solved (and it seems like it can), Matrix looks like it's going to be super awesome.
> Do the Matrix clients still phone-home to some centralized server (Vector, I think it's called)
There's no centralized server. There's a server with the majority of users for now (in matrix.org), but the network is decentralized, with many servers. The server implementation run by the majority is called Synapse, but there are others (Dentrite, Conduit, Construct..).
I'm puzzled by this comment. Can you elaborate on what you mean by phoning home?
Are you talking about https://element.io, which points to matrix.org homeserver and signing server, etc?
In that case, well, just use a different server and your own client. Or self-host, and select one of many clients: https://matrix.org/clients/. That's the whole point of federation.
You can even deploy your own web client pointing to whichever server default you want, many people do. E.g: element.debian.social, chat.mozilla.org, etc.
The comment you're linking to says that there are no current plans to change the default request to matrix.org from the login page, and that you explicitly plan to keep the current auto-updating behavior.
(also, of course "we will fix that" implies that it is currently still broken, which is what the original commenter was pointing out)
> As should be pretty clear from #11655 (comment), we're going to fix this, and yes, imo, autoupdates need an opt-in.
The comment #11655 says:
> Auto-updating for default Riot distributions will continue as before to ensure security fixes are delivered quickly
So at this point already it becomes unclear to the reader which of these has precedence.
So, assuming you're asking in good faith, if you wanted to be clearer you should have written something like:
"*Despite* what's written #11655 (comment), I disagree with the product teams assessment and think autoupdates need an opt-in, therefore I will make sure this is implemented."
(not sure if the content is 100% what you wanted to express in the comment, but you get the idea)
When can I install and run a matrix client without it connecting to you? I want to use your software but want to self-host it and not use any services whatsoever from your organization.
They could be referring to the vector identity server for email/phone number -> matrix matching. I believe this is enabled by default on synapse installations (self-hosted ones too), but the URL can be changed in the config.
It’s like a multiplayer hybrid of DMOZ and USENET and the read/write Web all rolled together. Once we start storing more interesting data streams than instant messages in it (eg forums, email, bulletin boards, DOMs, scene graphs, ticker data, IOT sensor data...) it really gets interesting :)
Plus you can use it to organise your own rooms and have Discord style communities or Slack style workspaces, but that’s the boring obvious bit ;)
Edit: for a user-facing rather than developer-facing overview, https://element.io/blog/spaces-the-next-frontier/ has more details.