Congrats! I've been watching conduit for a while and I'm glad to see it progressing so much.
Matrix is moving towards a model where the 'homeserver' can be embedded in each device in order for users to fully own their identity. A lean implementation that uses an embedded database is a big enabler of that vision.
What's the upside of that? Each device being it's own homeserver sounds like a huge increase in the number of federation requests. It will simultaneously make it possible to run on every device, but impractical to do because of the huge increase in federation requests.
The domain name also seems more important. I can already migrate my identity to any host I want because I own the domain. If I didn't own the domain, I don't see how I could migrate "myuser@matrix.org" to my own homeserver. You don't own your identity unless you can migrate it.
I still love Matrix, I just don't see who wins in an "everyone runs a homeserver on each device" scenario.
Well, if everyone is running their own homeserver Matrix wouldn't be "federated" anymore, it would be "distributed" instead. You seem to argue for Matrix to go in two different directions :)
The way Matrix is doing it blurs the line between "federated" and "distributed" into a continuum which is really the best of both worlds imo. The existing concept of federation as a bunch of dedicated servers constantly synchronizing with each other 24/7 will continue to work just fine. But now each individual client device will be a full federated node itself, though perhaps a bit more flaky than a normal node might be; but this isn't a problem because the matrix protocol was designed from the ground up to be robust to such issues.
Based on your diagram, the system can seamlessly blend between (B) and (C) based on the needs of users. Eventually the default would become (C) while being backwards compatible with existing systems, but it might blend down to (B) if a user sets up a homeserver for friends and family, or an organization may want to have a homeserver for continuity or legal reasons etc.
Makes it a lot of sense, and P2P is exactly what we're discussing here. But P2P can take a lot of shapes, two of those shapes are "federated" and "distributed".
My issue is not that Matrix is moving towards P2P, but this particular user saying "unless everyone is running a server it's not federation" which is strictly false.
I’m not saying everyone will run their own homeserver (setup/maintenance costs, maintenance skills required, and economies of scale combined guarantee some considerable degree of consolidation, as with email), but that if it should be capable of it. If it’s not capable of it, it won’t be capable of even limited federation at sufficient scale.
There's pros and cons for that. In a stateless "message passing" model having many different servers has little cost, but in a "state resolution" (consensus) model having many different servers can have a cost because servers may be offline at times (and then need to reconcile history) or have their connections blocked to other nodes.
There's also the "backup" problem. You wouldn't want a user loosing their phone to lose their digital identity completely. It should be possible to craft some "social recovery" protocol using Shamir's secret sharing, but we're not quite there yet.
That was for the cons, since i'm sure we're all convinced of the pros :)
I was trying dendrite a month back and there were some big features missing: expiring messages, message notifications. Additionally the user management endpoints were still in flux.
This says the only important features missing are:
* E2EE verification over federation
* Outgoing read receipts, typing, presence over federation
I'm hosting this on a server with a small disk, and there are rooms which are lots of noise - I'd want clients to archive it if they really want to but not the server.
Ah I see. This is not yet supported in Conduit, but I can tell you that the conduit.rs server, which currently stores over 2 million events uses 8.6GB of disk storage + 7.8GB for media.
Question about E2EE verification: that means encryption over federation works, but you can't verify the encryption over federation, yes? Like, no green checkmarks?
I hope Telegram adopts Matrix (alongside or instead of their own protocol). Their clients (on linux/windows/android) are really an exemplar of how should a refined snappy cross-platform app should be written.
I wouldn’t consider Telegram a well-done cross-platform app. It’s a mobile app, badly ported to desktop, and I find telegram-desktop quite annoying under Linux. Here are most of the ways where there are actual problems, quite apart from it just using foreign UI design paradigms that make it not fit in:
Its keyboard accessibility is very poor: about all you can do is switching between conversations, scrolling inside a conversation, posting messages, and editing recent messages. You can’t interact with any buttons, menus or popups that they open.
Its text editing fields (plain and rich text) get caret navigation wrong for the platform, e.g. Ctrl+Right moves to the start of the next word, rather than the end of the current word; and punctuation is treated as a separate “word” rather than being grouped with a preceding (in the direction of navigation) word.
Its rich text editing component (for sending messages) doesn’t let you format text while typing: “hello, <Ctrl+I>world” should make the second word italic, but that Ctrl+I doesn’t actually do anything: you have to type “hello, world”, then go back and select “world” and press Ctrl+I. (This is by far the most frustrating of the issues, resulting in me mostly just not formatting messages.)
When I type emoji in (via a Compose key in my case, but it’s hardly the only way of typing emoji natively), they appear as rectangles, and stay that way until you post the message or type a character before the emoji, at which point it swaps it for a graphical emoji.
If you don't consider telegram a well-done cross platform app, then what are your standards? There is literally no other app that is as X-platform as telegram. Nothing by MS, Facebook, Google, Apple or any other company in the world comes close to releasing an app for Every mobile and desktop platform. On top of that, Telegram is better in performance and offer far more features than competitors who release in fewer platforms... FOR FREE. There are so many more things to praise before thinking of any criticism about emojis not looking right before sending in Linux desktop lol.
I don’t know if there are any in this sort of space that I would consider well-done. But I do know that a single-codebase mobile app port to desktop is unlikely to ever qualify, even if in some other regards it’s excellent.
The unified-UI battle is long lost. Nowadays, I much prefer a sane style instead of whatever RAD design GNOME/Windows11/etc people have come up with....
> Emoji
IIRC, the compose key uses a limited set of old emojis. I've been using dmenu-wrappers for inserting emojis, like this one:
https://github.com/Mange/rofi-emoji
The points you mentioned are minor issues (I know, it adds up) but compare that to the alternatives: Discord, Slack, Element, Whatsapp...Mere starting up those apps and sending a `hi` to someone easily could take ~30 secs
Many times, the sluggish experience in Discord caused me to just ignore joining in my favorite servers and discussions.
On the UI side, telegram-desktop does things in a radically different way from typical desktop software, that’s my objection, structuring things in a way that only makes sense on small-screen devices.
The Compose key can be customised via ~/.XCompose. I’m talking about things like U+1F928 FACE WITH ONE EYEBROW RAISED. Anything that has a graphical representation gets mangled.
On the alternatives: I haven’t ever tried much in this space, so I don’t know if there are any that I would consider well-done. Performance is probably the biggest objection to many of them.
not disagreeing with any of your problems, but i guess the point was that there are barely any other significant mobile messengers that also _have_ a desktop app, let alone one that is snappy.
Disclamiar: telegram user on windows, linux, ios and android
consumer and soho routers are usually bridging wifi and lan together whereas professional and enterprise tend to run seperate networks, mostly for security reasons.
Just for reference, gitter.im did just that a while back. It's really nice to be able to join a gitter channel using a client of my choice and not a crappy web interface.
Personally when i had to join a gitter room, i would join from my XMPP account using the @matrix.org gateway (which despite being unreliable is very nice) so i have no clue.
Some of the Matrix folks are active on HN, maybe they mentioned something in the discussions back then? But I wouldn't be surprised if it wasn't much if anything - my impression was that Gitlab didn't really know what to do with gitter, so giving it to messaging-focused project would make some sense.
No, bridged rooms generally are reachable from any instance. It's just that the bridge talks to matrix.org and matrix.org "hosts" (that's not 100% accurate afaik to how Matrix works, but close enough for this) the room just like any other matrix.org room, that is also reachable from other instances.
From what I know, they are still bridging, but in the long run (I assume they are somewhat waiting for stable Dendrite, since Synapse is shit for massive deployments) I believe they want to run an actual homeserver in gitter.im and have a sort of Gitter themed/branded Element client for it.
Literally trying this out on a raspberry pi right before I saw this.
I wanted to try building from scratch. 0 issues with compiling. Still trying to talk to it—probably just some issues with my local DNS. Very impressive to see this kind of thing though! This looks like an excellent candidate for a p2p implementation!
UPDATE: It is running! I just need to get nginx set up. It was really easy to get everything else running, (really nice to not have to configure a db!) and that I think will be a boon to those interested in starting up their own home servers.
Conduit was built around sled in the past. It has been depreciated in favor of sqlite as sled was using too much memory. (and sqlite is used as a key-value storage)
As there is now an interface allowing to add several storage backend, more backends could be added later (for instance a real key-value storage rather than a RDBMS used as a KV-store)
Not the guy you replied to, but I'd had my eye on Conduit for a while--after going deep down the rabbit hole dealing with Synapse, I'd given up on self-hosting Matrix for the time being and waiting for Conduit or something similar to get far enough along.
After seeing the announcement, went and got it deployed out in like an hour (would've been less but I spent 45 minutes compiling because I'm running FreeBSD).
The whole process was super easy. I'll have to play a bit more tomorrow--if I can figure out how to get stickers or something working I can probably convince at least my wife to move our conversations off of Telegram.
- install other important tools (git, nginx, certbot, etc.) [3 mins]
- compile conduit from source [idk, something like 40 mins]
Yes, building conduit takes a long time.
- meanwhile, configure nginx, get certificates, etc. [concurrent with compile step]
- set up daemon to run conduit [5 mins; took me more like 50 mins because this is the first daemon I've set up in rc.d–I'm a FreeBSD admin newbie, so I had to learn a bunch]
- ???
- profit!
All in all, remarkably slick with near-zero hiccups. Most issues result from quirks of FreeBSD that the install instructions don't cover specifically, as they're targeted for a Debian-based install. (In that case, things will go very, very fast because they hold your hand through everything. Way to go Conduit team!)
Federation is working. It's largely just working. Mind flippin' blown. You do have to register on the web client https://app.element.io because something is broken with the mobile SDKs I guess. Once you create an account through the web, you can sign in on Element, Fluffychat, whatever on mobile without a problem.
Again, remarkably smooth setup. This is fantastic work.
I'd be more inclined to want to check this out if there was a "click here if you already have synapse running" sort of thing.
My synapse and coturn server has been running since the end of last year without any issues at all, not sure I want to: collide the name (DNS) and possibly collide the @user:homeserver stuff either, as I do have issues with people on Samsung devices and managed IT devices that delete state at the end of the day. These show up as red shields in matrix clients regardless of doing the verification.
I think a transition guide from synapse would be good. But I wonder if it's a good idea to make that leap for live usersright now, the title of the post is "Announcing Conduit beta" after all.
This looks wonderful and very important. In 1995, when I started working with the Internet I believed that it was going to bring freedom and equality - to allow anyone (big or small, rich or poor) to communicate with the rest of the world. It never crossed my mind the Internet would become the primary tool for corrupt governments and enterprises to censor and spy on us.
In 2015 the number of users who use messaging overtook the number of users who use social media. “Messaging is the one thing people do more than anything else on their phone… this is where it’s happening. And it’s a once in a generation opportunity to build it,” said David Marcus, VP of Messaging Products. I've been building Messenger apps and using Google's conversational AI engine, Dialogflow, to listen to and respond to messages and sometimes I feel like I'm selling my soul.
By 2020 AI-powered insights businesses stole $1.2 trillion for their less-informed peers according to Forrester Research. "… by understanding customers more deeply and using that insight to steal them from their competition." "Customer service and support is the most augmented business process with AI ... sales and marketing are high on the priority list," says Gartner. All this is centred around messaging, the channel where your customers spend their time.
The reason I said all this is to stress how important messaging is to both businesses and individuals. I believe that messaging is the next generation of the internet. In the last few weeks, Facebook has been shown to secretly censor links sent on Messenger and Apple, as a first step to listen to our messages, announced they are going to save our children from paedophiles.
I am as worried as any parent about protecting my children, from pedos and crazy people who will use secure messaging to organise terrorist attacks. But I'm more worried that my children will grow up in a world without freedom of speech and privacy. I am worried that they will live in a world where they are frightened to speak their mind because they understand that everything they say is being permanently recorded. I am worried that someday, for what they said, they may be penalised by a corrupt government, corporation or an AI.
For me, the Matrix (since it sends encrypted messages directly from person to person) is one of the most important technologies that may allow my kids to grow up in a world where humans are free and safe to speak freely. The First Amendment says "Congress shall make no law ... abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble..." These laws have been bent and broken and will continue to be, so that the people who break them can silence their opposition, to remain in control. It's very sad that most people in the US are not fighting to keep the very foundations that their country was built on alive.
I'd like to thank the developers of Conduit and the Matrix for their efforts to keep freedom and privacy in our world. Conduit with its small footprint is a great step forward. I hope someday it will enable anyone and everyone who owns a phone to provide the infrastructure needed to communicate freely. In the meantime, I will install it on my Raspberry Pi. I suppose I should volunteer to help you guys in some way and put action to my words. I am not an engineer but if you are interested in assistance with your strategy, marketing, design, UI, website, or messaging feel free to reach out to @t0c:matrix.org I will have a look on GitHub anyway and find something to do.
I feel you, the tech space is way less exciting that it was decades ago. The last succesful communication standards are email, http and IRC.
XMPP is succesful in a way that it's used behind the scene in a good amount of apps, but that stops there.
The only alternatives are good old phone calls or SMS.
All what have been done since is to fragment communication as much as possible by jailing people in apps.
I really hope Matrix will manage to disrupt this and change this momentum. Part of it though would be the ease of setting up a server and avoid the feature fragmentation of XMPP that could be hard to understand from a user point of view.
I agree. It's not easy for the average user to setup a Matrix server, yet. With Conduit's light footprint soon we may be able to setup a server by simply downloading it from Google Play or the App Store. If the rest of the setup could be done through an easy to understand visual tool (including bridging between different messaging services) I expect Matrix to achieve a lot of traction.
I'm worried though that the technology will be made illegal. The seven -- US, UK, Canada, Australia, New Zealand, India and Japan, for example, said, "We urge industry to address our serious concerns where encryption is applied in a way that wholly precludes any legal access to content .... We challenge the assertion that public safety cannot be protected without compromising privacy or cybersecurity." Bills in the US such as EARN IT (could eliminate P2P E2E messaging. The UK's GCHQ has come up with an idea called 'ghost protocol', which would add the government as a secret eavesdropper into every call. I understand and struggle with the good reasons why governments feel the need to do this. For example, as technology becomes more powerful it's not unrealistic to imagine a scenario that someday all it would take would be one crazy person to release a futuristic bio or nuclear weapon that would kill everyone in the world. I hope Matrix isn't hindered too much by the law. I guess there are backdoors built into the hardware and operating systems Matrix runs on, so the government can listen anyway. With E2E encryption at least Mark Zuckerburg won't be able to, though.
Unfortunately Matrix and Jabber/XMPP are just as fragmented when it comes to client features. But both ecosystems are climbing the hill, on XMPP side there's now yearly "Compliance Suites" to help in this regard, i don't know the equivalent in the matrix ecosystem.
Disclaimer: i'm part of the joinjabber.org project, and we're very interested in user/operator feedback and how to improve things overall, including interop with other networks
I do run an xmpp server and use the Conversations app on Android (which is an app that every other messaging app should look at, it's just perfect).
It definitely got much better but and is heading in the right direction, but to convince people to move over it needs to be really easy. That's the one thing that apps like whatsapp, signal or telegram provides, it's easy.
Setting up a feature-full XMPP server is not that hard, but requires a good amount of admin still. Understanding the impact of choosing a client over another requires to have some basic knowledge of the protocol itself. For example, which XEP is supported by the client and the server.
I do get that the extensibility of the protocol is really powerful, but it's also its weakness in my opinion, adding a complexity that is hard to understand for non tech users.
Agreed conversations is a great client! Speaking of which, are you aware the maintainers of conversations.im have paid offers for hosting your own domains? There's also other similar offers from different providers. Is that a step in the right direction in your view?
> That's the one thing that apps like whatsapp, signal or telegram provides, it's easy.
There's also the Snikket.org project which operates on an invitation principle (small server model). Any user on the server can send invites for new users, and they just click a few links to get started. Once again, feedback is welcome.
> Understanding the impact of choosing a client over another requires to have some basic knowledge of the protocol itself.
That's a big problem and that's actually one of the reasons we started the joinjabber.org project, to help people choose a server/client. Let us know if something is not clear or can be improved on there.
Also worth mentioning, the Snikket project friendly-forks a bunch of clients to provide a consistent featureset/experience across platforms, as explained here: https://snikket.org/blog/products-vs-protocols/
Definitely what Conversations author is doing is a big step in the right direction. I managed to convert some friend and one of my sibling to it.
In term of paid offer for server hosting, I struggle a bit to see who's the target. I would expect that tech enthusiasts who really care about Internet's freedom as a whole to already have some level of self-hosting and knowledge to setup their own server (that's what I'm doing)
Non tech users will not really expect to have to pay for a server, I think that most of them would only think of Instant messaging as the client app (a bit like for email).
Maybe I'm saying crap because my knowledge of XMPP is very limited, but I would think that for XMPP to take on in the instant messaging space, it means that the protocol itself has to be focused on that. From there some form of versioning of the protocol that could mean that for a server or a client to be considered compatible with a version of the protocol, it needs to support a set of versioned XEP.
I think that would make things easier for a people who are writing XMPP app to focus on a common set of goals.
> In term of paid offer for server hosting, I struggle a bit to see who's the target.
To be honest, me too. But hosted versions of free-software are often popular, if only to support financially the maintainers. For example, the wallabag.it maintainer had a blogpost detailing his revenues over the past years running such hosting service: https://nicolas.loeuillet.org/billets/quatre-annees-de-walla... (in french but you can probably understand the numbers)
Although there is demand for collaboration services from non-profits and coops who often rely on Google services. Those who have a somewhat-techie person on board usually selfhost with Yunohost (at least in the french-speaking world where it has become popular) but others pay for external services from a tech coop (eg. webarch.coop). In this usecase, "clients" usually don't want to deal with a bunch of subscriptions from various provider and will go with someone who can do web/email/IM (at least).
Then there's the compliance suites, published every year (for 3 years now i believe) to document the required/recommended specifications to implement for specific use-cases (eg. web client, audio/video, etc). The latest one is here: https://xmpp.org/extensions/xep-0443.html
Overall the ecosystem is progressing better/faster than it was a decade ago. But new suggestions/critique are always welcome to keep improving things :)
EDIT: I should also mention modernxmpp.org project which is useful for developing clients nowadays.
I had used synapse in its early days (and really like it), and then eventually had to spin down my homeserver (not due to any synapse issues, I just didn't have bandwidth at the time to maintain host and synapse plus other experimental servers and services i was playing with)...Anyway, i recently started planning to spin up another homeserver, and was on the fence on whether to play with synapse again, try dendrite, or conduit...well, now having read the deploy readme file, i think i will give conduit a try! Kudos to the conduit contributors and maintainers!! Kudos also to all folks contributing to the greater matrix homeservers and clients, as we need plenty of solid choices!
Is it really free to all or this code have some holes for Germany government?
There are many interesting news about getting access to scan private messaging for child porn detection... What are the guarantees that your code will not provide access to private correspondence?
Matrix is moving towards a model where the 'homeserver' can be embedded in each device in order for users to fully own their identity. A lean implementation that uses an embedded database is a big enabler of that vision.