They should at the very least evaluate ActivityPub. It's a W3C standard that reached "Recommended" status almost two years ago, is flexible enough for future use-cases, and already has multiple implementations like Mastodon.
In fact, any app can be made to act as a source, sink or both of ActivityPub events. I recently added ActivityPub support to learnawesome.org so that reviews can be consumed in any ActivityPub client. Implementation was easy and the data model is quite easy to understand.
ActivityPub is real-time pub/sub for the entire Web, something that Twitter could have been.
Three reasons why not: operational, business, and technical.
Operational: they don’t control features; adding them by contributing to the standard would leak planned features, and without contributing, would be seen as embrace-extend-extinguish.
Business: it would contribute work to their competitors.
Technical: implementing global search on it is implausible.
This is doomed from the start, then. There’s no way for them to work on an open standard without leaking features or contributing work to their competitors.
Publicly, organize social engagement. As long as they are at the helm, and at the forefront of tech innovation, they won't have issues on that goal.
Privately, diffuse regulatory pressure. If unlike Facebook, they can share blame with other actors, they won't singlehandedly be responsible for political mayhem, so there won't be calls to break up Twitter. It is the whack-a-mole approach, similar to BitTorrent vs. Napster.
That’s fair, but then why are they receiving any sort of praise for this? Neither of those goals benefit anyone who doesn’t work for or own stock in Twitter.
> If unlike Facebook, they can share blame with other actors, they won't singlehandedly be responsible for political mayhem, so there won't be calls to break up Twitter.
I don't buy this. Nothing is going to change from the user perspective, and almost nobody in the grand scheme of things will likely be aware of this open standard anyway, let alone people working in government. I don't see how this will change the public and political perception of social media websites whatsoever.
Maybe I misunderstand the scope of the technical details, but if the underlying flow of messages across social media was forcibly abstracted to a universal master hub, then could not individual clients (or sub-platforms) not then exert control over capabilities (filtering, censorship, etc) that is currently controlled entirely by the individual social media platforms who own the networks and userbase?
> and almost nobody in the grand scheme of things will likely be aware of this open standard anyway, let alone people working in government. I don't see how this will change the public and political perception of social media websites whatsoever.
I would argue this is a side-effect of the current state of affairs - obfuscation during studies and testimony (what little takes place), but mainly the inability to demonstrate superior implementations.
I'd rather we don't give up without having actually tried anything.
EDIT: Although, I could easily agree that this is simply a disingenuous smokescreen to buy time and mitigate legislative risks. In fact, I'd bet money on it.
> Although, I could easily agree that this is simply a disingenuous smokescreen to buy time and mitigate legislative risks. In fact, I'd bet money on it.
To be clear, this is roughly what I'm arguing. In theory, I openly welcome Twitter and Facebook adopting an open standard and giving users the option of easily moving their content to another service, but I don't think it will pan out that way in practice due to the current state of affairs. For one, brand recognition is very powerful, and I think Twitter knows this and is banking on momentum keeping users on their platform regardless of the technical ability to move to another service. Also, most people simply won't understand the implications of this standard or why they should care. ("Why would I use XYZ when I can just use Twitter/Facebook?")
well it would solve problems for all the people who really do want to use a different / better social network or chat app but cant get their friends to switch too, killing the idea on arrival. that would change perception of social media, because it'd allow every host-site could be distinct to their targeted users' qualities as a group, but maintain connections to everyone in their lives.
i mean, you arent really saying that you dont believe a massively-confederated social networking infrastructure would change the political perception of social media? i think it necessarily would, and change every individuals relationship to social media too. its like that marshall mcLuhan quote 'the medium is the message', ie the medium defines what it contains and what those contents mean.
Maybe I'm being cynical, but I'm skeptical that whatever "open standard" Twitter is working on will achieve that utopian future you're dreaming of. In practice, I think the vast majority of people won't even know it exists and will continue to exclusively use their preferred platform (Twitter, Facebook, whatever) despite all the issues with it. The idea of being able to move all of your content to another social network at a whim, with all your connections undisturbed, is (IMO) simply never going to be something that your average user does, no matter how technically feasible it is.
Hopefully I'm wrong and I'm not giving enough credit to social media users in aggregate.
yeah i mean honestly you're probably right. But in reality, the whole point of advocating for something that skeptics call "utopian" isn't to achieve that specific utopia per se. IT is to make improvements towards that "utopia". Like for example, if this social media interoperability and open standard world is our utopia, that would rely on having at least a framework for inter-network communication and migration from one to another.
What im saying is you probably never get to the utopia, but maybe building some of the requirements for that utopia will be beneficial in an ever imperfect world.
You actually describe yet another benefit of decentralisation.
Because this would certainly happen when the current movement of email into two major silo's (Gmail, Outlook) goes forward. When those two (or three, or four) have a majority of email-users on board, governments will start discouraging or forbidding usage of other mailservers. And will start making demands of those few leftover mailserver monopolists.
Indeed. If they aren't working on an open standard, what are they working on, and what's the point then? It won't solve any of the problems open standard is supposed to solve.
i feel like the function of success would be pushed into other areas like the actual value of the feature being made available to competitors. or speed of competitors adoption of said feature. basically, sharing a foundational open standard could make everything thats distinct about a feature's implementation, or that companies policies / mission, the decider about what differentiates success.
now, current leaders probably wouldnt want to adopt a system like this because it would be a test of their value on all those metrics, but i wouldnt say that makes the notion doomed from the start. i get what you mean though.
i mean, where would be right now without the open standards of 30 years ago?
Completely agree; commercial services prefer to use their own protocol over a common protocol for technical and commercial reasons. For example XMPP is a federated messaging protocol but Facebook Chat moved off XMPP, AOL AIM moved off XMPP, Google Talk moved of XMPP., even Twitter moved off XMPP at one point, because they all implemented custom features and eventually devolve from the base protocol.
I read this as a trivial 'truth' that I've internalized myself as well; but the way you worded it... makes me naively ask:
doesn't it speak to the limitations or choices of XMPP itself then, not the idea of a common protocol, if businesses can't reliably extend the protocol (only custom fork, and somehow barring eventual upstream contrib, whatever)?
I can think of an extensible protocol paradigm, with 'extensions' (really just libraries, packages, like those we fetch with `apt` or `npm` and optionally push to clients as well). Down to first principles, that's how we structured UNIX/Linux, most modern programming languages, the web itself (well, at least the Js and now wasm part of it).
I actually don't think it has anything to do with technology or problems with XMPP. Corporations can move standards better than anyone (look at web and chrome/google). If Google and Facebook had scale problems then they would just build better implementation.
Facebook messages happened when FB didn't have so many users and in chat space there were XMPP services like ICQ/Jabber. Network effect was against FB so it made lot of sense to federate. When FB was convinced that by killing federation more people come to FB instead of leaving they just pulled the plug.
I am sure that if 80% of email users were using gmail Google would do the same thing. There is a chance of that happening... First gmail adds some amazing but nonstandard features, people start relying on those features and in time gmail becomes the one. Only thing thats keeping that from happening is actually other big businesses.
I'm not sure what you would want from a standard. The basic idea is very simple, crawl the entire web starting from a few known points and visiting every single url you see on every page. Now you have an index of the internet you can run searches on.
The only problem is meaningfully ordering those results and having the computing power to do it. An open standard doesn't solve those.
Operational: activitypub can be extended with proprietary vocabulary without any issues, so they can develop features quietly before proposing them for inclusion in the spec, if they even do that.
Business: what kind of work would contribute to their competitors? On the standard? Well, 90% of open standards (invented number) was created by for profit companies. On libraries and services, why would it?
Technical: implementing global search on the www as a whole is also impossible.
About Twitter RSS: I recently discovered nitter.net. It's a Twitter front-end without all that tracking and with the addition of RSS. Jack's at https://nitter.net/jack and the RSS is https://nitter.net/jack/rss
I really like this. I had been looking for a cli client and wasn't impressed by any of them, and this seems like a good compromise for reading others timelines, especially since I am an RSS-aholic.
I am a daily and happy user of Mastodon. I honestly see no reason to use Twitter, nor one of their products at this time. Calling it now: too little, too late.
ActivityPub lacks cryptographic signing/identity (table stakes for 2020) and anti-censorship features. It is more of an MVP/example than a mature, forward-looking protocol design.
> They should at the very least evaluate ActivityPub. It's a W3C standard that reached "Recommended" status almost two years ago, is flexible enough for future use-cases, and already has multiple implementations like Mastodon.
Sadly, all centralized Facebook/Twitter/YouTube since 1st January of 2020 would add more restrictions (according each new ToS and Privacy rules).
BTW, I already prepared for fully migrating my Twitter activity to Mastodon.
@jack's idea probably involves some kind of cryptocurrency integration into the protocol. I think having a digital native currency can alter the economics of protocols. The end result may be some combination of ActivityPub and an economic incentive layer.
Best to think about it like email. You can send and receive email to anyone at any domain, if their server is set up for email. We all just happened to centralize ourselves onto Gmail and Google's servers.
Replace email with "social media content" and you have ActivityPub.
Not really? From my perspective as an outside observer, the WHATWG is the one built/captured by corporations. The W3C gave up and agrees now that the HTML spec is mostly just whatever WHATWG says it is, but that's partly because the W3C seems more interested in putting time/money/effort into standardizing things where they are wanted such as ActivityPub.
It has learned from those times, as much as you can even talk about "W3C" as an entity doing coherent things. Individual working groups have can have very little overlap with other groups, and have a lot of freedom in how they work. Which means they can putter about in ivory towers, or do effective work, as they wish.
I agree that sounds good, but can you point to something that they have actually done lately?
The ActivityPub process is a good example - yes, it got there, but much of the difficulties around the standardisation process seem related to the W3C process, not standardisation itself.
I was involved around the edges of the Atom standardisation process[1]. There's a reason that went through the IEFT and not the W3C[2].
A typical solution to this is to store the data outside of the blockchain and then checkpoint a hash of the data snapshot into the blockchain at regular intervals.
This provides tamper resistance to the data.
Incentivizing users or organizations to store large amounts of data is still a problem of course.
It costs ~$1500 for the Internet Archive to store a TB of data forever. Cut them a check or provide an ongoing endowment (along perhaps with some technical resources to scale up a distributed web instead of existing solely in SF) for them to be storage of last resort (everything else can be a tier of caching on your platform).
Let's Encrypt runs for around $3MM/year. Same budget for Internet Archive allows for adding storage for 2PB of content. Scale up accordingly. No blockchain! [1]
Twitter and Facebook and Google and others share one common concern that, if addressed, would see their staffing budgets massively reduced (and thus profits increased):
Human moderators.
Each grudgingly uses human moderators to squash the worst of the problems on their platform, and does a terrible job of it. Each underspends on human beings, using contractors without sufficient mental health care to ensure their well-being as they sift through our online sewer pipes.
Google outsources YouTube moderation to third-parties: the Content ID system is a labor shifting device designed to force the labor cost of enforcement outside of Google’s responsibility.
Twitter ended political advertisements rather than spend the human cost required to moderate them, and is now proposing a decentralized platform where Twitter is no longer responsible for content moderation for other platforms.
By doing so, they can continue to act as your “aggregator” of individual (RSS-like) Twitter accounts, so that they can continue showing you ads based on the data they harvest from you and your feeds — while outsourcing responsibility for moderation to others.
This is an effective strategy for increasing profits and if implemented correctly will permit mass layoffs of most of their content moderation workforce. This will also vastly increase the prevalence of abuse, racism, and other societal ills that infect Twitter with its underpowered moderation today.
Props to Twitter for identifying a way to externalize the cost of civility while continuing to profit from the resulting cesspool that will ensue.
The problem is online social networks. There is no fix.
Any medium supported by ads is toxic. This predates online social networks.
Any news feed (recommenders) is intrinsically toxic. Such feedback loops are just outrage machines.
It's highly doubtful that likes and ratings can be redeemed either. I'm fresh out of goodwill, so I don't care to try.
So what's left? The graph. Maybe that could be useful.
Meanwhile, it's hysterical that anyone tainted by Twitter imagines they have anything constructive to contribute. Now were he to completely renounce the whole effort, acknowledges it's deleterious impact on democracy and society, turn it off, and beg for our forgiveness. Well. That'd be a good start.
Google decided scale was more important than civility when they positioned text search - Pagerank - as a competitor to curation, rather than as a beneficiary of curation.
We used to have curated content indexes with human moderators guiding them - in print, Yahoo/DMOZ, AOL keywords, web rings, LiveJournal. Those required significant human labor compared to algorithmic text searching, and were not able to scale as rapidly as content creation tools did.
Was that lack of scaling a problem? No, not necessarily. Some percentage of people enjoy curation (thus Pinterest) and we could have celebrated their efforts and given them top billing in search.
Instead, we “democratized” search by harvesting all of those human rankings and feeding them into a machine algorithm that produces seemingly better-than-human results. Unfortunately, in doing so they did not highlight whose curation led to content being shown, and so curation became less popular over time.
Unfortunately, that curation is what led to Pagerank being so valuable. Without it, spam and liars and malicious activities have infected all “search” and “ranking” systems. Without human curation distinguishing “valuable” from “unevaluated”, search does not scale either.
We did society great harm when we sidelined curation, and no amount of machine-learning algorithms will heal that wound.
Recommenders, like bureaucracies, are misanthropic (anti-human).
The rules are meant to remove human judgement. While denying the baked in bias and dysfunction of the imposed ruleset.
Per Goodhart's Law, they arbitrarily state some things are worthwhile & meaningful, and everything else are not.
Black boxes which thwart inspection, transparency, accountability, explanation.
--
Forgive me for flogging this horse; I do have a point.
Also missing from online social networks are the concepts of fair and impartial adjudication.
Curation, adjudication, transparency, accountability... I'm sure we're omitting many other missing features. Because we're too close to the problem.
Going meta meta here: the common trait of all these "regulation arbitrage" unicorns is they profit by the destruction of our society's laws, checks & balances, social norms, and so forth.
I mean impartial in the sense that the adjudicators (judge, jury, arbiters, moderators, whomever) are not beholden to the publisher. So a third party. Like our court system is supposed to be.
IMHO, there's not enough daylight between ombudspersons and their paymasters.
I have no fixed opinion for standards of civility. I'm just trying to capture the notion that some dialog, rhetoric is out of bounds, as determined by the intended audience (context).
Nope, I've been scratching my head about those for a long time now, but haven't found anything better than adding a note wherever there might be a confusion... :/
Oh, I didn't think you were serious about that - this is probably not feasible because it would be insanely "expensive" ?
(I mean, Slashdot seems to have found the best middle ground here, haven't they ?)
I wouldn’t know, that question is apples and oranges different.
Twitter is creating a syndicated public one-to-all feed system; Email is a syndicated private one-to-many feed system. The public/private difference cannot be discarded and makes your question unrelated to my statement.
I encourage you to pursue a top-level thread about your concern instead, so that it’s not disregarded as a reply to mine.
Email lists are not private if you choose to sign up for one and post to it, the same way Twitter is not private if you choose to sign up and post to it. Email lists are very similar to Usenet which was the precursor to all these social media sites.
Apples can be orange in certain breeds or lighting conditions, but that doesn’t make them oranges.
Twitter is not equal to an email list. They share certain aspects but they are significantly dissimilar in UI, approach, social perception (by participants and by others), and publication.
M Twitter has not replaced mailing lists, and mailing lists continue to operate effectively.
Twitter is an electronic billboard that anyone with an app can post content to, and everyone can easily see it and do the same. Mailing lists are a town hall meeting inside a building: you can see them if you like, but participation from bystanders is quite rare and is often frowned upon or openly prohibited by the meeting’s coordinators.
You’re acting like there’s some extremely clear difference, but there isn’t. The only real difference is that email has kind of a bad UX. The comparison is not apples and oranges, it is grapefruit and oranges.
"the value of social media is shifting away from content hosting and removal, and towards recommendation algorithms directing one’s attention"
I really wish I could hack on the youtube recommendation algorithm. It seems to be tuned for zombie viewing pleasure, and totally not for finding surprising new content. Even just having several "personas" would be a big help.
Yes I feel like personal ownership of the recommendation engine is the key battleground..otherwise whatever the data is a third party is always deciding what you see.
Totally agree. The lack of configurability around recommendations is really frustrating. So often, the next video that plays is something tangentially related to what I'm watching, but still not what I want to watch.
I'd love a feature that defaults the next video to the newest unwatched video in a given channel or something like that, but we all know why it won't be built.
The problem is the revenue model. Zombie viewing is more profitable than surprising new content.
If you were asked to hack on YouTube's algorithm, any modifications you'd produce, no matter their merit, would be tested against the status quo for their ability to generate revenue.
I'm sure that is what they tell themselves at youtube HQ. But I'm not convinced. Why should one A.I. fit everyone? It just seems like laziness, ie. corporate inertia, to me. If they could open up an API for third party recommenders this would really be interesting.
Not really. Surprising new content has more active engagement which is crucial for acquisition ads. Revenue for pay-per-click is always better than display.
Yes. It creates another problem on a different level, which is that creators now try to hack the algorithm and generate contents that are more likely to be recommended.
Hm I disagree with that. I think for the short term it’s obviously aligned for zombie viewing but I’ve been introduced to completely different and new content via their recommendation algorithm on the homepage (no, not trending).
without user history to build a video similarity embedding with PCA from the user to video click matrix (or word2vec like click prediction), you would have to build something like .. topic based page rank.. and then show the highest ranked videos per topic.. that would require a fucking lot of mining
I wish they would made different algo for different "categories" of people, and you could just select from a dropdown list the one that works best for you. It would be win / win imo.
But Matrix is mostly built around chat rooms, not feeds like Twitter. I would assume any new system would take some parts of Matrix and some parts of ActivityPub but not the entire thing.
We built Matrix to synchronise any kind of comms data - we just call the primitive a “room” rather than (say) a “pubsub topic” or whatever more general term. You can absolutely build microblogging on top of Matrix; there are already a few toy examples already.
I was looking into building something on top of SecureScuttleButt, but didn't like some of the opinionated choices in it (same goes for ActivityPub). Looking at Matrix spec, the two seem to be somewhat conceptually similar, and Matrix seems to cover my needs. Never even thought to consider it before, but it might actually be a really good fit!
I'm considering launching a couple matrix instances this month,
any chance there are ways for me to dub a few users as moderators and give them the ability to silence / kick / ban users by ip, hostname, ip subnets and cidr?
I'll be looking into whatever bridge things there are for combining matrix with some microblogging, that would be a really perfect fit for 2 of the projects I've had on the idea shelf this year.
If it can connect with wordpress / buddypress for logging / blogging things from chat by user, that could prove interesting for some uses indeed. Of course if it connect to mastadon or anything else similar that could work too.
It's been a while since I've checked in on Matrix and what's new. I'd love to find someone who knows the code and could help me mod it a couple hours a month for a reasonable fee.
I should emphasise that microblogging in Matrix is pretty early, but we built the protocol to be able to scale for it. There are some very basic bridges to Wordpress and Mastodon etc but they could be way more sophisticated.
Moderation has improved loads though - https://matrix.org/docs/guides/moderation/ has the details. You can't ban users via IP though (unless they're on your server), as otherwise it's a massive privacy leak to expose IPs.
Very impressive progress on the moderation additions and the documentation is excellent (compared to my expectations for open source stuff especially!) - I can tell there is forward thinking and flexibility is being baked in.
Really glad to see this, I've been searching hard for a chat system replacement, and now it seems this might be it. With the ip addy api thing, if I can get someone who knows the code enough to help me use that to ban / block (on our own server) or bridge it to iptables, access.watch or something similar - this may be our future.
As far as a bridge to WP being basic at the moment, that is just fine, all I need is a way to port conversations from a certain room to posts, pages or whatever into WP and the rest can be handled from there later.
Once you strip the perceptive differences from the UI isn't twitter just a bunch of chat rooms and private conversations that update less 'live' than you'd expect a chat room to do.
I wrote a toy twitter clone using a discord server as a data source and it worked pretty coherently
what i think jack/twitter is moving towards is a model thats catering to the direction the youth are headed, with twitters implementation of topics, its akin to discords (gamer chat app) interest servers
Agreed (I'm a matrix user myself). When I saw the thread I was confused about what exactly (if anything) he wanted to achieve that couldn't be done with matrix. I'd be surprised if he wasn't familiar with it already.
Perhaps it's largely about branding/development control? I imagine it looks better to investors to say "we're making this new thing" rather than "we're gonna use this other thing that we don't own".
I stumbled upon Matrix the other day. The reason I passed it over was it sounds like all discussions happen in rooms, which made me think Matrix is basically chat room software. Is there a way to have a public discussion on Matrix with the entire user base like Twitter?
“Rooms” in Matrix are more a space in which you persist arbitrary data rather than a literal chatroom. They can absolutely be world visible and used to create per-user timeline or per-tag timelines as you need for microblogging.
There are lots of different ways of skinning it. One we've thought of in the past is to have a room per sending user, and have the members of the room be that user's followers. The rooms are of course publicly visible to allow everyone else to follow along too. You the need some mechanism to index the messages based on hashtag etc, but that could just be a search engine - or possibly copying/referencing the messages from per-hashtag etc rooms.
I highly doubt that Twitter approaches the Matrix Foundation with a "we have demands, you kneel to our will for our blessing / funding" and gets anything less than resounding applause.
The developers behind Matrix are currently funded from cryptocurrency money to the best of my knowledge.
at least with regards to identity management, twitter seems to favor a blockchain idea, which could actually simplify user management and make obsolete the need for home-server identifiers, 3rd party ids etc
I suspect you're confusing the protocol (which can scale fine) with the Synapse implementation (which was never written for performance in mind). The protocol itself should be fine for twitter-size amount of messages however, and there are more performant implementations in the works.
While it's probably just an extension of Jack's obsession over blockchain, it's a bit interesting that you'd have to use Twitter to apply: there are a few truly-decentralized/or federated solutions that have been proposed and have an active userbase, and the authors of the good stuff don't tend to be on Twitter.
For example, one of the people working on one of the most prominent pieces of software implementing the ActivityPub specification has proposed/is ~close to releasing a reference-implementation of one [a non-AP distributed protocol], which seems to solve most of the issues that Twitter would need to solve for privately-scoped posts (a major problem with most federated and decentralized social media), with plans to ~eventually get around to solving publicly-scoped things:
(Pleroma doesn't really do per-user threading, so the responses to the original post are mixed in with the author continuing the original thread. Reader beware, etc.)
OCAP is such a needed feature. One of my concerns with Twitter is that it's essentially unsafe -- while safety is a tricky thing to guarantee in decentralized networks, OCAP makes serious steps towards addressing the problem.
I'm glad they're planning on using ocaps. Not so optimistic that twitter understands ocaps though. They should hire Mark S. Miller to consult for them or something.
Exactly. I somehow doubt Dorsey doesn't know about ActivityPub - why not just endorse it publicly if he genuinely cared about decentralization?
Are there genuine technical shortcomings in the ActivityPub standard that Twitter's engineering team has identified? Why not talk about that and contribute improvements to the standard?
Are there shortcomings to being part of the standardization process in the W3C? Then talk about that and make the processes better.
Is it that he needs his name stamped on a new protocol? I'm not sure we can do much about the terminal egotism of the ultra wealthy and its pernicious effects on our society. Solving this will take a tremendous amount of collective effort.
> I somehow doubt Dorsey doesn't know about ActivityPub - why not just endorse it publicly if he genuinely cared about decentralization?
He doesn't care about decentralization. ActivityPub is impossible to directly control and it has no innate analytics, advertising or monetization layers. This makes it impossible for Twitter to utilize such a protocol.
If anything, Dorsey wants to embrace, extend, and extinguish AP by making an incompatible protocol that he can then pump VC money into in order to have it gain traction on Twitter's terms - with all the ads, spyware and analytics that Twitter desires.
Although knowing Mastodon doesn't mean you know ActivityPub. If I recall correctly, Mastodon didn't implement ActivityPub in the beginning but started recently (within last couple of years)
First of all, I literally read the spec (and related specs) last week in detail.
Secondly, there's a massive difference between decentralized and distributed systems (I build distributed systems for a living). Perhaps you're confused about that difference?
Thirdly, if you're going to knock a collective effort like ActivityPub by calling it a "horrible protocol" without genuine technical substantiation, you're effectively just trolling. Show us that you know what you're talking about by doing some in-depth analysis of the protocol and where its shortcomings are in terms of being decentralized.
I hate AP as a protocol, but also I'd still rather use it than try to invent a new one that wouldn't get a lot of traction (though I guess if you're a big corp, there will always be traction).
For two of the biggest uses of Twitter, privately-scoped posts and private messages, Twitter implementing ActivityPub would effectively make that information public. Not a good thing, and one that Twitter would get a lawsuit for. AP is broken for any use other than publicly-scoped posts, and given that Twitter is already a huge target, I think adding fuel to the fire would really be bad here.
Sorry for my ignorance but I don't understand how that's the case? As you've stated masterdon doesn't follow activitypub 100%, would would be stopping twitter from keeping their private chat between twitter users (or even adopting another open protocol such as matrix) whilst moving their public facing stuff to activitypub?
Twitter has "private accounts," which are accounts that have restricted followers, and only allow posting to followers. This is fine with Twitter's way of doing things, however breaks in a federated world.
Matrix is bad for many reasons, but it's not a direct fix over this problem more than a band-aid.
Keeping private stuff centralized would be reasonable, but they're not going to do it: expecting large entities to handle something like this with surgical precision rather than a sledgehammer is unrealistic.
It's certainly decentralized, just federated instead of distributed.
And yeah, it sucks as a protocol (but not that badly, the standard is just weak like everything else the w3c produces) but it's better than reinventing the wheel. Or letting twitter unilaterally dictate a standard.
And please stop spamming the same message down the page.
Email is federated (you pick a mail host and can communicate with people on different hosts), while BitTorrent is distributed (users send data to each other without needing a fixed set of servers to mediate).
Technically, decentralisation refers to anything that is not centralised (so both federated and distributed protocols would count) but in modern usage decentralisation refers more to distributed protocols.
Federated has some, perhaps different, individual or organization in charge of each server to which users are able to connect. There may be hundreds of different individuals and companies hosting and controlling different servers. All of which could either choose to communicate with each other, or not. Each of these servers would have, perhaps, millions of users who would then be beholden to the regulatory whims of the individual or organization controlling the server they join. An example would be Mastadon.
In a completely decentralized architecture, every client would also be a server. And every client could, potentially, connect to every other client. So every user is beholden only to himself. Other users would be free, of course, not to connect to that user, but the user could post whatever s/he wanted without fear of being cut off. There is no good example of this architecture out there right now.
Basically with Mastadon, you can be cut off. With a hypothetical completely distributed social network architecture, you can't. (Well, maybe your ISP could cut off your internet connection? But that's a whole other issue.)
> In a completely decentralized architecture, every client would also be a server. [...] There is no good example of this architecture out there right now.
BitTorrent ? Git ? Bitcoin ?
(They can also become centralized if there's a will, see Pirate Bay / Github / MtGox.)
While I'm all for open standards and protocols, Twitter has proven itself hostile to 3rd party developers. Even if their proposals are good Twitter has a long way to go to prove themselves trustworthy participants in open networks.
Yes exactly, instead of being open they’ve consistently shut the doors. You have no idea the lengths I’ve been through to keep that service up. In fact the technology decisions by the guy charged with leading this effort (CTO) have been more restrictive as of late not less. For example the web UI is now totally obfuscated server rendered code in effect nullifying any chrome extension that wants to work on top of it.
Mastodon has a number of instance and user-level boundary controls, like image muting for porn and sex work instances, that let people still federate with instances that could be trouble without that boundary. It would keep a hypothetical Twitter.com instance from causing what happened when AOL brought millions of people online.
I think Twitter should whitelabel its application and sell 'instances' to media, publishers, government agencies, and other organizations who would benefit over control of their namespace. G Suite seems to be a good model to follow.
What I'd really like to see would be a focus on nonprofits and grassroots political candidates, such as a NationBuilder for the Fediverse.
One of the massive "troubles" that social media has been having to deal with in recent years is the whole fake news problem. (in quotes, because it also has driven social media engagement, and thus revenue from ad impressions up, so from a business perspective it's a boon with the right narrative) Admittedly, it isn't a new problem, it just it's perceived impact has grown to the point of catching the public eye.
Twitter's response is that they're banning political speech entirely, to try and get out of the fake news domains that people care the most about.
Now, they're pushing a decentralized social media system that by it's very definition will be hard to control content on. In being a client for said open standard, they are certainly free to block whatever comes in that suits their fancy, but to me, it feels more than they're using this as a way to shift blame with regulation looking like it's on the horizon. It'd be an awful conveinent excuse to say "but Mr/Mrs/Miss regulator, we're just an aggregator, other entities are responsible for this content."
Maybe they really do believe the best approach to combatting this is to serve as a principled (by someone's definition) client of a sewer of public content, or maybe this is a strategic move to shift responsibility. I wonder which it'll be...
Webber's trashed it (the initial author) and is working on a way to mitigate the problems of it by implementing incompatible changes, the biggest piece of software claiming to implement it doesn't even implement it (Mastodon), the only piece of software that stays semi-faithful is full of devs who hate it (Pleroma).
AP is completely broken for anything but publicly-scoped content, relying on a lot of trust for every party involved. This gets broken frequently, and has had consequences so far on networks implementing it (like half of them are incompatible implementations, so I think it's completely fine to say "networks").
So let's assume you can get Twitter to implement ActivityPub perfectly to-spec. Great! It doesn't work with literally any pre-existing ActivityPub software, and users' DMs and are more or less public, with users' private accounts literally being public.
I use AP daily, and while it's fine for technical users with a reasonable understanding that anything they post is public, putting naive users' data at risk has never and will never be acceptable; pushing AP will harm everyone.
> Webber's trashed it (the initial author) and is working on a way to mitigate the problems of it by implementing incompatible changes,
This is not true on multiple levels
Webber has not trashed ActivtiyPub. I'd love if you could provide some evidence.
And the updates, in the form of OcapPub, are not at all incompatible with the current AP. They're an optional add-on that has a good default behavior.
Source: I worked on the same paper where Webber talks about improving ActivityPub, I presented my own white paper about ActivityPub. I gave a talk about these possible upgrades at the ActivityPub conference and I do a podcast with Webber where we talk about ActivityPub.
Sure, Mastodon doesn't implement the full ActivityPub spec [0]. They're still very much behind the effort though, it seems [1] (or at least they were in 2018).
I'm not a Mastodon developer, nor have I developed with it, but surely there are learnings there that could potentially be contributed back to the W3C to make ActivityPub better (including making it more rigorously spec'd).
The people at the W3C are not gods - they're human beings just like the rest of us. Why not get involved in the conversation there?
The only genuine reason I can think of to split away from the ActivityPub effort to build a completely different protocol would be if the W3C's working group on ActivityPub is filled with such toxic politics that it's impossible to make any progress on making the spec better. (That's been the only reason I could see in the past when I've been part of protocol standardization efforts).
Without evidence of that, however, I'd say that it's far better for the community to get involved in making the existing spec better. Robust debate (supported by evidence, such as that which Mastodon's built up) is incredibly valuable to the community as a whole.
Also, it's incredibly valuable from a political decentralization perspective to have as neutral a party as possible facilitating the standardization efforts, to minimize the risk of Big Tech shaping standards in ways that would suit them financially (and potentially against the interests of the average internet user).
Sure, Mastodon doesn't implement the full ActivityPub spec [0]. They're still very much behind the effort though, it seems [1] (or at least they were in 2018).
Not only this, but they go against it in multiple places. How they've implemented 'summary' and the bizarre things they've done to create compatibility with Misskey (but not in an actually-compatible way, Misskey's had to patch their own solution every time) has made every implementation have to implement an against-spec trashfire.
I'm not a Mastodon developer, nor have I developed with it, but surely there are learnings there that could potentially be contributed back to the W3C to make ActivityPub better (including making it more rigorously spec'd).
Alternatively, the AP spec is fundamentally broken in many ways, and that effort would be better off going into a protocol like Secure Scuttlebutt (not a fan, but it's much better than AP) or Diaspora (also not a fan, but it's much better than AP).
The people at the W3C are not gods - they're human beings just like the rest of us. Why not get involved in the conversation there?
The only genuine reason I can think of to split away from the ActivityPub effort to build a completely different protocol would be if the W3C's working group on ActivityPub is filled with such toxic politics that it's impossible to make any progress on making the spec better. (That's been the only reason I could see in the past when I've been part of protocol standardization efforts).
Not only politics, but it's incredibly slow. Webber has written a post on how troublesome the standardization process was, though he doesn't go into just how bad it was:
Without evidence of that, however, I'd say that it's far better for the community to get involved in making the existing spec better. Robust debate (supported by evidence, such as that which Mastodon's built up) is incredibly valuable to the community as a whole.
There are already existing specifications, none of them as bad as ActivityPub. I'm fully for going in one of those directions, but AP is more or less a failed experiment.
Also, it's incredibly valuable from a political decentralization perspective to have as neutral a party as possible facilitating the standardization efforts, to minimize the risk of Big Tech shaping standards in ways that would suit them financially (and potentially against the interests of the average internet user).
W3C is not neutral, and not only are they not neutral, but the RIAA and MPAA are people with enough sway to have messed up the entire web multiple times (see: making EME a standard).
A neutral party sounds great! The W3C should not get to be involved, however.
Hi, Chris Webber here. You're quoting my blogposts out of context.
Yes, the standardization process was a difficult one. Standards efforts often are, and social web ones especially. Nonetheless I think we did a good job.
The purpose of those posts is to ask people to stop fighting each other and try to work together; it feels highly ironic for you to quote them to increase division.
> W3C is not neutral, and not only are they not neutral, but the RIAA and MPAA are people with enough sway to have messed up the entire web multiple times (see: making EME a standard).
> A neutral party sounds great! The W3C should not get to be involved, however.
Perhaps I got lost along the way, but... we're still talking about Twitter here right?
Are you seriously calling out the W3C, an open but obviously flawed organisation, for finally capitulating on DRM, after huge public drama and well-documented opposition from within the W3C. DRM is something entities such as Twitter use as a matter of course; there is no public discourse, nor documentation.
This isn't even as basic as pots calling kettles black, any reservations one could have about W3C as an org are positively laughable when the alternative is a standard stewarded by a corporation with the size and influence of Twitter.
> Twitter making a standard would at least result in something not stupidly broken.
Agree with most of your comment, but I can't see where this is coming from. I would be shocked if any "standard" coming from Twitter would be anything but broken.
Again, you're saying W3C doesn't work in the interests of the user, but ignoring that Twitter works actively against the interest of the user.
W3C isn't ideal, but the alternative being discussed is devoid of any merit whatsoever.
The alternative being discussed is Twitter appointing a different neutral party: if they're bending to the W3C, they're already appointing a "neutral party" to do the work for them, the only difference is changing who they trust from another giant organization that's very clearly apathetic about user interests to one that does: the SSBC would be perfect, for example.
> Secure Scuttlebutt (not a fan, but it's much better than AP)
If you have any specific feedback about which improvements are most important to you, I'd love to hear it. It's hard to see things with fresh eyes after you've been working on them for a bit.
I actually think that the SSBC has done a fantastic job with it: the work that's been done is really impressive. I'm mostly just not a fan of the ground it's built on; the implementation itself is amazing, and the protocol is solid. I wish you good luck with it!
And to anyone unfamiliar, I recommend checking out the "Principles" section of the handbook; there's a lot of interesting stuff in it, and it proves that Scuttlebutt's really got the best culture/morality of any distributed network so far:
It's a quirk of the main feed encoding type, but there are other feed types (e.g. Gabby-Grove) that are major improvements. Right now the main hurdle is just adding support for more feed types to the major clients, which is something I'm actively working on.
SSB is great; I'm not sure if others would agree, but while I think it has a lot of problems, none seem inherent (i.e. while they mightn't necessarily be easy to fix, they wouldn't involve any major architectural overhauls or scratch rethinks; what's there is conceptually sound).
The main issue for me is performance and resource usage, which is tricky since a lot of the traditional solutions to perf/resource challenges are cloud-based, which would erode ssb's value. Thinking about SSB on older recycled or embedded systems, etc. is hard.
The other is NIH; this is definitely something I feel the community is acutely aware of however; I recall seeing discussions of parallel implementations in Kafka, etc. years ago. I haven't kept up with recent development though.
The performance problem is hard, but we have lots of active experiments that I'm optimistic about. Implementations in Golang and Rust, a handful of great database experiments (even simple stuff like Sqlite), and a bunch of people who are actively using SSB on low-resource devices daily. The Manyverse app in particular runs on Android devices, which are going to be very resource-constrained when compared to most computers.
The NIH problem is hard, but I think we're moving in the right direction.
I think this is a mischaracterization. I've read Webber's writing on OcapPub[0], and his more recent writing on ActivityPub[1]. I'm not getting the impression that Webber thinks ActivityPub is broken, or that he's stopped advocating for it, or that he's trying to replace it.
In his own words:
> "OcapPub" sounds like a new protocol, whereas it's really just a way-to-use ActivityPub mostly as it already exists. Maybe time for a new name for that?
Even if Webber does think ActivityPub has fundamental problems, he doesn't seem to be discouraging anyone from pushing it. I mean, go ahead and watch his most recent talk and decide whether or not Webber is "trashing" ActivityPub.[2]
> "It was 3 years of really hard work, and it wasn't clear in the process if it was going to pay off. And it has. We now have this as a standard that we can all use to be able to speak to each other, and I think this is really exciting."
> putting naive users' data at risk has never and will never be acceptable
That's a misrepresentation of history. Being responsible for your own data hygiene has been the norm in many online circles for a long time. I have often experienced pretend-stalking (e.g. via exif information not stripped from images) as a way of opening the eyes of community members about the data they leak.
Thank you for saying this. ActivityPub is so wrong in so many ways, and still people keep pushing it as if it was a silver bullet. I think people who talk about it and think it's very good never tried to understand it or write something with it.
I don't think people are pushing ActivityPub as a silver bullet, so much as they're using it as "the currently-available and less-awful alternative to closed silos". If someone is starving, they're not going to hold out for a gourmet cheesecake that's a year away from being made. They'll happily take an immediately-available baked potato instead.
People are pushing AP as a silver bullet. Look at this thread! Twitter is funding work to be done in more than an year and people are saying that should just use AP.
I disagree. I think most of the people in this thread are promoting Activitypub merely as a starting point. Sure, it could use some tweaks and improvements, but it's a good foundation to build on.
Your previous comment came closer to the truth: ActivityPub may work today _as a workaround_, a half-baked alternative to closed silos; but it doesn't have a solid foundation, its problems can't be solved by tweaks and improvements, and that's what every of its critics say.
I think this is going to be an "agree to disagree" situation. Frankly, I think Activitypub is just fine as the basis for building Twitter alternatives. Most complaints I've read about it are from people who dislike its lack of privacy and of the necessity of trusting servers with regards to private messages. I think those people are trying to fit a square peg in a round hole.
I'm actually building an Activitypub implementation myself. I quite like the protocol and the philosophy of "twitter without Twitter Inc.", but I'm thoroughly sick and tired of the rube goldberg "framework on top of framework on top of framework on top of obscure language for the backend" insanity that comes with the usual implementations. So I decided to do something about it. I'm working on an Activitypub server in Go, sqlite, and a pure vanilla javascript frontend. My goal is to have this be dead simple to install and maintain on a barebones VPS for anyone with a modicum of Linux commandline experience. One static binary, one small config file, one folder with a handful of javascript/css resources and html templates. And that's it. No bloody vue, react, angular, ember, or rails. And definitely no f---ing node.
(Breaking out this sentence into blocks because it's very large.)
The specification as-written is really, really bad (though I don't think this is a fault of the protocol),
Synapse is kind of...and work on Dendrite seems to have mostly stalled,
the billion people signed up on matrix.org has allowed for a sort of de facto centralization (not a huge problem, though in practice it sort of is, given how much the server buckles under the weight of all of the users; might be solved if it's ever moved onto a more performant implementation?),
the lack of encryption by default is iffy (though done for understandable reasons).
To lend credence to the specification being bad:
"I tried writing a matrix homeserver but I couldn't figure out how the state resolution process works" said by the author of Audacious/one of the authors of Pleroma/much more, so certainly a person who should be able to understand something that simple:
I know multiple people who've tried to implement the spec but who have gotten stuck on things that should be simple but are unclear if going by the spec.
> The specification as-written is really, really bad
We get a pretty wide range of feedback on the spec fwiw: some people seem to really like it. Others say that it’s “really, really bad” which doesn’t exactly give us much to go on...
> Synapse is kind of... and Dendrite is stalled
We have no choice but improve Synapse currently, and while it is still quite a resource hog it’s improved by at least 3-5x over the last year. Dendrite instead has become more of an R&D project for future homeserver shapes, but it’s not entirely stalled.
> The matrix.org server causes de-facto centralisation
The server has been less than 50% of the visible network for several years now - and ironically the datacenter perf issues (unrelated to Synapse) we had over the last few months have shifted that balance further - it’s about 35% and dropping. Ideally we will turn it off entirely once we have decentralised accounts.
> lack of e2ee by default is iffy
Our main project right now is to fix this. Cross-signing is mid flight; E2E search is done; Pantalaimon (E2E compat for dumb clients) is done; remaining key distribution bugs are in flight. We’re aiming to turn it in by default in Jan.
> “I couldn’t figure out how the state resolution works”
State resolution is the main technical novelty in Matrix, and yeah - it’s hard, much like git’s merge resolution is not exactly easy either. Unfortunately it comes with the territory; if you want to have consistent room state while replicating them over a byzantine network of servers to stop hijacks and other abuse, you have a relatively hard problem to solve. We got it wrong the first time; the current version gets it right (as far as we know).
It looks like you guys are on the right path for most things, and I wish you luck!
Why do you have no choice but improve Synapse?
One of the reasons you seem to only get negative opinions along the lines of "really, really bad" is because "The way it's written is stylistically bad/unapproachable and an entirely different approach should probably be taken" is something that takes a few hours to diffuse into something useful, and not something most people really receive in a way that's productive. It takes a lot more effort to give truly constructive, workable criticism than expressing any sort of opinion, negative or positive.
> It looks like you guys are on the right path for most things, and I wish you luck!
thanks :)
> Why do you have no choice but improve Synapse?
There are quite a lot of Synapses out there right now, and Synapse's featureset is pretty mature. Given we have pretty high profile people running Synapse (e.g. all of the French government) we have no choice but to invest the time to polish it and make it more efficient and stable. In an ideal world we'd have paused on Synapse ages ago and focused purely on a next-gen server, but we can't leave Synapse users in the lurch. Dendrite will be getting some attention in the coming months though, but not as a direct Synapse-replacement, but more of a playground for more experimental stuff.
> It takes a lot more effort to give truly constructive, workable criticism than expressing any sort of opinion, negative or positive.
True. fwiw, my guess is that kaniini was trying to implement state resolution before we published the first stable Server-Server API back in Feb - where it most certainly was the weakest part of the spec (and had design flaws too, which is why we hadn't bothered speccing it in detail).
Others have better responses to the rest of these points; many of them are still significant issues though a lot has progressed over the last couple years. I can preempt one though by saying alternatives to Synapse like https://github.com/matrix-construct/construct are probably something more aligned with your interests. (disclosure: I started the project with kaniini).
After more than two years in and around this space I'm still comfortable with stating that I'd take Matrix with all^wmost its flaws over ActivityPub. All of these protocols kind of suck. I see Matrix as having the lowest hamming distance between where it is and where it should be.
> I'd take Matrix with all^wmost its flaws over ActivityPub
protocol-wise, which flaws are currently the most egregious from your pov? (i.e. `all - most = ?`) (And if any are security issues, please chuck them to security@)
I'll just pick one. Matrix is resource-heavy, even in the best of times. There is a confluence of smaller contributing factors, and several effects on both its usability and developer/platform friendliness. Consider this anecdote of the overhead of sending the message "hi" as a baseline: in Matrix, this takes about 1KiB on average after all is said and done in JSON (and varies, unfavorably). So to over-simplify, that's 1K to the disk, 1K to each server in the chatroom, for which there could be several hundred, and 1K read on query from clients and other servers -- all for a 2 byte payload.
Before I'm accused of being unfair because there's always going to be a ridiculous ratio for something like a "hi" message, consider some alternatives for both the format and payloads of the fundamental protocol primitives. Most of the messages contain cryptographic metadata which has a succinct binary representation, but JSON requires base64; when represented naturally with CBOR the overhead can be reduced ~40%, and without encoding/decoding either. Consider that virtually all of the overhead in "hi" is either cryptographic hashes or signatures or integers (depth/ts) which would benefit from compact formats.
Does representation actually matter though? Why can't I just store Matrix in my format and federate with your format? I guess this is an example of where theory and reality collide. One cannot write a server which handles messages as abstractly as possible (leveraging these JSON/CBOR extensible formats) while at the same time knowing which fields have a more efficient binary representation and transforming that. In reality that just looks like a CBOR message with a bunch of base64 plaintext strings. It doesn't achieve that 40%.
All of this is important because of Matrix's (superior) design over its event abstraction. When the whole protocol is hinged on fundamental primitives (a good thing) attention to detail and focus must be given to those primitives. The more optimized they are, the more everything built with them is also optimized. When the foundation is efficient, developers can do more with matrix and enrich the user experience. For another example, matrix has been reluctant to give developers the power to store shared-program (i.e bot) state efficiently in a chat room. This is because there's no mechanism to delete state_keys, or even discard overwritten state itself. That's an important cornerstone that's missing while the rest of the tower is being piled on.
It might be possible to confuse any of these issues as trivialities, and their solutions as bike-shedding. I contest their importance is evident in how they emerge to shape the character of the entire system. Consider: does matrix require the entire DAG to be acquired like a bitcoin block-chain? Then I better be careful and conservative about messages. Does matrix allow gaps in the DAG and have a smaller chain for auth? Then I can be liberal about messages and delete stuff later. When I go to build an application which communicates over Matrix those qualities emerge as fundamental limitations or liberations of what users can experience.
So i think the summary is that the SS API needs to support alternative encodings (and thus signings) than JSON. Totally agreed; as you know we've already been experimenting with compact representations including CBOR (https://matrix.org/blog/2019/03/12/breaking-the-100-bps-barr... etc) which applied both to CS & SS API, which took the ratio down massively. However, we fudged it there by disabling the hashes entirely (given it was on a trusted private network). So from my pov Matrix is not remotely tied to the crappy HTTPS+JSON transport for S2S API in the long term, and it's only a matter of time before someone concretely proposes something better.
I selected some examples because I don't want to be lost in abstraction, but please, I am asking kindly that you not focus on my examples (unless it's pressing) because the issue here is systemic. I'm not giving these examples for you to just retreat back with your head down and address them. The locus is more fundamental: the etiology for why these errors even exist and why there continue to be new ones all the time; all at great cost, perhaps even existential cost. You should be concerned with that bigger picture, surely.
Scores of other issues could have been cited, but I included the representation example having confidence you'd probably tout those exact experiments and respond with that blog post. Your conclusion at the time was that CBOR and JSON had negligible differences. The company then proceeded to de-prioritize any serious work in that direction while simultaneously ramping up efforts for end-to-end encryption by default. This was a flawed conclusion and it has demonstrably added to your tower of debt. Even my "hi" example is already passé because now that too is another giant base64 string adding to this jumble. You can't bolt Ferrari body panels on a Honda and call it a Ferrari.
I can spend all day citing issues just as poignant, and you might even be kind enough to elevate them against the ~10,000 other [github] issues they're contending with. I can spend all day unpacking the string of missteps leading to all of your engineering disasters. It's not going to have any returns- not for either of us. It doesn't root the problems.
If I could impress upon you some serious advice, it would be to value people less transactionally. Don't estimate a person's worth by how they fill a few open gaps in your grand master plan and then dispense with them as if they won't be filling in any more. Avoid that by valuing people by how they think, and then don't try to think like them yourself. Delegate to them instead.
i think the tl;dr here is that we have a bunch of tech debt, and we’re not working on the things that you think we should be working on, and that we undervalue your input and should delegate to you rather than discussing specifics.
i’m afraid that nobody is going to delegate to you, but you are welcome to contribute via concrete spec proposals, which are assessed based on merits (and whose review is far from dominated by me; if they were my own proposals might move along faster than they do). if the spec process is too slow and doesn’t work for you, then feel free to spell out the proposal however you like.
Who am I to submit a proposal to fix fundamental problems in your company? That's called consulting. You just raised an 8 million dollar Series A you are soliciting me to work for you for free, in the leaf comments of a Hacker News thread? Are you for real?
I volunteered at least three solid examples of fundamental indecisions that have unambiguously cost your investors alone millions of dollars over several funding rounds; lest I mention also cost every poor soul that has volunteered their free time in the name of FOSS -- an ethically ambiguous embrace by you at best; every one of them is locked into the cost of your missteps.
I have clearly linked these indecisions to real-world outcomes. It's all history by now. The foundations of state resolution weren't in place when you built Communities/Groups so you didn't use the matrix chatroom; you built an entirely new inferior HTTP protocol without any state or replication whatsoever over several months. Now that you finally prioritized fixing that underlying problem with rooms there's a need to replace and spec an actual Groups system. In the meantime nobody can develop Matrix Groups into their software, not my server, not clients like Ditto, etc, because you tell everyone not to. You're going to overhaul the whole thing. You're doing them a favor by saving them that wasted effort. But Synapse and Riot use Groups and Matrix promotes it on matrix.org; still there's been no serious visible progress toward groups-as-rooms.
This is why Matrix is indeed a de facto centralized system. Let's not even get into issues in this system where every server trusts matrix.org for cryptographic keys. Matrix is really centralized around you.
In the end nobody is working with you. I have the only third-party implementation of this stack. Nobody is working with me either, but that's called the transitive property.
You have some time to think about this. I would say until the next recession hits. Then this is over. The way you do things won't survive that.
I've contribute to Mastodon and run an instance. Yes, it's very silly to think you can use Activity Pub for non-public things .. but isn't that going to be true of ANY distributed social networking protocol?
Direct point-to-point we can make secure (relatively) yes. You can run your own e-mail server, setup up PGP keys, and once you make it through that usability nightmare, you now have secure e-mail (if you trust your hosting provider) that will probably get dropped as SPAM 1/3 of the time when e-mailing anyone on Google/MS/Yahoo.
There's an interesting case where Mastodon and Pleroma respect and forward delete requests. Say you have a thread started on BigInstanceA and I reply from PersonalInstanceB. Someone from IAC replies to the same thread and B sees it, but A can't because C is banned from A. The reply gets dropped. You now have two conversations not in sync.
Say A deletes the original. B gets the delete request, but C never does, since it's a banned instance from A's perspective and doesn't exist ... even though it has a full copy of the deleted thread.
Deletes are truly fucking silly in the AP world. As much criticism as Zuckerberge gets (and deserves), back in the day he just wanted everything public, and I'm kinda for that too. If I post something online, it's never personal. If I need to talk about something personal, I call a friend, or text them, or meet them in a bar or plan my next vacation to be in their city. Real conversations don't happen in this space.
Not necessarily, if you use an actor-model where content is addressable using cryptographic keys, then you can get an object-capability system where having a reference to a piece of content is sufficient to grant access to it. This seems to be the way that some people are going with newer designs.
Also, the way you ban people is basically by having unique capabilities handed out to everyone. Then you can do various things to stop access (it's called a revoker in ocap jargon). See http://habitatchronicles.com/2017/05/what-are-capabilities/
Perhaps ActivityPub is struggling because it tries to be too much. If it does well at being Twitter, why must it also be Signal messenger? I don't use DM's much at Twitter, if that feature wasn't a part of AP/Mastodon, I wouldn't cry!
That comes across in delete requests. There's no way to guarantee that a server would honor that request. I don't think a blockchain style solution is needed -- just don't try to offer things that are impossible to offer. (There's this idea that AP/decentralized will be Nazi-free... what a ludicrous promise! Although, perhaps that's more an expectation of the users than the protocol authors...)
I would guess it's based on the fact that when most people hear "access control" they think of "access control lists", but it's possible to do decentralized access control. That's what ocaps are.
ActivityPub can be private in the same way email is private. All the people in the recipients chain will have a copy of the object you're sending. If in these recipients there's the Public namespace, then yes, you can't talk about "private" any more, but outside of that ActivityPub supports private interactions.
As a followup on this discussion, I actually spent half a day today to add private messaging to my ActivityPub service. It actually works.
Use search to find Kaniini's threads about Rapunzel.
Also, no, it's not something inherent in distributed social networking as a whole (previous attempts have avoided it), and it's not even inherent to push protocols, but without a bit of thinking behind it, it's easy to collide with.
ActivityPub? Who cares about what JSON schema gets adopted?
Much more interesting is the why, not the how.
#1 reason to decentralize:
"Centralized enforcement of global policy to address abuse and misleading information is unlikely to scale over the long-term without placing far too much burden on people."
#2 reason:
"The value of social media is shifting away from content hosting and removal, and towards recommendation algorithms directing one’s attention. Unfortunately, these algorithms are typically proprietary, and one can’t choose or build alternatives. Yet."
#3 reason:
"social media incentives frequently lead to attention being focused on content and conversation that sparks controversy and outrage, rather than conversation which informs and promotes health."
> "social media incentives frequently lead to attention being focused on content and conversation that sparks controversy and outrage, rather than conversation which informs and promotes health."
This seems to run anathema to the concept of social media platforms being run by for-profit companies. Monetization absolutely mandates the need for controversy and outrage, as opposed to social welfare.
That’s the thing, I’m not really sure it does. Never-ending social media political flamewars increase engagement for a few people but drive others away.
Personally I deleted my twitter account and barely use Facebook any more for this reason. If there was a fun happy version of these networks that showed me stuff I wanted to see rather than stuff that annoys me, I’d use it.
Why else would you need to insert a blockchain into the solution? Scuttlebutt (https://scuttlebutt.nz/) is already one possible solution and they wouldn't have to reinvent some new protocol.
It's worth mentioning that Secure Scuttlebutt uses a signature chain, which provides many benefits of conventional blockchains without forcing a consensus (which is the part that requires "mining"). My experience with SSB has been very positive.
> A distributed social network or federated social network is an Internet social networking service that is decentralized and distributed across distinct providers (something like email but for social networks), such as the Fediverse.
I can have a single non-distributed computer system and have that single computer be decentralized by having the programs it runs be done through achieving consensus. Your definitions are a bit off because Mastadon and Bitcoin are both distributed and decentralized systems.
"Finally, new technologies have emerged to make a decentralized approach more viable. Blockchain points to a series of decentralized solutions for open and durable hosting, governance, and even monetization. Much work to be done, but the fundamentals are there."
All blockchains fundamentally do not scale. Global universal consensus and a global centralized datastore (ledger) is an anti-pattern found nowhere in nature.
If the Twitter team go for a blockchain stack, they're doing so for the mass surveillance and control feature.
Actually a hybrid approach is the way. Nobody wants to wait hours to download GBs of new gossip to get started. Streaming the gossip is necessary to make the technology viable.
I m surprised every time i see people here asking for the evil that is censorship.
Dorsey wants a decentralized system exactly because he won't be held liable for not censoring people. It makes sense business-wise and it's a win-win for humanity.
Twitter has become a trigger-happy-people-fest because each group knows they can weaponize censorship against each other. With systems that are uncensored , nobody complaints, e.g. people don't make a fuss that email is not censorious enough
All moderation is censorship, to some degree. Even a voting system like HN's has censorship-like qualities: it is specifically designed to make unpopular comments less visible than popular ones. And that is fine, I wouldn't want to only have forums or social media with zero moderation.
Censorship only becomes a problem when it is _unavoidable_, that is, you cannot choose to live outside of some entity's censorship influence. This is where things get complicated when you have giant entities like Facebook or Twitter, which are for many people difficult to avoid (and there certainly isn't an alternative to them).
The problem arises when these entities need to choose which messages to show you. Even without putting any conscious bias in this algorithm, there is a system there which prioritizes some messages over others in a completely opaque way, which is often game-able (bad-faith actors can abuse the system to spread their propaganda, for instance). So moderation is not merely necessary but _unavoidable_, simply because there is more content than you can be presented with, and unfortunately the huge power of moderating large social media networks rests in the hands of a handful of engineers.
Decentralization is an interesting solution here because it would make the social graph and the recommender system separate entities, with clients being able to choose which recommender/moderation system they subscribe to. The need for censorship wouldn't be gone, but with people being able to choose their own censor, the massive power imbalance is not quite as bad.
I think censorship is no longer censorship when it happens purely on the client side, it’s just filtering. As long as the tweets all continue to exist out there somewhere, someone can write a client to display them.
This will, I predict, make a lot of people unhappy because “omg there are Nazi tweets on my blockchain” but personally I’m happy with it.
i think there is a difference between suppression of minority opinions and censorship, which completely bans them. it depends on definition of course, but in its most common use, the word censorship is not subject to gradation
>I m surprised every time i see people here asking for the evil that is censorship.
I'm sadly not surprised. While HN does attract hobbyists like me interested in technology and how it affects society, it also attracts a lot of those who dream of building and controlling the next big data silo for personal profit. Permanently liberating an entire mode of communication into an uncensorable distributed system that can't be easily replaced due to the network effect (such as your example of email) precludes that path to profit.
> It makes sense business-wise and it's a win-win for humanity.
Does it make sense business-wise? What business will want to be on a system that has no moderation? It's just asking for your brand to be accidentally or intentionally associated with undesirable things.
The difference between Google's moderation and Twitter's moderation is, in the Google case, that the user is in control of what is "spam" and what is not (and can still consult the "spam" if any).
>I m surprised every time i see people here asking for the evil that is censorship.
You're posting this in a place (HN) that itself is not averse to deleting posts, shadow banning, manipulating the rank of articles, etc. Some have argued that HN is a good discussion environment precisely because of those things, though I don't necessarily agree. What I'm getting at is that you have reason to not expect sympathy for your position here.
On HN, everyone sees the same things - the same front page, the same comments, and so on. On sites like Twitter and Facebook, this is not the case. What's good for a site like HN isn't necessarily what's good for a site like Twitter.
This seems more like an appeal to emotion to justify the creation of censorship infrastructure than an actual solution to a problem.
I haven't heard of unsolicited CP being a common spam issue. And those who exchange CP intentionally can and do use encryption which makes such filters pointless.
Then you are not familiar with the Fediverse's patterns of establishing various levels of tolerance on different networks of instances for loli content (blah blah it's fiction blah blah) because otherwise it federates onto people's timelines and they're revolted. CWs do a lot to soften this.
The original argument seemed about illegal content. If you're just talking about unwanted porn, well, that's an argument for better local content filters, which is different from eradicating content at the source.
you would call that a filter, like spam, and of course it's essential and can be deployed in multiple points (before sending, between servers, at the client). Both users and server instance owners can have control of what they allow. There is little disagreement that these filters should exist, so i expect all server to implement them.
Censorship is about allowing e.g. governments to request takedowns globally. most servers will reject such lists.
I'd like to see moderation work like adblocking. You subscribe to whatever moderation feed you like, and it blocks tweets that the moderators decide to block.
If you don't like what the moderators are doing, you can select a different moderation feed, or start your own.
This way we decouple the moderation from the platform.
I like this idea. Another feature that could be helpful for users who are eary of group think thought bubbles is to have some kind of analytics on the most common types of words, phrases, and topics that the moderator censors.
Moderation's easy in a decentralized model: run your own server, and refuse to federate with servers that are run by (or used by) people you consider to be assholes. Server admins can easily band together to contain (or isolate) "bad" servers as most of the Fediverse has done with Gab.
> How is decentralized twitter supposed to solve misinformation?
There will be a space where different people and organizations can build filters that rate information for veracity and other traits. Over time these filters will develop a track record. People will be able to filter their feed through such filters.
Centralized orgs can already do this, but they make a central and opaque choice about it. This means that there is a lot of pressure from various interest groups to make sure that the hidden filter is biased towards their favored viewpoint, and the users cannot really see what the result of this fight is.
It's like how it probably wouldn't be a good idea to have just one news organization that served the whole US.
> It mostly makes moderation (or censorship) impossible.
Making total censorship impossible is mostly congruent with the goals of society in my opinion. Censorship has done immense damage, e.g. Lysenkoism in the USSR (to pick a particular example).
However there is still the option for users to voluntarily opt in to certain kinds of filters. Most likely most people would voluntarily opt in to a filter that rejects child porn, for example. It's an open question whether a decentralized system with filters would lead to better overall environment, but I think it's worth trying.
Same concern - I'm working on getting some info removed from mastodon. it's terrifying how hard it is to get someone (volunteer mods) to actually remove something...
This sort of case makes me an advocate for centralized access to moderate/regulate/also be accountable... but then if there's this magic backdoor to data access and removal...?
In fact, any app can be made to act as a source, sink or both of ActivityPub events. I recently added ActivityPub support to learnawesome.org so that reviews can be consumed in any ActivityPub client. Implementation was easy and the data model is quite easy to understand.
ActivityPub is real-time pub/sub for the entire Web, something that Twitter could have been.
https://w3c.github.io/activitypub/
http://activitypub.rocks/