I keep hearing about Scuttlebutt for a long time but haven't seen anyone actually using it? Why?
edit: hm, the recommended application on the website itself is ... from 2021, and the github of the app itself is archived now. And there is "bleeding edge" one ... from 2019.
Scuttlebutt had the Bitcoin like problem of requiring everybody to have a local copy of the entire database, with a confusing / glitchy "syncing" process to download the latest posts. I tried to use it for a few days on my Android, but it never showed any new posts and the syncing process took like 30 seconds every time I opened the app.
Admittedly my only exposure to the Secure Scuttlebutt Protocol(SSB) is through manyver.se [1] - a FOSS private(optionally) social media app. And the syncing issue you mention has been one for me too. Sometimes it's even worse - it fails to sync entirely until another app restart. Maybe this is a manyverse issue, maybe a SSB issue. Either way, hitherto at least, it doesn't seem reliable.
If there are alternative ways to have an offline(over Wi-Fi/bluetooth, not internet) social media/messaging app, that works on both Android and ios, please do let me know.
At a certain point in the history of SSB the manlyverse team took over the maintenance of SSB and the apps locking until the end of a sync became the norm. Prior to this there were websocket-based web and Electron apps where you could at least see that progress was being made.
SSB always sank from the first post to the latest. IMO it's better to sync from the latest backwards.
Not disagreeing with you, just nitpicking on the use of a word:
What you are describing is simply a signed Merkle tree, not a blockchain. You cannot really call an individual signing it's own ledger a consensus mechanism. Maybe "validity" or "authenticity" would be better terms here.
I know a lot of people make this mistake of labelling anything that has a Merkle-tree like structure a blockchain but it's impractical when you need to talk about the technology rigorously to have a word that meant something precise suddenly encompass so many things, including tools such as Git.
Coming back to Scuttlebutt, even the website of Manyverse, the mobile client, explicitly says "no blockchains", so it seems at least some of the people involved in Scuttlebutt development agree with this.
Many of these systems also don’t have a Merkle tree, they have a Merkle linked list. Which I think contributes a lot to the syncing problem. A skip list structure where each node has multiple hashes/identities would let you skip the last five years of edits and pull down only the individual edits for the last month.
The “Merkle tree” I worked on before this blockchain nonsense started was a Merkle DAG, representing a mostly linear state flow with some parallel elements.
> A skip list structure where each node has multiple hashes/identities would let you skip the last five years of edits and pull down only the individual edits for the last month.
Do you have any references or reading material on this pattern? Would love to learn more.
Not in crypto, but it’s a linked list structure with a logarithmic number of additional pointers that refer forward (or in the case of a diff algorithm, backward) into the list, to provide closer to log n lookup time.
I think they were invented by Bill Pugh, a professor who also had some noteworthiness in Java concurrency fifteen years ago, which is how I heard of them.
So the way I see it, you’d overlay the linear tree with one that refers to a single, recent hash, a much older hash (or another meta node of the same degree) and a roll up delta from that older hash, with a new hash that satisfies both. Then every time the number of records double, you add another layer over the previous one.
> In cryptography and computer science, a hash tree or Merkle tree is a tree in which every "leaf" (node) is labelled with the cryptographic hash of a data block, and every node that is not a leaf (called a branch, inner node, or inode) is labelled with the cryptographic hash of the labels of its child nodes. A hash tree allows efficient and secure verification of the contents of a large data structure. A hash tree is a generalization of a hash list and a hash chain.
If you have hashes co-located with data, and data spread throughout the chain, it is not a merkle tree.
A blockchain is a linear chain of hashes of the content in each block of the chain. In a blockchain, for any given block, there is only one valid next block.
When a blockchain is propagated over a network, there is a consensus algorithm used to agree on state of that chain. Bitcoin uses proof-of-work to agree on the next block. Anything that allows multiple nodes to reach consensus on the state of a blockchain is a consensus algorithm. SSB uses proof-of-signature, and the sequence number, as its consensus algorithm. The network reaches consensus on the state of any particular chain by validating the signatures and that the sequence numbers are in the correct order.
Colloquially, blockchains have been conflated with their consensus schemes. When people use the term blockchain conversationally, they generally mean a blockchain that propagates over a network using some sort of consensus algorithm. This conflation of consensus with blockchains has lead to many people in the p2p space rejecting the term blockchain to distance themselves from the crypto hype bubble, even though the data structure itself is quite useful.
Finally there is a Merkle-DAG, introduced by Juan Benet, that acts as a unifying data structure for both blockchains and merkle-trees. Merkle-DAGs can also express novel data structures that are neither a merkle-tree nor a blockchain.
tl;dr: a blockchain is not a merkle-tree, and a merkle-dag might be capable of representing a blockchain but they are separate concepts. A blockchain is independent of its consensus algorithm, although colloquially the two have been conflated.
P.S. Merkle-DAG might be capable of representing git, would be fun to give it a try sometime using something like IPLD.
> The network reaches consensus on the state of any particular chain by validating the signatures
Ok I can see that. It's the distributed consensus problem in a centralized setting. It still feels strange to call that a blockchain but maybe it can be qualified like "centralized blockchain" (which questions the need for such a structure to begin with).
> "centralized blockchain" (which questions the need for such a structure to begin with).
I’ve seen centralized blockchains used to great effect in ledger contexts where the chain is not synced, but references (hashes) of the chain are shared externally (I.e. transaction ids).
If an external auditor has a set of transaction ids that have been communicated externally, they can later audit the centralized chain and prove it has not been tampered with without having to have the full log of blocks.
> If you have hashes co-located with data, and data spread throughout the chain, it is not a merkle tree.
Yes that's the theoretical structure but in practice its usage almost always implies data co-located with the intermediate hashes, if only because it's used to validate the integrity of a tree of files and directory and it's easier to manipulate a single structure with all the information. That's how it's done in Git internally, as well as in IPFS for example. I believe it's also how Merke-trees are used in some backup/synchronization tools.
> In a blockchain, for any given block, there is only one valid next block.
Yes, but that's just a particular kind of tree, where each node only has one child.
> Yes, but that's just a particular kind of tree, where each node only has one child.
Yes!
I’m personally building a SSB data structure that forks into a tree, forking to allow writes from each new writer, to allow multi-device writes! It is a multi-writer append database that behaves like a CRDT.
It’s a blocktree, not a blockchain. Which is also expressable as a merkledag.
Yeah, sorry, when I said "entire database" I was handwaving over "the entire database of posts relevant to them". The Merkle tree structure is the root of the scaling issues which is why I think it's fair to describe it as "bitcoin-like" and it serves as an intuition pump for people to understand why it was so difficult to use on mobile, regardless of whether it was actually quite as bad as Bitcoin or not.
A true Satoshi whitepaper Bitcoin would include everyone having a local copy of the database as well as peer-to-peer connections. No necessary middlemen. Is this correct.
A Bitcoin of my dreams would have close to zero if not zero transactions cost with everyone on the network running at least full nodes fully synchronized with the main network. The fact that it costs money to send some Bitcoin from one wallet to another is sad.
Social networks are strongly dependent on network effects. For a time SSB was actually used by some of my friends, but it required a desktop app, had a confusing onboarding process, IIRC had some rough edges around syncing content, and was just not smooth enough to compete against highly funded centralized companies in the cuthroat social networking space.
Everyone here is focused on the tech and I'm here wondering how I'm supposed to convince my friends and family to join me on something called Scuttlebutt.
The name was the part I liked best! "Scuttlebutt" is a naval term for a water keg, and it became slang for conversation around the watering hole. In other words, it was gossip around the water cooler from before we had water coolers.
...and this is why 99% of niche developer attempts don't flourish into the next big social network. Might as well call it "Codswallop", which is also a valid historical slang that is equally completely ridiculous for mass appeal.
It was never meant to. It was founded by a guy who lived on his boat and wanted something for his friends. It became a hotbed for experimenting with P2P social networks, for a time.
...and this is why 99% of developers will defend for-profit companies stealing data, enabling mass genocides and foregoing all privacy: because everything must become the next big XXX.
Append-only logs are fine as long as they only contain pointers to content.
You can store 10k blocks in the size of a JPG image.
And, when trees grow large, with stale data that has been forgotten by the network, you can do garbage collection/compaction by shaking out the dead references and sharing a new tree.
It had a major technical flaw in that it depends on some weird, Javascript-only JSON-serialisation bugs (iirc), and these basically had to be reimplemented in other languages so nobody really ended up writing any other clients for it.
Oh Scuttlebot. Such a great idea, and for a time it actually worked. I haven't been actively developing with the codebase for years, but I believe the current status of the project is that there were too many code rewrites that simply broke things that previously worked at one point or another.
I tell people sometimes that I'd put together a team to do a retro Scuttlebot revival with a Patchwork-style app and of course git-ssb! If you're interested in this kind of project email or call me. Contacts are in my bio.
I'm mostly looking for devs who want to dive into an old Node.js codebase, who want to recapture the magic of the original protocol, and then upgrade all of the modules to modern standards. Maybe Deno or Bun even.
Not sure if you're aware of this but mycognocist has continued a rust implementation and it's really getting there in terms of "what one would need for a patchwork style application"
I've also been toying with this, exactly for that kind of unapologetically desktop-only "classic" client. Suffice to say I'm also a bit butt(hehe)hurt about how the desktop client story worked out after manyverse decided dump the original protocol some years into the effort.
Yeah, the guy who renamed it to "Glimpse" is now worldwide famous for solving the only problem with the project. Designers are dropping Adobe Photoshop in droves and giving the project the complete recognition it rightfully deserved.
I wanted to hate on it, but I just downloaded the latest version and I see that after like 20 years, they finally realized that each tool window doesn't have to be a separate application.
Good for them, for once I might be able to try actually try it.
Rumor has it, Microsoft balked away from a billion dollar acquisition of this source control and project management tool because they didn't want to be associated with a collaboration portal for gits.
Scuttlebutt has a context and a meaning behind it. People might hear it without knowing the context, think "that it's a weird name, why is it so?", and then judge whether they like it or not. But to just hear the name and claim that it is inappropriate/bad/not-to-be-used because it reminds you of something that has nothing to do with the original context is ridiculous.
Codemonkey became codemirror. And it is now quite well established (in chrome dev tools for example).
But usually FOSS projects don't get renamed at all, or if they do the renaming causes so much drama that kills the project. So nothing changes, also not obscure design choices - and they remain niche.
You should probably ask the maintainer those questions. All I know he did made that decision at some point(before it took off) and he likely had a reason. Probably to be taken seriously, as he wants to live off his work. (I think he can)
And greasemonkey and tampermonkey are not really mainstream, nor do they make money as far as I know.
I searched a bit to see if I could find myself the answer, and what I found instead is that there is another project called "code monkey", which seems to an educational tool to help people learn to code. That would explain the name change a lot better than "the author changed the name because he wanted to be taken seriously".
Besides, the original criticism of Scuttlebutt was "it will never catch on because it has a silly name", not "it will make money". By making it about "making money" you are moving the goalposts a bit, aren't you?
"Besides, the original criticism of Scuttlebutt was "it will never catch on because it has a silly name", not "it will make money". By making it about "making money" you are moving the goalposts a bit, aren't you? "
Operating a social global network, even decentralized, is not possible without money, if you are aiming to get mainstream adoption(which was the main question/goal here). But you are very welcome, to proof me wrong.
No offense, but you're repeatedly using the term "midwit", which demonstrates that you have a higher-than-average tolerance/liking for non-layman vocabulary, and so might not be the best judge of what the average person is going to accept. Most people are not going to bother looking up the etymology of a word before dismissing a product named using it. Personally, I'm not a fan of the name because it's a word that, like "jitterbug", I associate with old people, because the only people I've known to use it in conversation are over 65.
Orgs spend millions on naming/branding because they know a name that doesn't hit right to the ear is going to be at a disadvantage. The vast majority of folks aren't going to bother giving your product a second thought if it doesn't appeal to them or has a negative connotation in their head right off the bat, regardless of the name's deeper relevance or origin.
My point (in relation to the top the of the thread I responded to) is that there are other things beyond the name that are more important/relevant when evaluating why the product wasn't successful.
If Scuttlebutt had an amazing feature that had a viral component to attract people, was easy to use and had no roadblocks for distribution, it would likely succeed, regardless of the name (like github).
But it's not. Scuttlebutt is a complex system, requires users to understand concepts like public keys and distributed systems, content discovery literally involves chasing other people around the internet to spread the gossip to you, the client was primitive, the development team had a size of one, etc, etc, etc... Those factors are way more relevant to understand why it hasn't taken off. Even if Scuttlebutt had a perfectly "appropriate" name, it wouldn't matter.
To put it another way: "Glimpse" is a fork of GIMP, and the author forked only because of the name. How many people actually cared enough about it to switch? How many people left Photoshop and started using "Glimpse" because that was the tipping point? Or the inverse: look at the whole Twitter -> X "branding". How many people left Twitter because of the name change vs because of the other changes since Elon took over?
Aye, and most people who use social networks are midwits!
Butt in the name will absolutely make the project a joke and cause it to misfire. As a fellow midwit I only just heard of the project and dismissed it before I got to the technical pain in the arse you describe it being in other comments.
Butt, lmao. Now Mastodon has disowned the term they should start calling their messages toots.
> I get not liking a name, but to pass judgement on the quality of a product/project based on its name alone seems like a complete midwit thing to do.
I get to make n-many "midwit" decisions in life and retain a certain success envelope / gradient. I'm happy for this to be one of them.
Just think about all of the everyday decisions being made just like this. People passing over your resume because they don't like your font, etc. This kind of petty little thing bears an absolutely astronomically impact on the deterministic evolution of the universe.
Why? I think it's a fine name, better than most, because it's a normal English word that says exactly what it does / what it's for. You're not just saying that because the word happens to end in "butt" are you? Or am I missing something?
> You're not just saying that because the word happens to end in "butt" are you?
They're saying precisely that and they're 100% right, and it seems you're missing a lot of things.
It's a social network. Its only value is in it having people using it and coming back.
Just try and imagine any person in your IRL circles (who's not on HN) actually using a social network with butt in its name. I seriously can't believe you see no issue there whatsoever. A teenager. A young artist. A YouTuber. "Follow me on TikTok, Instagram, Scuttlebutt and Poopypants". I dare you to find a creator willing to say these words.
Or if you just hang out with smart, like-minded tech-savvy middle-ages people who see past these things, then great, we have a new "social network" for smart, like-minded tech-savvy middle-aged people.
I mean, I wouldn't expect problems if the thing were called "Buttercup" would you? Or what about "Button"? "Saturday" (which has "turd" in it)? Are there problems with a Harpoon ("poon") IPA?
Clearly the presence of a sequence of four silly letters isn't in and of itself disqualifying. Native speakers often won't even register the juvenile bit inside it.
My only surprise here is that "scuttlebutt" doesn't work that way for other native speakers. It must be just my dialect or circles then, because it's an ordinary word for me which subsumes the "butt" within it (like I'm certain "button" does for you). But I guess it's not universally like that and the word is odd or rare enough that it doesn't do the same for others.
> I think it's a fine name, better than most, because it's a normal English word that says exactly what it does / what it's for
Just because the name says exactly what the product does and what it's for, that doesn't automatically mean it is good. In fact, imo it is bad in a lot of cases.
Do you think Instagram would have gotten as popular if it was named FlexGallery, or if Nextdoor was named SpreadRumorsActVaguelyRacistForum, or if Twitter/X was named PopularityContestShortMessagesAndBadNewsSourceSpaceWithSomePorn?
I personally would say "no" to all of those, and it isn't just because the names get unnecessarily long and ugly. McDonalds would never rename itself to Cheap Passable Fast Food Restaurant.
Some things really can buck the trend - ChatGPT is a _terrible_ name but provided so much obvious utility that it managed to reach a large consumer market despite that.
I mean, it's an English word, for sure. I have never once in my many years heard someone actually say it seriously. Maybe a handful of times when someone was purposefully trying to be quirky.
A name that makes many of your friends smile ironically is not a name that would induce a network effect, except maybe in marginal communities ("bitards").
A name must not be even remotely silly, else it will be laughed out of the (already crowded) room.
https://www.manyver.se/ is one of the leading implementations of Scuttlebutt. They're currently working on a protocol replacement called PPPPP which addresses some of the shortcomings they identified, the leading one being storage requirements and growth.
SSB was designed by an ocean sailor to be a social network that was infrequently synchronized and then only by local connection. The idea being, when sailors went ashore or rafted up for a gam to catch up on the scuttlebutt, they could synch their databases and have something to read on the next leg of their journey.
Blockchain tech was used to assure that messages created by others had not been altered in transit.
SSB ran into design limitations when the more recent developers tried to imitate something like Mastodon. It can work, but it's definitely idiosyncratic. It does tend to grow a large db, but space is rather cheap these days.
He is pivoting into a new protocol which is intended to preserve the community aspect of SSB, while providing important factors, like the need to delete previous posts.
The last 3 times I tried to use it the ~5 projects you had to run/put together to get any higher-level usable application were always in some way incompatible with each other, as they always seem to be in a constant mode of refactoring. Figuring out a set of compatible versions looked like more like a dark ritual than anything else.
I'm not sure. The last time I tried using it, I had a similar experience.
Nobody was ever able to coalesce a coherent UX/UI around ssb, the core contributors at the time weren't interested in scaling the network without outside funding.
I was working on implementing lighting fast discovery for ssb using a distributed BitTorrent-like tracker. Getting onto the ssb network took way too long. Still, there were not many users on the network, and nobody seemed interested in growing.
Scuettlebutt is an incredibly important data structure.
Its horribly under rated and horribly under invested in.
Its core data structure is a personal blockchain whose consensus scheme is a proof of signature and a proof of history (through chained hashes). There is no global consensus, identity is fiat as it is in the real world.
This makes the data structure gossipable and verifiable.
It has scaling challenges.
It needs to be turned into a blocktree to support multi-device support.
The content needs to be moved out of the tree.
But most of this is solvable with very little effort.
Scuttlebutt is an overlay network for the internet that enables us to build a truly open web.
Its one step away from making humans, their devices, and their content, addressable for the web in a meaningful way.
That small data structure removes the need for a central server.
The future will remember what dtarr did on a sailboat.
My favorite social network! Took quite a bit of problem solving to get in, and a shift in expectations to handle the wait times for p2p data transfer, but the high walls make for a chill community.
I really don't see why decentralised social networks are any better privacy-wise than centralised. If big corp has financial incentives to gather and use your data, then smaller servers do, too. At least big corp is a legal entity that you can sue.
Scuttlebot users encrypted messages onto their append-only log towards recipients using their ed25519 private signing key converted to an ed25519curve encryption key.
This made it possible to send private messages that you could open if you had the private signing key that the messages were directed towards.
Back in the day you could actually send messages using "Private Box" to up to 7 recipients! by appending them to your feed of course.
I recall reading somewhere that it was created by someone who lived offline and had to take a boat to get connectivity, so that's why the thing has this "sync to local device" structure. Can anyone corroborate?
> Today, Tarr lives on a sailboat—another Kiwi staple, alongside sheep and distance. Connectivity is worse on the boat than on the farm, and even less reliable. But that’s by design rather than by misfortune. Tarr started living on the boat after burning out at a previous job and discovering that the peripatetic lifestyle suited him. Unreliable and sporadic internet connectivity became an interesting engineering challenge. What if isolation and disconnection could actually be desirable conditions for a computer network?
I'm happier just running an Elgg instance on a cloud server and inviting the people I know to post there. 10-12 people sharing shit they found elsewhere and starting conversations is more than enough for me.
Kinda too late, it's a 10 year old project. Besides, scuttlebutt is the name of the water barrel on a ship, and is synonymous with gossip. As a gossip protocol originally written by someone who lives on a ship, I think it's pretty fitting.
This matters little. The majority of potential users live on land and take water from a tap. The subtlety if the naming is lost for them. But the schoolchild-level silliness of it persists and repels.
I think I understand the weird comments about the name now: you're thinking it has something to do with "derriere". But it doesn't, because the people who came up with the name would never have thought of that. Instead they'd have called it "scuttlearse".
Why? If someone will not use an app because it has a silly sounding name, or because they're so immature that "butt" is objectionable to them, I'm happy not to have them bring that attitude into a social space I'm part of.
All people are rather different. No matter who you are, the art of popularity is the art of attracting people who are unlike you. Just because most people are unlike you, and unlike each other. Yes, this assumes removing some nice edginess, and appearing slightly generic.
The whole design of the protocol is anathema to popularity. It's based around syncing posts with users you are in physical proximity to. I doubt the cute name is what's going to hold it back from mainstream adoption.
Scuttlebutt in america is a very common word used to describe gossip.
I love the parallelism between 'scuttlebutt' and 'water cooler talk' .. they both describe gossip and they are both named for people standing around a source of drinking water telling stories.
edit: hm, the recommended application on the website itself is ... from 2021, and the github of the app itself is archived now. And there is "bleeding edge" one ... from 2019.
OK I can see that there might be problems