Hyperswarm is a Kademlia DHT with hole-punching builtin. It's designed to arrange p2p connections. For context as to why you'd want this, when you create a p2p connection (like with WebRTC) you typically need a "signaling server" to arrange the connection. Hyperswarm replaces that server with a distributed signaling system. We built it for the dat protocol, but it's reusable in a lot of other contexts. (In fact, the creator of the Manyverse mobile app which released yesterday [1] has voiced an interest in using to Hyperswarm [2].)
Author of both here. I need to update the README of hyperdht, but we basically just iterated that a bit and moved it hyperswarm/dht to as it's part of the stack now. I'll update the hyperdht README to reflect that.
Curious for more details on how the dht signaling / hole punching works. I'm working on a peer to peer chat and file sharing system right now and upnp / nat pmp are pretty wack. I'd be interested in using it if there ever is a Go port.
Had y'all considered using any of the js-libp2p (i.e. ipfs) stuff for this? I have used Go DHT lib w/ success and have used the connecting (not participating) part of the JS DHT. I will admit the dozens of JS modules all interconnected in the loose libp2p way is a bit annoying as is the fact they aren't all completed yet.
Also, was any effort made to make this work in the browser sans browserify? Meaning, can I replace the networking component and maybe put in my own webrtc connector impls and still leverage all the kad stuff ala peer buckets/closeness?
The majority of Hyperswarm's code was written before libp2p. We had been sitting on it and waiting for the right time to deploy. We're not against working/merging with libp2p, but since we do like being able to iterate quickly within our team for now.
This project isn't possible in current browsers. Browsers just don't expose the interfaces needed. That's why we work on https://beakerbrowser.com/, which is the main project that Hyperswarm was built for.
The lack of browser support is the biggest stop to getting PWAs to better decentralized/robust-offline.
Stronger PWA support would help a lot, especially in trying to minimize the number of wrappers like Electron and Cordova in the wild.
Certainly most of the interfaces needed aren't high on web platform priorities, and it may not be Beaker's intent yet to push on WHATWG to adopt these technologies more platform-wide, but even the ability to bootstrap the DHT in an ugly, but entirely browser-capable fashion (a dumb REST API bootstrap call to known HTTP endpoints, as one off-the-cuff idea) could do a lot for the kinds of decentralized applications that could be built entirely as PWA with no other application wrapper needed.
libp2p seems to have this goal in mind on their issue tracker, but so far as skimming the issues, I don't think I've seen a timeline for releasing it.
> This project isn't possible in current browsers. Browsers just don't expose the interfaces needed.
While I agree on the networking front, with pluggable bidirectional networking impl that others can write, the DHT still has value.
I am familiar w/ the Beaker project and definitely appreciate all of y'all's work (I did a much smaller PoC back in the day [0] when I thought MaidSafe had legs). I also definitely understand/appreciate the simplicity of some of the hyperswarm code and also understand fears of overabstraction for external use cases.
I'd be interested to hear other perspectives on this, but I think p2p computing in general got skipped over when the cloud became a thing. DHTs have had steady use in the p2p world though, including with BitTorrent.
DHTs are not without their problems. I know some other projects in our space decided it wasn't tenable to solve the Sybil attack and so they created a peer-routing system that bootstraps off of a blockchain. As brad0 pointed out, there have also been performance issues in some deployments. We're going to iterate on the security and performance and see how it goes.
> I'd be interested to hear other perspectives on this, but I think p2p computing in general got skipped over when the cloud became a thing
There are few commercial benefits/incentives for a DHT provider and few guarantees p2p tech can give a company requiring an always-on central presence.
> I know some other projects in our space decided it wasn't tenable to solve the Sybil attack and so they created a peer-routing system that bootstraps off of a blockchain.
One approach is to do the S/Kademlia thing where you require all peers to generate IDs (hash of a key) and require the it start with a certain number of 0 bits and then require another random X that, when xor'd w/ the ID, starts with a certain number of 0 bits.
> There are few commercial benefits/incentives for a DHT provider and few guarantees p2p tech can give a company requiring an always-on central presence.
There are quite a few commercial benefits! It's just that nobody's properly banked on them yet. P2P distributes costs, simplifies configuration & ops, and improves data privacy. For the politically-minded, P2P also distributes power more equitably by moving business logic to user devices.
P2P will not remove the need for always-on presences. You need a peer of last resort. However, running a peer will be cheaper and easier than running a server.
> One approach is to do the S/Kademlia thing where you require all peers to generate IDs (hash of a key) and require the it start with a certain number of 0 bits and then require another random X that, when xor'd w/ the ID, starts with a certain number of 0 bits.
We're looking at the crypto-puzzle approach, but there needs to be a work-requirement asymmetry between honest nodes and attackers, which I think is only the case with the Eclipse attacks and not the Sybil attacks. Otherwise, the puzzles only provide security if honest nodes outspend attackers, and we don't want to turn our userbase into mining farms.
That said, it seems like "what makes it better" is that it hasn't received a take-down because this isn't attempting to be a solution for copyright circumvention ;)
1. https://news.ycombinator.com/item?id=18065567
2. https://twitter.com/andrestaltz/status/1044973699940659200