A while back I built out a proof-of-concept distributed web based on webtorrents - https://github.com/tom-james-watson/wtp-ext. Once this change makes its way into popular clients, it makes the idea a whole lot more viable.
Unfortunately it looks like Mozilla's interest in the distributed web died when they made redundant the main stakeholders in the project last year - https://github.com/mozilla/libdweb/issues/109.
It's really fucking sad that Mozilla is struggling so much financially. They didn't participate on the iOS platform for a long time, then they burned a huge pile of cash on FirefoxOS, and now they have to somehow match the engineering resources of Google and Microsoft.
I personally work on a decentralized web project ... but even I can't fault them for shutting down side projects. Every project either needs to provide alternative revenue streams or feed into getting Firefox on par with Chrome's speed, security, and feature set. Hopefully Chrome will follow through with neutering their ad-blocking and people will switch back to Firefox.
When I was last looking at the libdweb stuff they didn't have WebRTC support working in the service workers to let you use it from a protocol handler (and they somehow didn't even consider that important as they insisted no one would even want to use WebRTC, which made no sense to me); did they eventually fix that so you could do WebTorrent?
You can implement your own protocol + handler with the functionality exposed by the library as far as I understood.
It doesn't really matter anyway since you have access anything firefox can do with webextensions experiments, which was required since libdeweb isn't a part of firefox(and won't be anytime soon because AFAIK most people who were working it has been laid off) and to run experiments you need firefox nightly.
Pretty much this. The code runs as an extension - it registers the protocol handler in the extension's background script. That means the code for webrtc has no need to run in a service worker.
OK, so as I don't actually do much in-browser development, and because this was also two years ago, "of course" I managed to get the terminology wrong: the issue was that WebRTC wasn't supported in a "WebExtension" (not a "service worker", which happened to be the subject of a massive thread I was reading a few weeks ago on WebRTC not working there either).
Here's the bug I had referred to the libdweb person at the time in November of 2018 (due to comment #2):
> RTCPeerConnection.createOffer doesn nothing in WebExtension background script
Which was supposedly fixed? I'm honestly a bit concerned still, though, as this bug slowly got dragged from being about createOffer to being about permissions dialogs for audio/video, and it kind of sounds like there's some weird thing where a "foreground page" has to vouch for the extension, which shouldn't be required. Everyone who works on WebRTC stuff in browsers conflates it all with getUserMedia, but all of us distributed web people just want peer-to-peer data channels :/. I take it this is actually fixed then, and you can now use WebRTC in this context?
For some fun confusion, there's this libtorrent library, commonly called libtorrent-rasterbar by distros. And libTorrent by rakshasa, which is used by rTorrent.
I've considered something like this for a web music player I have, but it doesn't seem to have an easy setup for downloading individual files from a torrent. And the latency would add gaps between songs.
You can use prioritize_files() combined with the sequential_download flag to "stream" a track (and prioritize_pieces() if you need the last chunk for an m4a file).
Schedule your fetches intelligently and it shouldn't be hard to keep a music player fed. 320kbps is only 40KB/s.
I've experimented with using youtube-dl as a backend before for my own music player. The latency to start is only about 2-3 seconds, slower than "webtorrent --mpv <torrent>" is for video.
If you have some centralized server for hosting the application anyhow, you're probably better off just using WebRTC directly, or for that matter, just serving the music files from there.
although from how I understood it, web seeding really assumes you can control the folder structure on the servers.
so you can't just curated a data collection from publicly accessible sources and freely map torrent contents to files on http for bootstrapping/reliability.
too bad, really. it seems like a cool thing to just make a torrent from e.g. public datasets that are hosted on flimsy university servers. make the torrent, distribute it, the swarm keeps the files available and reduces load on the university servers, and the server keeps the torrent from dying even if the dataset is niche.
There are two BEPs for web seeding. One is defined in terms of base URL and relative paths the other in terms of pieces and offsets into them. Few clients implement both web seeding protocol extensions.
Kind of a tangent, but how large or small is the risk of nefarious sites running torrent nodes without the user's knowledge or consent?
We've seen something similar with using JS to mine bitcoin. That gets you money, so that probably has a stronger incentive, but someone might want to gain access to the resources this would allow.
> WebTorrent clients running on one domain can connect to clients on any other domain. No silos! The same-origin policy does not apply to WebRTC connections since they are not client-to-server.
Does the browser do anything to protect the user against something like this? Is there a permission for network access, and is it allowed or denied by default?
> Is there a permission for network access, and is it allowed or denied by default?
WebRTC data channels are allowed by default.
> Does the browser do anything to protect the user against something like this?
A nefarious website can already use the fetch() API to upload or download massive amounts of data if their goal is waste bandwidth. WebRTC doesn't introduce any new vulnerability here.
unlikely. Torrents are slower, in most scenarios, than straight HTTP. With small files, they generate more traffic for chatting/handshaking than the file is actually worth.
It could however lower costs for people who distribute large files, where the final user can tolerate some download latency. I'm thinking browser games that can download a big archive via torrent on first launch, similarly to what Steam does now in their client.
A large majority of web clients must be operating on battery now, too. Willingness to burn battery participating in Torrent and other distributed protocols rather than HTTP will be low among those, let alone willingness to leave such a thing operating as a daemon in the background. It’s why I don’t see much future for these things (distributed file-sharing protocols) except in infrastructure or in applications where many users may be expected to be on desktop.
It would help lessen the cost of bandwidth for servers and smaller ISPs, as traffic is very expensive for anyone that isn't Comcast: saturating a 1 Gigabit line can cost an ISP ~$1K/month!
It’s also part of why so many things that’d be better on TCP sockets end up being implemented on top of HTTP... and then actually see adoption in that form. A combo of JS limitations and firewalls aggressively blocking everything that’s not HTTP.
Considering SSH is forbidden in most enterprise networks I've seen, your attitude is clearly in the minority.
Enterprise-IT people tend to shutdown first and ask questions later, to cover their ass (and in their shoes I'd probably do the same). If it becomes trivial to torrent via mainstream browsers, I expect they will lock down whichever feature is responsible or apply massive pressure on vendors to remove the feature.
Of course, but they still make an effort (and a policy that will be applied to you if found out). They have to, it's their responsibility. If suddenly half the network is saturated by people using browsers to torrent the latest movies, you can bet that the answer won't be "we need a bigger pipe"...
The enterprise networks I’ve been on don’t give endpoints direct internet access. You can only access the web through an authenticated proxy. This forbids SSH, unless you set up additional infrastructure to tunnel it in HTTP.
Removing the feature seems pretty unlikely, since most web-based video conferencing products rely on WebRTC. In today's remote work world, that would be quite the throwing of the baby out with the bathwater.
Enterprise IT people here. We cover our ass in true enterprise style by consuming website filter lists from a provider and MitM TLS. Doesn't matter if they use WebRTC or HTTP, as long as the website is in the correct categories.
Worst case we could disallow WebRTC via DPI and whitelist it for category conferencing.
The only problem I see here would be created by themselves.
The price they pay for postponing questions is losing WebRTC and those "serious applications" in the meantime.
I get the argument about blaming tools, but there's precedent here that suggests a ban could happen. Apple bans torrent-related applications from their app store despite the protocol ostensibly having nothing to do with piracy.
I think that's less likely to happen with WebRTC. But for workplaces that don't otherwise use it? Maybe.
I'm not sure I follow. Bittorrent is absolutely NOT a "serious" application from an enterprise perspective. Most companies don't use torrents and actively block the protocol (since the most common usages are unsavoury or related to entertainment, not business).
Whereas IMs are an accepted part of business processes (i.e. "serious").
> Whereas IMs are an accepted part of business processes
There was a very long period when IMs were not acceptable and actively blocked, especially at financial institutions that have a legal obligation to log and retain all internal communication for X years. I saw this at many different firms for many years.
That changed. perhaps torrent will, too. Nah, probably not :)
Everything can be blocked if you're willing to block everything else. Blocking something while leaving other services unaffected isn't trivial, and gets harder the smaller the data is. eg. if you want to block some bad guy from exfiltrating a 256 bit key, it's almost impossible to do because there are a million ways to do smuggle it out with stenography.
Enterprise IT already regularly installs their own certs on their boxes to MITM all TLS. The have the means to DPI and shut this down, though they'll need a firmware update for their firewall box.
This is great news and will help lead to wider adoption. I hope that some progress will be made for handling large files since currently it is stored in-memory: https://github.com/webtorrent/webtorrent/issues/86
I've been using Webtorrent Desktop for a while now, and really enjoy using it, but there are some glaring issues around random freezes on fast connections, plenty of open issues on it with other users reporting the same experience: https://github.com/webtorrent/webtorrent-desktop/issues
I've personally been working around this using a custom fork that messes around with the simultaneous connections settings, and it seems to help quite a bit, but I still get freezing from time to time when I download a large number of torrents simultaneously.
I wonder if an alternative implementation in something like Rust on top of WASM might make those kinds of issues easier to avoid by making more efficient use of resources.
Once WebTorrent is ubiquitous among desktop and headless clients, the networks will become a lot stronger. Especially for applications like BitChute and for anyone who copies their approach. It solves a major problem with the cost of baseline hosting vs. hosting at scale.
The problem is that many people will want to run bittorrent over a VPN connection, while they'd like to run their normal browser data over their normal (low-latency) internet connection.
You're assuming torrents are intended for copyright infringement which is unfortunately a big part of its usage right now, however if torrents become as seamless as HTTP (like not requiring a separate client) I can see them becoming widely used for totally legitimate purposes.
Torrents can be a great, standardized way of delivering updates for example, and network administrators can easily deploy a platform-agnostic way to deliver & cache updates on their network by just deploying a torrent client somewhere and their fleet would automatically discover them (using local peer discovery) instead of having to deploy multiple proprietary, platform-specific solutions.
> You're assuming torrents are intended for copyright infringement
I think they're just assuming that you don't want everyone to know exactly what you're doing online at all times by exposing your IP to everyone and their brother that uploads a resource.
This. It's a privacy nightmare. It essentially broadcasts to arbitrary other internet users find out what resources you've accessed. That's… pretty dire.
More flexibility in proxy configuration will make this easier to overcome, and there are already some plugins for Firefox which allows specifying specific proxies for specific sites.
Is this normal ? I know nothing about WebTorrent, but isn't the file supposed to be streamed from pairs, and not requested from a central server in an HTTP request ?
And leaks your IP address and makes it possible to pirate content for simply visiting a website. You could target someone with all kinds of illegal stuff.
It's already possible for a website to get your browser to download illegal content simply by visiting the page, and expose your IP in the process to whoever they want. You don't even need to use JavaScript to do the downloading (just a image or video tag, for example), and the uploading can be done with xmlhttprequest/fetch.
On the one hand it means the web moves (or can move) closer to decentralized modes of operation. Bittorrent is tried and tested and Works (TM).
On the other hand, it also means yet one more thing that moves into the freaking browser. We have enough things nowadays that are shoehorned into HTML/JS when really they would have been more usable even if written in tcl/tk.
Alternatively: every bittorrent client I have used (at least on macOS) is really, really bad. I used to use Deluge, which was buggy and slow to begin with. Then they stopped distributing an installer for it, so you had to install it with pip (!) and run it from the command line. God forbid I `brew upgrade python` or my bittorrent client will probably break.
Transmission crashes for me, and has little activity. Sometimes it just can't open torrent files.
The web doesn't break. You can load a URL, get some decent software with an easily-improvable UI—and assuming you're on a modern browser—expect it to work with no installation, in seconds. And if a site pushes an update, I hit refresh and get the update without needing to run a 45-minute `pip install` command.
fwiw, I've used qBittorrent on osx and found it to be usable. I'll admit that was a few years back, so maybe they (read: apple) broke that one in the meantime.
Oh, also, coming out of an hour long intercom session trying to explain to a customer rep how "no, I didn't change anything in my firefox, your site just suddenly doesn't log me in any longer" I'll posit that the web breaks all the time and all over the place, in many small, annoying ways.
Alternatively, it means BitTorrent is now supported by any number of esoteric operating systems, rather than just the ones the developer using Tcl/Tk chose to target :)
I'll let that count, but only because of the nice snideness of it. :P
Seriously though, you'll be hard-pressed to find a platform where you can't run at least rtorrent or such. and if you're running some esoteric unix from way back when, I expect you to be able to compile it for that yourself. :P
Is there a good book that explains all the BT technology? I’d like to use or build a robust client/server but most learning materials seem to be trivial or daunting.
Unfortunately it looks like Mozilla's interest in the distributed web died when they made redundant the main stakeholders in the project last year - https://github.com/mozilla/libdweb/issues/109.