It isn't based on WireGuard, but is a true mesh network, unlike everything you've listed. No central coordination point, every node is equal.
Think of it as BitTorrent with a few initial peers you set up through a config file, and it learns of every other peer at runtime (like BitTorrent does through Peer Exchange).
It can forward traffic through other nodes (like Tailscale and unlike Nebula) and recomputes the most efficient route as peers become (un)available.
I use tinc myself and like it. I like that I only have to update a single node and all the other nodes will accept those peers from that node. I've been playing with wireguard, and having the other nodes discover and communicate with each other directly doesn't happen without you adding the other's public key to both peers.
I remember there was talk about switching the backend for Wireguard for security and speed, but it seems to be on the backburner: https://github.com/gsliepen/tinc/issues/179
I really like it for my use case but just assume it's not secure at all. I change the default Ubuntu install to not run the daemon as root (a major flaw in my view) and then just assume that network is as insecure as any other. It's really useful to have all my personal machines always connected between each other at the same IP independently of network config. Security comes from SSH/TLS on top of it.
Considering that the wireguard mesh solutions seem to require static IPs, I'm now wondering whether it might be a good idea to run a wireguard mesh on top of tinc, basically just using tinc as a way to get static IPs and routing connectivity between nodes. That still leaves the issue of updating the wireguard configuration, though.
It seems that the value-add magic sauce of TailScale is the work they've put into NAT traversal etc. There are some interesting blog posts from the team regarding work in this area.
I wonder if the work they put in can be extracted to some kind of tool the other projects can use as well. NAT traversal always seems like the limiting factor in projects like this.
NAT traversal is an impossible problem in practice in the general case (boo symmetric NATs), and the best-effort version is a solved problem since at least 2003 with the STUN protocol[1] which is part of WebRTC, and there are STUN implementations available in many languages[2].
I had enough problems that I had to stop using it, though when it worked it was fantastic.
Client intermittently had seizures (at least on Windows) and I couldn't join/unjoin networks properly. Would have to restart the service and close the program for it to begin to work again.
Sometimes the traffic just didn't flow ... at all. Randomly couldn't connect to other clients on the network.
I haven't had to use it much since covid started as I'm mostly at home these days and I used it to access my LAN when I was out.
I have tried to use it over mobile data and wasn't very lucky, and that's something that used to work great. I couldn't tell if it was due to ZT or the carrier. There's CGNAT on both ends, which could be getting in the way, although like I said it used to work great.
I have also noticed that it takes longer than I was used to get IP, find routes between nodes, etc but once those are up it works well.
I feel like my whole life got simpler with less headaches when I discovered ZT. I adore it. Maybe there are other things out there which do the same thing but ZT is the single 'it just works' thing that has made my life easier the most.
Not associated with them in any way other than being a user.
... and "public" VPNs, where all you need is the network ID to connect. Seems great for playing LAN games. It also has a userspace implementation, which I was able to use on bitbucket CI/CD where I couldn't bring up a tun/tap device, to connect via SSH to a private server. It also has Ad-Hoc networks that are public, IPv6 addressed, and based on port-range, so you can expose your SSH server to anyone with the IPv6 address, for example.
A major differentiator for the (many) mesh net projects is whether they are GPL (or similar) or BSD. The latter can be distributed in the iOS/iPadOS App Store, so have a lot more potential value for both hobbyist and business end users. Of course, if it's mostly a server-to-server mesh net with few end user devices, then it hardly matters.
is a building block that can be used to achieve something similar, minus TURN-like NAT traversal fallback. The peer discovery leverages DNS and UDP hole punching uses WireGuard itself.
I made this: https://github.com/naggie/dsnet/ -- a simple command to manage a wireguard VPN. Think wg-quick but quicker: key generation + address allocation.
https://codeberg.org/vula/vula aims to thwart surveillance adversaries by automatically building wireguard tunnels between LAN hosts with zero configuration using existing IP addresses and .local hostnames inside and outside the tunnels, and uses CSIDH to provide post-quantum protection. Unlike these other projects, it doesn't (yet?) do anything over the internet, and as a result doesn't require a server or any infrastructure. If it is installed on the router, internet-bound traffic is also protected on the LAN.
Everyone mentions tailscale but they want to charge you per user. Their client is open source, the thing the client connects to is not. I think there are better options.
https://github.com/juanfont/headscale replaces the tailscale control panel with something open source and self-hosted. However, it doesn't have all the features of the official tailscale control panel.
Wireguard based solutions get all the attention these days, but I really love working with Nebula. It's dead simple to set up and the mobile client works well.
That said, Nebula feels way less popular, and I don't know if it's had a third party security audit. WG's popularity means it should presumably have had a lot of attention given to both its architecture and its code. This is one area where using a "niche" solution feels somewhat risky.
However, the Android client does not allow for specifying a DNS server and breaks the Android system DNS over TLS. Google thinks the VPN app/client should handle DNS and Nebula app development seems very slow. This means you cannot use adblocking DNS while using Nebula.
The semi-official (worked on by folks involved with the core product from, as far as I can tell, early on, but not released by Slack like core Nebula is) mobile clients don't have a license associated with them, which is a bit of a blocker to further development or distribution on certain stores (FOSS-only ones, for instance, and really no-one else can distribute them at all right now without risk).
> Thankfully both Nebula and Wireguard are built on top of the Noise Framework.
> So neither is a compromise security-design wise.
Security involves much more than the crypto framework being used; just crypto requires much more than the framework. Noise could be implemented insecurely, as a basic example.
Are there any simple tools to manage a small WireGuard VPN with a handful of servers and something like low double-digit number of clients? By servers I mean Linux servers/VMs that expose some web services and clients are simply desktop/notebooks that need to access the servers, but don't expose services to the VPN.
From the description, this tool isn't what I want. I have configured a simple version by hand with Wireguard, but it did seem like raw WireGuard is too low-level for my purpose and my networking knowledge. Setting up the keys does need some level of automation for anything larger than a sandbox. And I didn't figure out a way to resolve domains within the VPN without interfering with the DNS on the clients for everything else.
I've never used it myself, but a sister comment mentioned related projects working with wireguard and innernet seems to be exactly that use case from a casual glance.
You can specify full cidr rules, so it should be possible to prohibit connections between nodes by segmenting the IPs
There are easier options around if all you wish is ssh access to your servers though.
Personally I'd recommend gravitational teleport, mainly because the name is so hilarious
Teleport also has some really neat approval workflows where you can say "2 of these 3 people need to approve a request for people in group Y to access servers in group X". The incoming request can ping you on slack and then you go to the teleport webui (U2F authenticated with SSO) to approve.
It also has some neat stuff around eBPF session recording if you want to be able to playback user sessions for high security environments. Teleport is good tech.
Bothers me that I can not set up a backup U2F stick.
Also, I had issues to configure it for my situation with no static IP and behind a proxy. I have it working but not really exactly how I wanted it. Maybe its my lack of understanding, I'm sure it could be made to do exactly what I want somehow.
All said, it pretty nice but I reverted to using the FIDO2 based keys sticks directly.
You might want to look at tailscale.com - I'm not 100% certain I understand exactly what you want - but it does allow any client to expose services; assuming non-server wireguard/tailscale-clients don't expose services/have firewall set up -I think it does what you need?
The problem is people keep thinking WireGuard is a VPN when it's more of a network tunnel + routing, like OpenSSH's -w option. A "real" VPN has a ton more features to provide the user experience needed for typical VPNs. I believe OpenVPN is still king here, probably one or two others as well.
Agreed. At a minimum it would be good to provide a way to protect wireguard credentials at rest, so they aren't just sitting out in the open on disk as a single factor for access to the network. If I had more experience I would love to submit a patch for the default client software, but I don't have much experience, and you don't want inexperienced crypto programming, especially for new features....
Router Nodes is what I would really like to have, I want a node that doesn't have internet connection, but in the local network with another node to get access to the internet through that node. Wasn't able to achieve it with plain Wireguard, my linux networking kungfu is weak.
EDIT: thanks, I'll try mentioned projects, Yggdrasil seems most interesting.
tinc supports this since forever, FWIW, and is fully open / self-hosted, no ties to a central server at all (so one node going down does not take your whole network with it).
I omitted some details, there was a third node that I was trying to use as an exit. It was a while back, but whatever I tried wasn't working, if it is trivial for someone else it wasn't for me.
For someone who has no idea what this means, can someone please offer a simple explanation? When I read "mesh network" I think about those routers with modules you can use to get good WiFi coverage at home. This sounds more like connecting your devices across the Internet in some private manner. What would you use this for?
WireGuard is a VPN protocol. There are roughly two kinds of VPN. Those that you use to hide who you are when accessing the Internet (route all your traffic via the VPN to an exit node) and those that you use to build a private network over a public one for e.g. a company or other organization.
Wiretrustee seems to be for generating a private org network. Could be personal, work, circle of friends or anything. Most traditional work VPNs have one or more nodes that all traffic pass through and that all connect to. A mesh VPN instead allows VPN peers to send traffic directly to each other over the public Internet. There seems to be a central coordination/signaling server, as with tailscale, headscale and innernet. But that is mainly used to find where the traffic should be sent, not for sending all traffic through it.
I use it for connecting my tiny SMTP ingress machine to my big SMTP backend machine and doing the ClamAV scanning over the VPN link (because it's now far too heavy for the tiny SMTP machine these days but the big machine has plenty of resources free.)
I don't get the obsession with awesomelists. Seems like every time I go to check them out, there's five different lists for the same subject, and none of them are kept up to date. Just seems like someone's weekend project that gets immediately forgotten.
Currently running our corporate VPN on Wireguard, would highly recommend. Wiretrustee looks like a very useful featurset on top of Wireguard, nice work.
Correct, peers must have a static IP. We could imagine to build a small extra tool to automate regenerate the config in case of IP change but could be a bit tricky :)
Looks good. I use ansible to automate WireGuard setup between my nodes (n=5, m=5, m being personal devices such as phones, iPad etc and n being servers) and it works well. Can’t automate the part where I’ve to manually scan a QR code for phones though.
I would love for this to be integrated into Consul Connect so all our systems just run in one mesh and all internal traffic is encrypted without even thinking about it.
The promise of most mesh nets is that they're (more or less) a self-configuring distributed VPN. Give a node some credentials and (usually) a few real, publicly-routable IP addresses to "bootstrap" nodes, and that's it. It's on the network.
There's no "real" network that they provide a gateway to, necessarily, just a bunch of nodes, potentially none of which are on the same physical or logical local network, aside from the mesh network itself, which is an overlay network—think of an overlay network like a virtual machine, but for networks, incidentally: a protocol stack built on top of, or within, if you prefer, another protocol stack. You feel like you're talking TCP/IP over it, but under the hood your TCP packets are carried in other packets (may or may not also be TCP—often it's UDP when possible) over the normal Internet, then reconstructed back into this virtualized, if you will, packet stream on the other end.
With a traditional VPN, you'd connect to a real device that provides a gateway to, say, a corporate network, which may have routing connecting multiple locations but is essentially a normal network. Mesh networks operate much more peer-to-peer than that, usually leveraging known peers to find routes to others that are (for whatever reason) unreachable or initially unknown. The "mesh" part refers to the routing, mainly, the job of which is to search out and find good paths to nodes. If "corporate" has fiber cut by a backhoe, "the VPN" isn't down—many, many nodes might be, but the network might survive in a useful form.
I'm a huge fan of Tailscale, and in an effort to understand better everything they've done under the hood to make it work, I decided to redo my network with native WireGuard.
A) WireGuard works great when it's all set up! Config is a bear but it's a great protocol. Works well on lots of different devices in different usages - Android, laptops, desktops, headless servers, IoT.
B) I have even more respect for Tailscale for doing all of this in such a polished, seamless way.
We’ve been using Tailscale and it’s been excellent. I’d like the ACL system to be a bit more robust but I totally get that They’re putting their energy into a seamless experience with the nuts and bolts of the actual tech first and it really shows in how solid it has been for us.
Assuming this tool results in two peers connected over WireGuard (no relay server), the UDP performance should be great. Probably fine with a relay server too but I can't say definitively.
Good answer!
Wiretrustee uses kernel Wireguard if available, therefore the performance is the same as plain Wireguard.
There are some caveats though. We use a fixed mtu value of 1280 to be on a safe side while wg-quick dynamically detects the best value. We will improve it.
If the Wireguard kernel module isn't available (true for older Linux, Mac and Windows) then userspace implementation is used (wireguard-go) which is slower.
- https://tailscale.com/
- https://github.com/juanfont/headscale
- https://github.com/tonarino/innernet
- https://github.com/slackhq/nebula
Something I'm missing?