Hacker News new | past | comments | ask | show | jobs | submit login
Innernet: open source Rust based Tailscale alternative (tonari.no)
265 points by ricklamers on March 29, 2021 | hide | past | favorite | 70 comments



This is awesome, thanks for sharing!

I really like the simple client/server architecture, that it's easily self-hostable and there are no servers outside of my control.

The invite system reminds me of the way Tinc[1] handles it, which is great. It's so good to see user friendly tooling on top of WireGuard.

[1]: https://tinc-vpn.org/


I have been working on wrapper around Tinc to simplify setup. Project: https://github.com/reddec/tinc-boot Article: https://dev.to/reddec/tinc-boot-full-mesh-vpn-without-pain-3...


I love this tool a lot! For now tinc still has many advantages over wireguard and this makes managing it a lot simple. Thanks for all your work!


I have been using Tailscale personally on all my machines and it's really cool.. Thanks for creating an OSS version of it !

Request to HN floks : I can setup a basic home network, but I want to really learn networking ( Setting up subnets , understanding CIDR, etc ).. Where should I start ?


I found interactive CIDR visualization tools like https://cidr.xyz/ to be very helpful in understanding the notation.

I also end up using https://gitlab.com/ipcalc/ipcalc a lot, and am definitely planning on similarly making it easier in the terminal to manage and visualize the CIDRs in innernet networks. I'm hoping innernet can become a fun way to learn networking in a safe (and cheap) virtual environment.


I took a CCNA 200-301 class on udemy to learn all that stuff.


https://www.davidc.net/sites/default/subnets/subnets.html

This is the best subnet calculator, since you can split and join subnets visually.


If you have a lot of free time, Computer Networks By Andrew S. Tanenbaum is top notch.


Check this excellent guide by 3COM, Understanding IP Addressing: Everything You Ever Wanted To Know

https://pages.di.unipi.it/ricci/501302.pdf


This look great, thanks for the post and open-sourcing the project.

I played around with Wireguard directly and having better ergonomics without vendor lock-in is great, i look forward to taking it for a spin.


I cannot wait to start using this as it looks like it will make vpn's a heck of a lot easier to manage. Here are my two questions:

1. Is it possible to use the same subnet on different innernets?

2. Could you please provide installation instructions for generic linux, as I am looking to host on almalinux and opensuse leap, neither of which use dpkg.

Thanks for sharing!


> 1. Is it possible to use the same subnet on different innernets?

As moviuro mentioned, no, not unless you want to get fancy with independent network namespaces (https://man7.org/linux/man-pages/man8/ip-netns.8.html).

If you want to be more confident of not having an address space conflict, I recommend using a randomly generated private IPv6 block using the RFC 4193 specification: https://en.wikipedia.org/wiki/Private_network#Private_IPv6_a...

> 2. Could you please provide installation instructions for generic linux, as I am looking to host on almalinux and opensuse leap, neither of which use dpkg.

Our Arch PKGBUILD (https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=inner...) may be the simplest existing guide for making your own package for your distro. If it's not a lot of work to add, I'm happy to maintain other package formats, or help you be a maintainer.

Thanks! Looking forward to hearing how it goes for you.


Regarding 1.: no. AllowedIPs in the WireGuard world have the double meaning:

* I expect anything coming from that peer to have an IP address in AllowedIPs

* I know that a peer's AllowedIPs can only be reached through that specific peer

Also, RFC 1918 lists 2^24 + 2^20 + 2^16 = 17 million IPs. You should normally not have an issue finding networks that don't overlap.

10/8, 172.16/12 and 192.168/16

Also, using the same net on different interfaces would probably confuse your server. Maybe if you do some magic with VRFs you could. (And it doesn't mean you should)


How does one update an oldschool web admin page to interact with this model?

For example, suppose Kermpany had already been up and running for a while with a standard Django website running behind Cloudflare with an admin page at example.com/admin/.

Now the things in the blog post have happened, and Kermpany wants to make sure that only machines on the "humans" CIDR can connect to the routes hosted at example.com/admin/.

What happens next? Does the admin tool move to a new domain?

The manual wireguard solution I know of is to add the example.com IP to the list of AllowedIPs, so the wireguard interface gets used for all requests from the local machine to example.com, and then restrict the /admin/ route in nginx to just the wireguard server's IP. But that takes a lot of bookkeeping and I feel like I'm missing something.


I don't think there's another way than the two ways you've already described.

IMO, moving the admin tool to a separate domain (and server!) would be the best option here in terms of security. That way, there are clear boundaries and fewer attack vectors and you also don't need to do as much bookkeeping.


Somewhat unrelated but in case anyone from tonari.no is reading this: I've been following your project for some time[0] and would love to know more about it! Please please update your blog more frequently and maybe even upload some demo videos! :)

[0]: Ever since someone posted https://news.ycombinator.com/item?id=23540586


We have some more blog posts planned :)

Is there anything in particular you'd like to read about?


> We have some more blog posts planned :)

That's great, looking forward to them! :)

> Is there anything in particular you'd like to read about?

I'd love to get a better impression of what your setup, well, feels like. Specifically, three UX questions have been going through my mind lately – probably because I've been working from home for far too long haha :( and having a portal like yours would be fantastic!

1) How important, in your experience, is it for people to look (or have the impression of looking) into each other's eyes? What ways have you thought of that could accomplish that? Do you maybe even happen to have a solution? My gut feeling has been that being able to hold eye contact would go a long way towards making people on each side of the portal feel "close" to each other. So if this were possible somehow, his would be a game changer.

2) How would one do conference calls with your setup, UI-wise? (Let's assume for a moment that bandwidth is not a constraint.)

3) What about eye contact (question 1) when there's a group on each side of the "portal"? It seems almost impossible to achieve with a 2D screen.


Addendum, now that I've had even more time to think about this:

4) General question (I realized I should have asked this first): Where do you currently place the camera? Putting a webcam on top of a 27" monitor already makes holding eye contact challenging but I imagine it's even worse with a large screen where you put the camera at the very top or bottom because the camera will always film you at an angle.

5) In the context of question 2, the question of eye contact seems to be quite complicated, too. I imagine that at the very least one would need arrange all participants of the call in some fixed spatial layout that's the same for everyone but the camera angle still remains a challenge.


> Where do you currently place the camera?

If you look closely at the images and videos on our website, you'll be able to spot it. It's often a fun game we play when giving a demo to someone new. Many people take quite awhile to find it and the AV nerds point it out almost instantly, haha

https://tonari.no/

But yes, with the current setup you can't get perfect eye contact in all scenarios. It works surprisingly well in multi-person conversations though, as you can clearly see when someone shifts their focus on you vs. someone else next to you.


> But yes, with the current setup you can't get perfect eye contact in all scenarios. It works surprisingly well in multi-person conversations though, as you can clearly see when someone shifts their focus on you vs. someone else next to you.

Sweet, that's good to know!

> If you look closely at the images and videos on our website, you'll be able to spot it. It's often a fun game we play when giving a demo to someone new.

Hmmm… you made me have another, closer look this time. :) In one of the photos it looks like on the other party's side there is a projector near the ceiling in the back but then in subsequent photos that projector's light is gone and there's a different, stronger light that seems much closer to their camera/screen. Is this correct?

Also, have you considered/tried using an actual LCD screen? At our company we're currently dabbling with digital signage devices (for entirely different purposes) and the big ones are quite impressive. I could imagine that, in terms of dimensions and appearance, they would work quite well for your purposes.


> In one of the photos it looks like on the other party's side there is a projector near the ceiling in the back but then in subsequent photos that projector's light is gone and there's a different, stronger light that seems much closer to their camera/screen. Is this correct?

In those cases those are just normal overhead lights, nothing from a projector. The camera is actually in the screen!

> Also, have you considered/tried using an actual LCD screen? At our company we're currently dabbling with digital signage devices (for entirely different purposes) and the big ones are quite impressive.

Yeah we definitely want to move to panel displays when it becomes more feasible to do so. We haven't yet made a prototype with an LCD screen but I look forward to the day when we can, because projectors are a lot fussier to deal with.


How to use the same low latency tricks as the Tonari mirror in other applications. E.g. cloud gaming, conference calls


Like @chaz6 i'm really interested by tutorial/docs for example to use software on a RPI4 / nixos as an alternative to Tailscale :)


This is amazing! I've been thinking of writing a system for managing a wireguard network in rust for quite a while now but I'm still happy this popped up. Perhaps it would be nice to expand it into a bigger ecosystem with UI based interfaces for different platforms.


We'd love your help if you're interested in it! Supporting more platforms and making graphical frontends are high on the priority list.


Could someone help me out? I understand how Wireguard and Tailscale work. But I don’t understand the various ways they could be used for personal and business use cases apart from a workaround for geoblocking. Could people currently using Tailscale chime in?


My personal use case for tailscale is to connect my home PC, MacBook Air, Raspberry pi, iPhone, iPad and my VPS into a single network without running a VPN server.

The best part is that you don’t have to open any ports on your router and it “just works” out of the box.

So even if I’m in a different country I can use the nextcloud app on my phone to connect to my raspberry pi in a secure manner and backup photos, which in turn backs them up to my VPS.


Do you have a blog post on how did you do it? I am very interested in the same use-case.

If you tell me "it's easy, just read the docs" then I am going to cry.


Unfortunately no. I pretty much just installed and configured my gmail account on all the devices. Only tricky part was my VPS because it didn’t have a GUI.

I don’t use magicdns yet since it’s in beta.


I'm using Tailscale for managing an old Mac Mini I have in the office, that does some simple 'cron' tasks. But it's behind NAT, but with Tailscale, I'm able to SSH into the box regardless, which is really nice. I believe it's also possible for instance to have Tailscale running on your servers, and only expose port 22 on the Tailscale interface, which is also quite neat.


We don't use Tailscale, but we do run a Wireguard server. It allows our remote workers that need local network access. For example we run an internal kubernetes cluster that is not accessible over the internet. And we also have SMB shares that a remote worker might need to look into.

Basically the use of a VPN is to add an extra layer of defense. Instead of finding a security hole in our Kubernetes management interface or samba server, an attacker would first have to compromise a workers system, and only then attack our internal servers.


Instead of VPN'ing to the cloud to access my servers i just installed tailscale on one of the vms in the cloud and exposed all the routes. I work on 5 different pcs, no more up and down vpn.

The only thing missing from tailscale for me is being logged into multiple accounts at the same time.


Not using Tailscale but knocked up a VPN between my various servers to allow e.g. my tiny mail frontend to use the extremely heavy ClamAV on my big mail backend without having to expose it to the wider internet. Same with allowing all the servers access to each others' Redis's for synchronisation of things like rspamd whitelists (easier than setting up Redis replication, for me.)


How name resolution is handled? I am going to play with it tonight, but it isn’t clear how is DNS setup. Is that something completely separated, or integrated on the tool? Thanks!


Currently it makes non-destructive edits to /etc/hosts. It would be interesting to explore more scalable ways to handle this for larger networks though.


You could run a local DNS server using something like https://github.com/bluejekyll/trust-dns . Or, you could install an NSS module to resolve names via the innernet client.


DNS seems to be the weakest point of many of these overlay network products: hamachi, zerotier, tailscale, and probably this one all use some convoluted nonsense despite the fact that split dns mechanics exist out of the box now in Windows, MacOS and Linux.


> split dns mechanics exist out of the box now in Windows, MacOS and Linux

Could you elaborate more on what split DNS options are available out of the box on the various systems? I'm genuinely curious what a no-convoluted-nonsense DNS setup looks like for this kind of tool.


Innernet appears to use /etc/hosts on the clients and updates it from the server when peers are added/removed etc.

To configure split DNS on the server, you'd need an authoritative server listening on the wg interface and update the zone file when peers are added and removed.

To configure the split DNS on the clients:

Linux: systemd-resolverd supports split DNS see https://fedoramagazine.org/systemd-resolved-introduction-to-...

MacOS: Supports similar split DNS by domain/interface, see resolver(5) for the details.

Windows: I'm not a windows person, but it appears that Windows has trouble and you need to play with interface metrics, one solution was to install unbound and use it instead of Windows' native resolver.


Windows has NRPT which is mature and works across all applications. It actually appears to be rather more capable in some ways as compared to what systemd resolver or macOS can do. I have been using it with regular Win10 clients for about 18 months now without any trouble.


Windows does not have a comprehensive solution to split dns. The vpn api hacked on something that resembles that, but the core networking stack doesn’t really support it. Windows Server can be configured to handle split brain DNS but that’s a different matter altogether.


Wouldn't Name Resolution Policy Tables (NRPT) be the comprehensive Windows solution?


So (the last time I checked) the actual code is there but there are no sanctioned user/developer-visible knobs/APIs to tweak NRPT on client Windows SKUs and only limited configuration specifically limited to the resolution of accepted client DNS requests is available on Windows Server.

I believe there is basically a optional network stack path that utilizes NRPT but it’s not the default path, so only MS-blessed code that is actively using NRPT (so the new VPN stack and their DNS server) actually works with it.


NRPT has worked fine for me on regular Win10 clients ever since I first started configuring it about 18 months ago. With the exception of apps like Firefox being configured for DNS-over-HTTP and bypassing the OS resolver entirely, it works in all applications and with all the interstitial resolvers in virtual environments I have had occasion to use (VMware workstation, HyperV, Docker, WSL1/2, etc.)


I assume "split DNS" requires extra software anyway and is going to update `/etc/resolv.conf`. So far I have been using `dnsmasq` which is pretty common in the industry although not something seamless.


No; support is baked into the default OS resolvers if you like, but obviously you can also use outbound software to achieve it too. The stuff in /etc/resolv.conf have not been "the whole story" in Linux name resolution ever since systemd, and at times has really tripped me up.


Wow, so neat! Thank you for developing this, and open sourcing it.


Looks cool!

Nebula has mobile apps. Might this be possible for Innernet in some distant future? It seems like no if it needs root.


All these seem nice but I never seem to find one that will interface with rootless nodes, amd link back to wherever my laptop is. There's always a requirement for a kernel module or mounting a new interface. For now I use chisel, but it's a hack and I need to manage addresses and ports manually.


A userspace implementation would use a lot of battery power on laptops and be less performant. Wireguard is now in the Linux kernel and does not require kernel modules. You will however require root privileges. I don't want an unprivileged user to be able to route all my traffic though some tunnel. Changing system wide routes should require root.


Isn't the tool linked at the bottom, Nebula [1], capable of running without root?

Personally, I'm not surprised in the slightest that messing with networking requires administrative privileges. I don't really understand the use case for rootless nodes or how they're normally managed, but I can see how those are too niche for most networking software to work with.

I've never tried it, but perhaps you van get Innernet to work with the usermode WireGuard client (the one written in Go [2]), that seems to work on Android without any kernel support or root privileges. Innernet is a daemon over the wg command line tool so it might work out of the box?

[1]: https://github.com/slackhq/nebula [2]: https://git.zx2c4.com/wireguard-go/about/


Just a heads-up: there's an old command-line news program named "inn" that might produce name conflicts for the innernet command-line tool.

Fortunately, it sounds like that's just an alias, so people who care about "inn" can just spell out "innernet" or make a different alias themselves.


Thanks for the heads-up. Before adding the alias, I did a quick search (https://packages.ubuntu.com/search?suite=xenial&arch=any&mod...) and did see the "inn" package, but it didn't seem like it included any "inn" binary that would cause name conflicts.

Happy to change the way that alias works if it ends up being a problem.


Ah, sorry. I checked and saw the cron and init scripts were called "inn", and missed that the binary was called "innd". Nevermind.


I’m going to try this because tailscale is extremely slow when you want to fully utilize your bandwidth.


I think tailscale uses a userland TUN/TAP interface[0] which negotiates at 10MiB/s; that’ll be the largest bottleneck and likely applies to Innernet too.

Tailscale does use considerable CPU on my Mac though.

[0]: https://github.com/tailscale/tailscale/blob/main/net/tstun/t...


Latency and bandwidth was a big issue for us - innernet uses the WireGuard kernel module on Linux when available, which is about as good as you can get (easily achieving saturated gigabit line speeds).

macOS is a different story, since there are only userspace implementations at the moment. Innernet currently looks for the official "wireguard-go" implementation, but you can swap out userspace implementations as you like. I'll add an environment variable check to make that easier without needing to recompile.


This looks great!

Is a TCP mode planned? This would be useful for networks where outbound UDP isn't allowed. (hotel wifi, other public wifis)

Do you plan to add automatic key rollover/expiry?


Thanks!

There aren't any current plans to bake in TCP support, but you can rig it up yourself using something like udptunnel.

Related old HN comment with basic instructions: https://news.ycombinator.com/item?id=17847008

Also see: the "TCP Mode" section in https://www.wireguard.com/known-limitations/.


I know about those. Since the whole point of innernet is making wireguard configuration simpler, I figured maybe integrating something like that is worth doing.


Ah, gotcha - sorry I misunderstood your question.

I feel that adding TCP support would push innernet beyond a WireGuard configuration manager and into something a bit more behemoth. I'm quite fond of the fact that innernet doesn't "touch the packets" in its current state.

That said, if a strong need arises over time it's not out of the question, and in the mean time anybody is welcome to add their own wrapper around innernet or fork it to support that.


Do you feel Innernet could one day work through QUIC? It's kind of neither TCP nor UDP (although definitely much more UDP).


Aye, noted. Do you support the case where nodes can not talk directly to each other and relaying is required?


Not currently, but I'm interested in supporting that.


private DNS does not work with tailscale. that means No Block Lists or nextdns.io.

if you private dns nameservers, DNS queries are made over plaintext.

Magic DNS is not a hard-sell.


Kudos to the team. Great!


What about my e-viruses and e-worms?


You can kiss those e-viruses goodbye!


This was the name of my hometown ISP back in the late 90s.

You see, kids, back in the day, the internet was not ran exclusively by gigantic mega corporations whose only argument against monopoly was "but but but WE wanted to be the monopoly!" Before Walmart pushed out all the mom-and-pop grocery stores, we had mom-and-pop internet service providers, and they didn't have to be called "artisanal" or "organic" to get anyone to care about them.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: