‘ Fundamentally, sudo adds significant attack surface because it is a powerful tool with many features like support for security policies defined in LDAP, so the system is likely more secure without it. ’
This is uncontexualized hype clearly written by someone with no real world experience, hoping to gaslight users into thinking their current system is insecure. Then I scroll up and noticed the article is written by a company who sell presumably consulting services in unix. Go figure.
Failing that, there’s limited reasons to use BSD systems instead of Linux unless you have highly specialised requirements. Under many cases I’d imagine it would be more expensive in terms of effort required to hire/train skilled staff.
Yeah, so I went down the Freebsd rabbit hole last year for my personal hosting and I have some mixed feelings about it. It's a much nicer/less changing environment than Linux, and it's really good for the things it does well. But the moment you step off the happy path then you will find yourself lost in the woods.
Basically I ended up using the Freebsd box as a fileserver (ZFS and Minio), and still running an Alpine Linux box with zfs for most other things.
I did briefly run a Minecraft server on the Freebsd box which worked really well.
Could you expand on "But the moment you step off the happy path then you will find yourself lost in the woods."? Because from my experience FreeBSD is very predictable, and knowing how things are laid out you can figure things out for other things that one is not familiar with. If anything, I see Linux as a bit of a mess that in many places is inconsistent.
Setting up NFSv4, I spent two evenings on that before abandoning it.
Virtual network for a bunch of jails with nat.
I wanted exposed ports on the host machine mapped to a reverse proxy jail that did tls termination before forwarding to other jails on a virtual network (while allowing the jails on the virtual net outgoing internet access).
Several applications that were apparently built for Linux, so they did not compile on Freebsd (and no alternatives available).
Getting newer versions of software.
Often the version available in pkg is several major versions out of date.
Documentation for how to do things in general.
Yeah when there are docs/tutorials they can often be of high quality, but frequently there is just nothing or the available docs does not cover the edge case you are trying to solve.
> Virtual network for a bunch of jails with nat. I wanted exposed ports on the host machine mapped to a reverse proxy jail that did tls termination before forwarding to other jails on a virtual network (while allowing the jails on the virtual net outgoing internet access).
Jails are old, and there's a wealth of outdated documentation around which makes it confusing when trying to understand how things works.
In case someone else is reading this, it's not different from what Docker does automatically for you.
I typically use vnet (which gives the jail an entire virtualized network stack, something like macvlan?) and either use NAT or bridges for connectivity. NAT for machines with only one public IP (like VPSes) and bridges for homelab setup where I want a more visible setup.
When setting up NAT I just use PF as my firewall, and assign static addresses to each jail.
Virtualized network stacks in Linux are network namespaces with a difference of not being tied to a specific "prison". So every Docker network regardless of type creates one.
Ah, understandable. I remember being slightly confused when starting to use jails again, after being used to the near magical setup of Docker/etc on Linux. Like manually setting up bridges and VLANs to make it work like I wanted? WTH.
Now I dislike using Docker for similar tasks, as I find it to be more work than just doing the same setup with VNET, bridges and VLANs. I'm a fossil and I want to run different services on different VLANs, which in Docker lingo is considered "legacy application setup" (looking at the macvlan documentation).
>Documentation for how to do things in general. Yeah when there are docs/tutorials they can often be of high quality, but frequently there is just nothing or the available docs does not cover the edge case you are trying to solve.
Can you give some examples so one could do something against it? NFSv4 with kerb and ldap is one..i know ;)
With jails, I could really use a tutorial that goes through a standard proxy/app/db setup (nginx, wordpress and mariadb maybe?) that does require multiple ip addresses on the physical network. So request comes in to the host on port 80/443 -> proxy -> app -> db. All jails can do outgoing connections for things like sending email, downloading updates, etc. The db and app are on a network with the proxy and only the proxy can be reached from the outside. So what would be a pretty normal setup with Docker.
And yes, firewall config should be part of that rather large tutorial, but it I really feel like a complete tutorial is missing. The ones I have seen only cover a part of it and it's left as an exercise for the reader to figure out how to glue them together.
Some of the other issues I had with incompatible dependencies would be solved if I know how to use jails correctly.
How to get a Linux jail up and running (Alpine Linux).
ZFS + NFSv4 + Kerb + ldap would be great. I ended up solving that particular problem by using minio instead, because it was so much easier to setup, but I would like to know how to get nfs working for future things.
I was wondering if there's a builder at some point, it may improve the compatibility for the whole ecosystem. There's definitely a lot of fixes going upstream from darwin build failures.
Nix is cool tech, but really suffers from a lack of good docs. I tried to use it for dev environments, but it's a struggle and asdf just has the user experience nailed, so I abandoned nix and went with asdf instead.
> Getting newer versions of software. Often the version available in pkg is several major versions out of date.
The default settings for pkg use the quarterly branch. Remove the comment from the line for using the latest branch in /usr/local/etc/pkg/repos/FreeBSD.conf.
> hoping to gaslight users into thinking their current system is insecure.
I mean, https://nvd.nist.gov/vuln/detail/CVE-2022-43995 is lest than a week old, so probably yes. (That is, a lot of systems haven't been updated and have a nice open CVE that wouldn't be there if they weren't using sudo.)
If you compare the risk of having sudo vs the risk of granting users less constrained access to root or similar, you’ll have less cves, but be more likely to be breached due to weakened rbac.
Furthermore, there would be no vulnerabilities if you just unplug your computer and walk away, at some point you need to make usability and risk compromises.
I can think of very few instances where you should be giving people shell access, let alone need to worry about RBAC.
Furthermore, the UNIX user model and file system hierarchy already provides you with RBAC. Combine that with ACLs and SetUID (et al) bits and you’ve already got sudo baked into the core system.
Your point about it being a convenience tool is bang on: it makes it easier to administer all of the above but that convenience, like nearly all convenience tools in computing, comes with a slight security trade off.
To be clear, I don’t agree with the author’s recommendation to ditch sudo. But they are absolutely correct in the technical sense. Just as you are. In fact you are both basically arguing the same point but from different perspectives: “sudo adds a vector of attack”. It’s then up to your threat modelling to decide if that is appropriate or not.
But to come back to my original point: just don’t give people shell access. There’s no need in 99% of cases of servers these days and if we are talking about local user machines, then RBAC is less of an issue since it would be a machine per user.
Even (or perhaps especially) with a machine per user, it's critical to keep that user from accidentally shooting themselves in the foot. Typing sudo does help remind a user they're doing something different. I don't want to always have the right to force delete root!
Your comment seems to suggest the alternative of not installing `sudo` is to run your desktop as root. Which clearly isn't the case at all. There are plenty of "sudo" alternatives: `su -c`, `doas`, `login`, etc
Also since we're talking about running as root: ironically I've found a lot of people seem to put the following line in their sudoers file:
ALL=(ALL) NOPASSWD:ALL
which is definitely better than logging in as root....but not by a lot.
You misunderstand it, if you don't need sudo you probably better uninstall it, what's your "real-world"-problem with that? If you work with images you might want to "deinstall" root as well....
No kidding, I am still waiting for Linux distros and other BSDs to switch to doas(1) as their default. Similar to how openssh took over from the proprietary version of ssh years ago.
In addition to what you've said, nobody is forcing anybody to use sudo. Most of my systems don't have it at all — it's simply not necessary. Some of the others use the lightweight doas ported from OpenBSD:
Some folks kind of are; for instance, last time I looked Arch Linux has sudo as part of base so it's hard to remove.
Edit: Sorry, apparently Arch does let you remove sudo (now?). If you substitute in RHEL et al. the general point stands (and I actually checked this time and Rocky 8 makes unhappy noises if you ask it to remove sudo), but my initial example was wrong.
Don't remember it ever being a problem tbh. Haven't had it installed for quite some time (since I discovered doas). I just installed it and then removed to re-check:
$ yay -Rsn sudo
checking dependencies...
:: libisoburn optionally requires sudo: for use with xorriso-dd-target
:: pacman-contrib optionally requires sudo: privilege elevation for several scripts
:: refind optionally requires sudo: for privilege elevation in refind-install and refind-mkdefault
:: wireguard-tools optionally requires sudo: elevate privileges
:: yay optionally requires sudo: privilege elevation
Package (1) Old Version Net Change
sudo 1.9.12.p1-1 -7.10 MiB
Total Removed Size: 7.10 MiB
:: Do you want to remove these packages? [Y/n]
:: Processing package changes...
(1/1) removing sudo [###############################################] 100%
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
(2/2) Arming ConditionNeedsUpdate...
libisoburn, refind, wg, yay can be elevated by doas or su.
pacman-contrib is a bunch of package development scripts. Your typical user won't have them. Also the previous point applies.
Strange; now I've tried it and it does indeed work. I'm fairly sure I tried that before and it wouldn't let me remove sudo, which was irritating because I switched to using doas and didn't like having sudo around. Assuming my memory isn't failing me, that must have been fixed fairly recently.
Looks like nothing else in core depends on sudo, most packages from extra/community that need it have it as an optional dependency, looks like one can remove sudo on Arch
It's not like you need to use sudo on Linux. And sudo has been maintained by an OpenBSD developer since the early 90s.
Aside from what the problems and benefits of sudo may or may not be, bringing up sudo in a "BSD vs. Linux" debate is a red herring: it's just completely unrelated.
> Failing that, there’s limited reasons to use BSD systems instead of Linux unless you have highly specialised requirements. Under many cases I’d imagine it would be more expensive in terms of effort required to hire/train skilled staff.
I'd expect you'd need more training for Linux, since it changes more often.
Linux systems hasn't changed a lot in the last 4 years or os. We mostly run Ubuntu LTS at my work and the 18.04 -> 20.04 -> 22.04 deltas have been really low on the server side, and not that significant on the desktop side either
During this time frame we've also migrated our last FreeBSD servers to Ubuntu
We deferred to switching to the new installer until 22.04 and kept using the legacy net boot image for as long as possible. The switchover wasn't as bad as I was afraid it would and took only a few days to make new internal app to drive our installations. The worst part of the new installer is it requires a minimum of 2GiB of RAM to run, so all of our new VMs have to start with at least that much and be scaled down later
My goodness. How can you possibly say that Ubuntu hasn't changed much from 18 -> 22? There are whole how-tos that are completely useless from those changes. All sorts of things are different. That's ridiculous.
The other comment pointed out the installer, which I did forget about. But other then that what significant has changed?
Ubuntu's documentation, and lots of third party documentation targeting ubuntu, is like 10 years out of date for lots of things anyways, and even when it _works_ often gives bad advice that either was never true or hasn't been true for years.
Netplan happened on 20.04 AFAIR. Doesn't matter much for your cloud vps you never change networking on, but still.
From my perspective, it's not changed much too - I was using apt-get install nginx - I keep using it. Was using systemctl - keep using it. Cannot say I've adjusted my habits during that moves.
> Linux systems hasn't changed a lot in the last 4 years or os. We mostly run Ubuntu LTS at my work and the 18.04 -> 20.04 -> 22.04 deltas have been really low on the server side, and not that significant on the desktop side either
You know, I actually set this comment aside because I was curious about how HN would engage with it. So far, it seems like folks are pointing out that there indeed have been changes in certain aspects of the OS, as well as probably the introduction of the whole snap mechanism.
Regardless, to me Ubuntu LTS seems like one of the better OS distros out there, both for desktop and server use. It's not ideal, but it's good enough in most respects, to the point where it's hard to recommend anything else.
Previously I recall many looking in the direction of CentOS as something to install on servers and have it serve them for a long time with security updates and pretty good stability, but I guess now one would have to count on Rocky Linux and Alma Linux for that. Additionally, I have to admit that some of the choices that RPM distros make can be a bit odd and the software support isn't where I'd like it to be, things like Docker were admittedly broken (DNS resolution, firewall masquerades and other oddness with SELinux) for a while. It feels like they have their own enterprise-focused way of doing everything, OpenShift being an example of this.
On the other hand, there's Debian and Ubuntu. Personally, I might even like Debian better, but Ubuntu's LTS lifecycle is far more worry free. You might not get 10 years of support, but setting up something and running it for 5 years (with security and other updates) is pretty freeing, especially when you're primarily interested in just keeping things running in a stable manner and don't always need the latest packages. There are changes, of course, there are packages that change names, that break or are deprecated, sometimes you get other growing pains that all of the OSes out there need to deal with.
But Ubuntu LTS to me feels like it's good enough for both server use in both development/testing/prod, as well as personal use, as a daily driver.
And nowadays I'm considering migrating my Debian servers after the EOL to Ubuntu, my new ones already running it.
I will say that every distro has its place and in certain domains there are better options than Ubuntu, of course, and OSes like the *BSDs have consistency and engineering that Linux cannot quite match (due to the "engineered vs grown" approach to building them), but sometimes you just want something that is simple to run, has ample tutorials and resources, and won't make you chase down odd issues.
That wasn’t their point. But to be honest I do think FreeBSD is.
Linux wins in terms of market share which makes it appear easier (eg more software support, bore install guides, etc). But if you look at the base OS itself, FreeBSD is a lot more consistent to manage.
I think it’s a great pity FreeBSD doesn’t get more traction to be honest with you. I have no major qualms with Linux on the whole but FreeBSD is a really nice platform to work on too.
I can't speak for the person you're asking, but BSD is worlds easier than using Linux.
A how-to for FreeBSD can be used with NetBSD with minimal changes. Why? Because the BSDs don't expend effort to deliberately try to differentiate themselves from each other. They're consistent and don't change gratuitously.
Try writing a how-to that has even the slightest consistency between the common Linux distros - Red Hat, Ubuntu, Debian, Arch. Heck - many how-tos written for Ubuntu 18 can't be used with newer Ubuntu! It's a mess.
No, they're suggesting that BSD changes less. If you learn FreeBSD, and you learn, let's say Ubuntu, and then you keep using them for a decade, the stated position is that FreeBSD would require you to learn fewer new things than ex. Ubuntu over the same timespan.
I've been using Linux for 27 years, and it hasn't changed all that much in that time. Packaging has got better, systemd is a bit of a pain in the tits but necessary if you want to support the kind of things we expect to do with "real" computers instead of servers sitting in a dark room untouched for years.
I've been using NetBSD for about the same amount of time, and it hasn't changed all that much either.
> I've been using Linux for 27 years, and it hasn't changed all that much in that time.
First there's networking configuration. On Debian and Ubuntu the interfaces(5) used to be nice and simple, but dog knows what to use on any particular system nowadays since there are like three ways to do things: interface, netplan, system-networkd, other?
Plus there's resolv.conf(5) being fiddled with a myriad of ways and you have to figure out which system to use there as well.
ifconfig kind of works and kind of doesn't for some things.
How may firewall system changes have there been in the last 27 years? Are we supposed to be using iptables or nftables in 2022?
In roughly the same amount of time I've owned about ten or fifteen cars.
Every single one of them had the dashboard controls in slightly different places, with slightly different options. Some had four-speed gearboxes, some five-speed, some automatic with three-, four-, five-, or in one case 18-speed gearboxes.
Every single one of them was basically just a car, and drove me to the same places in the same way.
I have not tried recently to use BSD as a desktop, which is likely to have problems when using many recent GPUs, due to the lack of device drivers (but NVIDIA provides FreeBSD drivers, not only Linux, so I have used FreeBSD on a signage appliance with a fanless NVIDIA GPU).
However, when using a system without a GUI, e.g. for a networking or storage server, in most cases FreeBSD is definitely much easier to use than Linux.
There are a huge number of small details that matter. Even for something as simple as seeing the status of the network interfaces. If I type "ifconfig" without arguments on FreeBSD, it shows everything that I want to know, about all network interfaces. To obtain the same information on a Linux system, I have to type a large number of "ip" commands, each with many arguments that I fail to remember even after decades of using "ip", so I may need to consult the "man" page for "ip". (The legacy ifconfig of Linux does not provide the functionality of FreeBSD ifconfig)
In general, for any network-related task, using ifconfig and ipfw on FreeBSD is much easier than using ip and iptables on Linux.
There also many more niche applications where there are big differences. For example the "mt" command for magnetic tape control sucks badly on Linux, but it is very convenient on FreeBSD. Also for SCSI devices FreeBSD has more flexible commands.
While audio seems to be improving on Linux with things like pipewire, for a long time audio was in many cases easier to configure on FreeBSD than on Linux with the dreaded ALSA or pulseaudio, if the audio hardware was supported. Also, when I had to configure once some USB surveillance cameras, that was easier on FreeBSD with webcamd, than on Linux, but like for audio, FreeBSD has a shorter list of supported devices.
Some of the differences in the user interface between FreeBSD and Linux are caused by FreeBSD being more conservative, while in Linux some older tools have been replaced with newer tools, but the newer tools have been made with worse user interfaces, like in the replacement of ifconfig with ip, or in the introduction of ALSA and pulseaudio, which both have an extremely poor user interface, especially for computers with multiple audio devices.
Yes, much easier, Linux is a mess, the firewall disgusting, LVM...buah. The only problem BSD have, are drivers. But you obviously never worked with BSD...unbloated systems ARE easier to use.
Firewalls are the one thing I don't really like on FreeBSD. It gives you 3 options but the handbook doesn't provide much guidance on the pros and cons of each.
I'd prefer it if there was one favored one that all the effort went into to make it really good, just like with everything else FreeBSD. This is not arch where you have to handpick each component :)
Of the 3 FreeBSD firewalls, ipfw is the native firewall, like iptables is for Linux.
The pf firewall is a fork of the OpenBSD firewall. It is the normal choice for someone already familiar with using OpenBSD firewalls. Otherwise, a new user should choose between ipfw and pf the one whose configuration seems easier to understand.
I happen to prefer ipfw, because writing a configuration file for ipfw is exactly like writing a program in a programming language (a language very much like BASIC), which examines the incoming or outgoing packets, by executing instructions sequentially. Others find the syntax used by pf more pleasant, but I find it more difficult to reason about which is the matching rule that will be selected by pf for execution for a certain packet.
The third firewall, ipfilter, is also provided mainly for the former users of it on other operating systems. I am not aware of any particular advantage for it.
Exactly, I found out the same with some googling, but the handbook specifically mentions it will describe the differences and then neglects to do it :)
I was first considering pf because I used it on Mac before, but the problem is the freebsd version was forked quite a long time ago and has different features than the openbsd version as a result.
I went for ipfw mainly because it seems to see the most active development.
But really, I wish FreeBSD would just pick one as standard and keep the others around for compatibility reasons if needed. One of the reasons I like FreeBSD is that you don't have to make such choices. Things have well chosen defaults.
Contrast this with arch which I mentioned, that even lets you decide which mechanism to use for network management (e.g. net manager, connman, systemd-networkd, netctl). It describes this choice really well in its excellent wiki, but when I tried arch it meant I had to do this kind of decision for pretty much every part of the OS :) What I like for FreeBSD is that they keep things simple.
I think Arch and FreeBSD have a lot in common, they're both intended for users that don't expect to click through everything and just dump you on a command prompt where you can set up stuff the way you want, instead focusing on excellent documentation.
I use FreeBSD on my daily driver PC now as I thought Arch was too much work with all the choices, and on the other hand the one thing I did want to change (systemd) it didn't allow.
>>CRUX is a lightweight Linux distribution for the x86-64 architecture targeted at experienced Linux users. The primary focus of this distribution is keep it simple, which is reflected in a straightforward tar.gz-based package system, BSD-style initscripts, and a relatively small collection of trimmed packages.
>>Inspired by CRUX, another minimalist distribution, Judd Vinet started the Arch Linux project in March 2002. The name was chosen because Vinet liked the word's meaning of "the principal," as in "arch-enemy"
BTW Crux is still a great distro, limited packages yes, but absolutely simple and rocksolid.
I like FreeBSB too...well it's my main system since years >8 probably. But if it need's to be linux then Alpine Arch Crux....or the RHEL's Debian's...if i really have to use them.
It is. I honestly struggle to see how anyone would pick Linux over Windows but then pick Linux over BSD. It feels like the worst of both worlds - with Linux you don't get the hardware support and keep-going-at-all-costs of Windows, but you don't get the consistency and comprehensibility of BSD either.
Any commercial OS, like Windows, Mac OS or Android, will be easier to use than any free operating system, for those users who never attempt to do any action that is not on the restricted list of actions for which the commercial OS was designed.
Nevertheless, there are many people like myself, who always discover, when using a computer, that they want to do an action not anticipated by the designers of the commercial OS, either due to lack of awareness about the needs of some people, or because they do not bother to support those that are only a small percentage of their users, or because they have an interest in preventing the owners of the computers to use their property in certain ways.
Whenever it happens that I want to do such an action, I discover that it is much more difficult to do it on Windows than on Linux or on any *BSD, and sometimes even impossible. That is the reason why I have abandoned the use of MS Windows or Mac OS, many years ago, except for various occasions when I am forced to use them temporarily for professional reasons.
Being a BSD user myself, i just said that Windows is easier (overall). I especially talk about Active Directory and Group Policies, on Windows a matter of Hour's on Linux/BSD (a comparable system)...Weeks?
But yes *nix is much more flexible..but that comes with a cost, sometimes a massive one.
Good one. Care to elaborate on your thoughts though? I work in security, I see problems caused by people trying to be edgy in architecture design all the time. You don’t need k8s, or bsd, or rust if you’re a pizza shop website.
> This is uncontexualized hype clearly written by someone with no real world experience, hoping to gaslight users into thinking their current system is insecure.
So typical for BSD fans, find a single flaw then try overblow it and show how their flavour of *BSD is better. It's frankly tiring at this point.
Rather than telling the difference between how command line utilities take arguments and how the file structure is differently laid out, I expected the article to present FreeBSD advantages and to tell me which software works on FreeBSD.
I don't use *Nix for the desktop but I would love to find out that FreeBSD allows me to run Kubernetes, Elastic Search, Redis. That I can achieve more throughput with FreeBSD, that Postgres and MongoDB are more performant than on Linux and that the system is more reliable than Linux.
How ls and grep works on FreeBSD is no concern of mine. But running Docker is.
K8s and Docker are purely Linux things. Use Linux for that. Its like you would say "I really like these Jails, on which Linux distribution I can run them on?" :) On FreeBSD You use Jails and Bhyve.
Which I happen to know. But it was an example. If I am going to switch to FreeBSD it has to run the software I need, not to merely provide alternatives. And it has to do it better.
Saying that FreeBSD has jails as an alternative is akin to saying Linux has Gimp as an alternative to Photoshop. Not many are going to switch operating systems to use Gimp instead of Photoshop.
Docker is just namespaces and cgroups. You do not have separation between host and the containers on Linux - you need additional software for that - like AppArmor or SELinux. The FreeBSD Jails are secure and separated from the host on the contrary. FreeBSD Host can access Jails but Jails can not access Host resources - unless consciously configured to do so.
As for the usage ... FreeBSD Jails can have their OWN network stack (with own firewall and interfaces) while Docker can only use Linux network namespaces.
You can launch/start a single process Jail and a single process Docker - here they are similar. You can also unpack entire operating system userland in both and it will also run - with own packages database etc.
Okay, thanks for the information! Sounds like I would still prefer jails.
Managing Docker always sounded complicated compared to managing jails. It is literally just a configuration file and a copy of userland on the file system. If you are a little clever, you can share the userland and additionally create templates which are really just some directories on your file system with additional stuff you want to put in.
Docker is more on ease of use and distribution. Convince over all major parties (Windows, MacOS, Linux). Docker Compose for building interconnected services ( it even does dns resolution for you, so you can reach your services by short names not caring on IPs).
Then it become natural for teams to share dockerfiles as way for sharing environment setups in all the same way.
Using `docker run something` requires zero to none knowledge of the OS, Networks (nat included), packages. Everything will be auto downloaded.
Okay, interesting! I want to be in control of all those things(OS, Networks(nat included), packages).
I roll my own packages using poudriere that I then distribute using FreeBSDs binary package management system... to my jails. XD
That way I get a coherent well configured system everywhere.
And hundreds of thousands of devs [nowdays we say developer, really meaning web related developer like Javascript] don't care.
They want be in control of their API, but not boring sysadmins stuff.
Jails are targeting sysadmins, thus Docker wins the adoption.
Like cars with manual transmission.
> But the manual transmission is dying out. Just 41 out of the 327 new car models sold in the United States in 2020, or 13%, are offered with a manual transmission, according to data from Edmunds. That is a tremendous drop from less than a decade ago. In 2011, 37% percent came with manuals.
I don't think the analogy to manual transmission is correct - The way I have configured poudriere/pkg it's very rare I have to intervene, unlike with a manual transmission where the driver is directly involved in the control loop.
But since I am in charge of the control loop, I can change it any time I please.
Right now I am changing my poudriere configuration for the first time in years to build my packages using LibreSSL instead of OpenSSL. Sure, it requires some tinkering(not alot though) but that tinkering is completely optional on my part.
I find transmission sample in good correlation of decreased jails (manual, sysadmin way transmission) usage and increased docker usage (automated gearbox tooling).
Yet I have practical question for every Jails user, especially when I see "I did..." statements, like in yours:
>The way I have configured poudriere/pkg
How it works for _team_, how it works for "we", where common ground is expected. I have seen teams of 5-10 using Docker as base ground for their dev (local laptops with Windows, MacOS,Linux)/stage/prod envs, but I have not heard on any of such kind with Jails over last 10 years.
I hardly can imagine that without forcing everyone to switch to FreeBSD as daily driver. Can you share experience of your team?
As you require Docker/k8s there is not much that can be done here - just stick to Linux (or use them in a FreeBSD Bhyve VM but we know its better to just use Linux).
Just like with your Photoshop (or Lightroom) example - if you need to run one - you use macOS or Windows for that - Linux also not possible here.
Correct, K8s and Docker will only work one single platform, namely Linux and Windows. No, wait, that's two. K8s and Docker can only possibly run on these two platforms: Linux, Windows and macOS. Ok, let me start over. The three platforms that support K8s and Docker are Linux, Windows, macOS and Oracle Solaris[1]. That's not... ok, the FOUR and ONLY four platforms that can run K8s and Docker are Linux, Windows, macOS, Illumos[2], Solaris, SmartOS, and FreeBSD.[3] How many did I say?
K8s and Docker run anywhere someone is driven to make them run.
[2] https://illumos.org/docs/about/features/
"LX-branded zones provide the Linux system call interface, allowing you to run most Linux application binaries without recompiling them for illumos. This facility is available in several illumos distributions, including SmartOS and OmniOS."
"The Docker platform runs natively on Linux (on x86-64, ARM and many other CPU architectures) and on Windows (x86-64)."[1]
macOS Docker performance leaves much to be desired, but this is moving goal posts and suggestive of infinite ignorance: nothing other than Docker on Linux is acceptable. But Docker is just software and it will work wherever, and it has its limitations and plenty of competition to do what it doesn't do or does poorly. Die on the Docker Linux hill if you wish, but, last I checked, platform agnostics are versatile, more marketable, and more valuable, while penguinistas have become all too common. "Give me Philips head screwdrivers or give me death! How can you still use flathead drivers? Or star bit drivers? LOL You're not even screwing seriously. Real men use Phillips." Same thing, really. When you need tools to work, you'll take care of them. But precious few, if any, actually worship them.
Chances are that PostgreSQL will be more performant on FreeBSD, but that would require a wee-bit of very specific fine-tuning for specific workload pgsql is doing and it would ultimately be thanks to ZFS, which I think is default on FreeBSD ( would work on linux as well, if setup correctly ). See here for more info on how and why's - https://people.freebsd.org/~seanc/postgresql/scale15x-2017-p...
I have used jails, though a slightly harder to set up, once setup it works great. I used it to open up ssh to an empty jail as a kind of file server. Mainly to sftp files to that I change on my laptop at a remote site. This is in the days before the "cloud".
I am always mystified why FreeBSD and Jails never took off as "the" cloud environment.
I think you know that Docker is built on cgroups, which is a Linux kernel feature and that as a result it will work as badly as it does on other things that are not Linux.
Many FreeBSD user do care about jails, but they’re certainly not as easy to use as docker and so they’re not (anymore) going to get anyone rushing to drop Linux and move to FreeBSD.
Docker spanked a ton of VC money hosting the worlds software to create the smooth developer experience that we’ve come to expect from them. I don’t really see how an open source project is supposed to compete with that. Particularly as, whilst they gained a lot of mindshare, they don’t show any signs of getting their money back.
> I don’t really see how an open source project is supposed to compete with that.
Two parts here.
Part 1
docker/cli is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
Seems to be OpenSource project for me.
Part 2
Jails were first introduced in FreeBSD version 4.0, that was released on March 14, 2000.
while Docker was publicly released on March 2013. 13 years of advance to give developers what they need. To give enterprises what they need. To get the hype at least.
From my POV - more than enough time to create solution on this. TrueNAS did for NASes.
The VNET enabled FreeBSD Jails (separate network stack) are more complicated - at start only - you need to get to know how to use the new epair(4) VNET interfaces etc.
It's probably easier for specific use cases where Jails better be compared to LXD and when you need "system" inside your VE (virtual environment)/Jail. And that's not what docker targeting much.
For the even simple real life use case, I and my team had for running tests over our site with https://www.sitespeed.io/ :
Use our Docker container to get an environment with Firefox, Chrome, Edge, XVFB and sitespeed.io up and running as fast as you can download them. They work extremely well together with Graphite/InfluxDB and Grafana that you can use to monitor your web site.
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:26.1.0 https://www.supersite.io/
Our fronted dev and SEO guys can do from their MacOS/Windows machines with 0 knowledge of Linux and docker, just by copypaste the command in _under_ 5 minutes.
Achieving the same with Jails in even 1 hour is beyond my imagination.
Still struggling to hear any real life good samples of team work with Jails instead of Docker.
As someone who has used FreeBSD exclusively on bare-metal corporate production servers for the last 10 years, I still wouldn't use it on desktops/workstations. It's built as a server OS and my opinion is that it does this very well, I have tried a few times over the years to use it as a destkop OS, I spent more time configuring my machine than I did actually using it. I can install Ubuntu and 30mins later be using it for work with no issues.
If you run your own servers, especially ones that you physically have to maintain the hardware on, FreeBSD is a very reliable choice and the upgrade paths on it are fantastic. I now run some services in Azure and use docker, but all my "on prem" stuff, is still freeBSD powered.
OpenBSD is often considered the desktop BSD. You install it and it works or it doesn't. The only real hangup tends to be configuring the sound if you want to do something different but once you get it configured that configuration is forever and solid.
Yea, I have tried OpenBSD a few times, but sadly I just don't have enough time to invest in my OS to get the most out of it. I use my machines until they fail, then replace them with a new one/part. A fresh install of ubuntu and they are ready for work in 30mins. I used NixOS for a small period of time for this reason, but even this became more of a headache than it was worth.
Modern software development is driven by fashion and trends. All you need is a few tech “influencers” to switch to freebsd then crowds will follow. FreeBSD was and remains my first love, but not being fashionable there is nowhere i can use it.
> All you need is a few tech “influencers” to switch to freebsd then crowds will follow.
As someone who would like to switch to FreeBSD but cannot, let me give you a few counterpoints. I also loosely follow actual fashion trends in clothing.
I actually think that FreeBSD is already fashionable. It scaled up WhatsApp and Netflix, two of the most impressive startups! Tarsnap uses it. It seems pretty stable, and I would be perfectly fine adopting it.
Where would I adopt FreeBSD? In two places – on the desktop/laptop, and on the server.
On the desktop/laptop: I need perfect screen resolution on my laptop and external monitor, Bluetooth and Wi-Fi, and a way to correctly use my plugged-in microphone and headphones. Ubuntu delivers all of this with some grumbling, and MacOS and Windows do it flawlessly. What does FreeBSD do? Reports seem to say that Wi-Fi and Bluetooth are problematic.
On the server: I need a compelling way to manage thousands of servers if needed, and something like Ansible won't cut it. For whatever reason, distributed systems on top of FreeBSD seems to be something that the community refuses to build on. I'm not sure what kinds of frameworks WhatsApp and Netflix built to manage their servers, but none of it seems to be public.
So, I kind of disagree; I think FreeBSD is fashionable but fails to be utilitarian enough. Fix the problems on the desktop/laptop; or deliver a way of showing how to scale up quickly with FreeBSD, and you'll get your adoption.
The problem of FreeBSD adoption is not really for the FreeBSD folks to solve. Fashionable, popular packages have made the explicit decision to avoid BSD support. Over the years I've run into issues with things like Node, Elixir, Visual Studio Code (or any electron app really), Python (Conda) being lax or hostile to supporting anything other than Linux/Mac/Windows.
For config management all of the usual suspects should work and all will be painful to scale in non-FreeBSD specific ways. The Opscode related nightmares finally subsided sometime during the pandemic. Was there a specific package you had trouble with?
VSCode remote not supporting FreeBSD is still so annoying. It's available only as a binary and there's no real, practical reason why it couldn't work beyond Microsoft just not wanting to do so...
Have you tried it in the Linux emulation on FreeBSD? My experience is limited to just my laptop, but significant software builds for Linux (like Chrome) run fine for me. I imagine this could be the case for other things not requiring systemd.
the actual server itself probably works under emulation but the issue is then that when it ssh's in it still uses shell commands to detect the host type, which I can't so easily hack to make it look like Linux without breaking everything else.
The problem isn't in getting a random Linux binary to run. The problem is that you're running a software suite that's designed to not run on FreeBSD.
If memory serves uname(3) should return Linux when you're running a Linux binary, but then you're back to running a hostile program and working around anything that may arise once you have to break out of the Linux userland.
I have no idea why the hardware/software industry all focused on the linux kernel as the common base for so many hardware support. The amount of chips supported in the linux kernel is mind boggling.
I worked with FreeBSD in my early career. How do I miss the `make world`. So easy, all-in-one, clean and elegant system.
Because everybody runs Linux for last like 20 years. Data centers, network appliances, storage appliances, smaller devices of all kinds. It's in the open, the mindshare is colossal, expertise working with kernel and writing drivers is relatively easy to come by.
Everybody runs Linux because it's free, there is no danger of closed private forks gaining any traction, there is no danger of it playing MongoDB. So everyone can safely pick it and invest into it. The pace of progress is very high because of that, and anything broken would be fixed very-very soon.
Because of that ability to rely on Linux being a common thing that nobody can privatize, huge companies like Red Hat and Canonical could form, and entities like IBM, Oracle, Intel, Google, etc could keep investing in the development.
All the above is sort of problematic with FreeBSD. It's too cathedral on one side, and has too few strings attached to hold to it on the other side. So it remains the choice of connoisseurs who need its particular technical brilliance (e.g. the network stack) or its very permissive license (see PlayStation).
> Grsecurity fully complies with the license of the Linux kernel, the GPLv2. Since grsecurity is delivered as a source code patch, it is not possible under the terms of the GPL to offer a free version under an actual restriction that it be used only for evaluation purposes. Any customer receiving a grsecurity patch receives all the GPL-granted rights and responsibilities, including the right to redistribute patches in their possession or even to sell them to others.
Forks are completely fine. A lot of hardware, often not even esoteric, is supported by non-mainline kernel versions.
> I have no idea why the hardware/software industry all focused on the linux kernel
I'll die mad about it.
So many in the OSS community applaud vendors for upstreaming a driver to the Linux kernel even though they haven't given any basic form of documentation. Not only is this a massive pain if you happen to work on a project that isn't GPL compatible (back to square one, legally required to clean room implement the algorithm if you want to work with Linux source), but you still have no idea how the device actually works and what the sharp edges are (good luck writing bug fixes!). It's utterly baffling that people have generally accepted Linux as if it is the only FOSS kernel that anyone should ever care about.
> this a massive pain if you happen to work on a project that isn't GPL compatible
Good. And I hope that continues to be the case.
The harder life is for non-copyleft licensed software and ecosystem the more society stands to benefit and the more the software industry stands to redeem itself.
For those of us that treasure freedom, through diversity of ideologies there is strength. Why one would actively put a pox on those with nearly the same goals as yourself is beyond me. But, hey… you do you.
Trying to spring the GPL on people because they dare to try and understand how their wifi card works is not, in my opinion, a reasonable nor appropriate use of copy-left. Copyright is for derived works, not technical knowledge (i.e. specifications) but when you vendors refuse to provide documentation and only code, they force everyone into this weird position of creating a pseudo-derived work if they dare to (or can even be accused of having!) come near the Linux kernel source. If vendors provided a simple PDF everyone would be happier because now anyone can write the driver free and clear to the best of their ability.
>I have no idea why the hardware/software industry all focused on the linux kernel as the common base for so many hardware support.
Example #2124 out of 251295912931, BSD devs were absolutely hardcore adamant to stick to CVS and SVN as their source control for years after the rest of the world has moved on.
Is there a specific (new) laptop you would recommend using with freebsd? One that is as compatible as you say with wifi and bluetooth drivers. I really want to use it even if just for nostalgia. I love how organised and logical the os is.
cfengine3 and puppet both work on FreeBSD, at least. Looks like people are using chef with it too. And, as you point out, Ansible treats it as any other Unix-like system.
I'm not sure what you've tried that does not work!
Ansible and Chef are inadequate tools for the job that cloud server boxes do.
What is needed but is sort of missing:
- A way to run containers. I see that *BSD may have had container-like features (e.g. jails) long before Linux containers existed. The truth us that the prevailing format is Linux-style containers. Ideally one should be able to run ready-made containers in Linux compatibility mode. An ability to build a native container from a Dockerfile is as important.
- An orchestration engine based around containers. Ideally a port of a subset of k8s, but smaller tools the class of Nomad or Docker Swarm would be a good start.
- A good replacement for Docker / Podman / Containerd, preferably with a modicum of compatibility / convertibility from those. You want an easy local development environment.
I bet bits and pieces of it sort of kind of exist, and the kernel is capable enough. Now someone should put (quite) some effort in building that for real, releasing under a reasonable open license, and maintaining it for years.
This is to say nothing of (desktop /laptop) hardware compatibility yet.
Yeah, what Docker (and the ecosystem around it) got right is the user experience, it's just second to none. It just doesn't matter that Illumos zones or BSD jails might be technically better when the user experience of Docker is so much better.
On the smaller end of the scale, docker-compose is fantastic for managing apps if you only have a few systems.
I remember that there of course were efforts to adorn jails / zones with an easy Docker-like UX. I hope this effort continues and will result (or has resulted without me knowing!) in a great and simple developer experience.
Okay, no. If you're going to set the bar at "must be identical to Linux" then you may as well just use Linux. Docker specifically (and anything based on cgroups) is so closely tied to Linux that you're not going to find a clone on a non-Linux platform. Nor should you.
Insofar as encapsulating a container in a file-like format, the primitives are all there: you can export a ZFS snapshot of your jail to a file. Higher level tooling is there as well: iocage has an export subcommand.
Building jails is an exercise left to the reader. I've always found Dockerfiles to be archaic and generally unpleasant to deal with such that I feel like the lack of a Docker compatible template file is a feature not a bug. While I use ansible and some shell scripts, things like Nomad already support (iocage) jails.
I have tried to get cfengine working multiple times, but it's just too difficult compared to every other configuration management solution I can think off.
I have used FreeBSD for a over a decade now. ZFS was the new cool thing I feel in love with.
Today its a different story. FreeBSD is cool, and you will learn more about Linux by learning FreeBSD and it works really well.
However I am currently moving my last system away from FreeBSD. It is just that, Linux/Centos is better, and easier to maintain. Especially third-party packages. There are other things like Linux has support for NVMe-oF which I use together with ZFS on Linux. Applications, especially databases has often 1x-2x better performance. Virtualization is better and more performant. Better NUMA.
But Linux is not made for long running systems, you make a image and throw it away if it's not booting anymore, it's like smartphones and ioT devices, if they stop booting throw them away, linux and it's filesystems are good enough for that ;)
Void Linux is probably as close as you'll get to that. OpenZFS is supported across every packaged kernel, and it has native packages for ZFSBootMenu. You can't just click-through to a root-on-ZFS setup like you can with FreeBSD - but Voids' installer is rarely used to begin with. Boot environments aren't as deeply integrated into Void as they are into FreeBSD. That's more a lack of desire/will than a technical matter.
Edit: I forgot about Project Trident. They migrated away from FreeBSD to Void Linux with ZFSBootMenu. They did actually support root-on-ZFS through their installer. They gave up after about 18 months, they weren't able to differentiate enough after they stopped being based on FreeBSD.
Thanks for at least a partial solution. Installation can be scripted.
I remember the disappointment after PC-BSD died and Project Trident invested their 'time' into Void Linux instead of FreeBSD ... then I thought that it actually my be a good move - so finally some Linux would have some ZFS and ZFS Boot Environments love - especially as these people used ZFS Boot Environments on FreeBSD earlier ... and then it all died.
A lot of potential killed/lost there. First - PC-BSD introduced PBI packages - something that is today known on Linux as SNAP for example - some application packed together with libraries and dependencies in one 'package' - then for some reason they killed this idea - while Linux 'reinvented' it as SNAP (and others) and 'The World' was (and sometimes still) is fascinated by it. Sick. Same with Docker that was introduced in 2013. The so called 'World' went crazy with 'containers everywhere' ideology while FreeBSD Jails were available 13 years later since 2000. Sometimes I just got the idea that Linux people do not see any other innovations other then that happened with Linux system. Like FreeBSD Jails or Solaris Zones or IBM WPARs or HP-UX nPars never existed ...
I am currently in the process of migrating my own servers to FreeBSD. So far it's a slow process to learn these new tools, instead of what I am used to, i.e ip addr/route show, iptables, nft list ruleset and different default directories for many things. Can anyone report how is Wireguard support? Since I am using it heavily, there is no way without it. So far it looks like only thing I am going to miss is qemu and will probably skip VMs at all and try to utilize jails for everything. Apart from that I am really looking forward to utilize ZFS.
Wireguard has kernel support now. It was originally a botched implementation that almost made it into the kernel but it was rescued by Jason Donenfeld himself at the last minute.
> The base system includes all necessary core functionality, and is updated as a single unit. Additional optional software is installed from ports or packages and managed separately.
I started working on https://github.com/NixOS/nixpkgs/pull/82131 because the idea that different kernel => reshuffled userland is a stupid an a massive source of unneeded switching costs.
I am not mad at Linux, but monoliths at the 10M LoC order always need competition. Otherwise we have a major source of unchecked complexity growth with no escape.
These guys are selling FreeBSD consultancy, so I don't know how this article can be unbiased.
> On Linux systems, it is very common to use sudo for operations that require root permissions
Not common unless authors used Ubuntu desktop, which pushed sudo usage to the extremes. I don't even have it installed. However, frequent sudo usage usually indicates something is wrong with how you use the system, maybe account permissions or lack of knowledge of how to use it properly.
Honestly, I don't have any issues with "sudo" it is an excellent tool for the job - it temporarily executes a command as a different user. But Ubuntu moved that to the next level, and in documentation, you'd frequently find shell samples with multiple successive sudo calls, executed as a root user. That should be replaced with a single "su".
On the other hand, I have seen excessive sudo usage only in Ubuntu and some Debian/Arch docs. Maybe, this is the reasoning behind their comment that sudo is frequently used.
<joke> Also, you can easily spot Ubuntu users: they'll type almost any command with sudo :D </joke>
For servers, I use FreeBSD whenever I can because of its speed and simplicity (and ZFS boot environments).
However, on a laptop and used as a graphical workstation, it can be painful. I'd love to see a hardware vendor (e.g., like System76) bring out a pre-packaged FreeBSD laptop. I think a lot of people in the *nix community would jump on that, myself included.
The article is more balanced than I expected given the comments TBH.
I find it interesting that the Linux programs lspci, lsusb etc have ended up with arguably more consistent naming than the BSD equivalents pciconf and usbconfig.
The term wheel was first applied to computer user privilege levels after the introduction of the TENEX operating system, later distributed under the name TOPS-20 in the 1960s and early 1970s. The term was derived from the slang phrase big wheel, referring to a person with great power or influence.
Can I give the most stupidest and shameful reason I have not yet moved to FreeBSD from Linux (been using Linux as my daily driver for last 15'ish years).
Google Chrome Passwords. For some reason you can't install Google Chrome on FreeBSD, you can install Chromium but all my saved passwords are stored in Google-Chrome, and you can't (it seems) port it !
So now I'm stucked with, Ok do I spend a day and move everything (manually) to Firefox/Chromium or do I just suck it up and stay where I am.
Yes yes I know there are many "solutions" how to store passwords, that is not the problem, the problem is PORTING my 100+ passwords/logins Ive gathered over the years to a new shiny solution.
This is uncontexualized hype clearly written by someone with no real world experience, hoping to gaslight users into thinking their current system is insecure. Then I scroll up and noticed the article is written by a company who sell presumably consulting services in unix. Go figure.
Failing that, there’s limited reasons to use BSD systems instead of Linux unless you have highly specialised requirements. Under many cases I’d imagine it would be more expensive in terms of effort required to hire/train skilled staff.