It seems to me that the author's system was in an incredibly unstable state to begin with. He's running a Debian prerelease with an unsupported filesystem, that had gone without updates for months at a time. I have a hard time seeing how changing the init system would not cause problems in a setup like that.
Agreed. I run Debian Testing and was able to upgrade ~5 machines without a hitch. Yes, I do have some odd behavior with the brightness key (but only when a second display is plugged in...), but I'm running Testing so I'm happy to trade an occasional annoyance for more recent packages.
The biggest problem I've had with systemd (I now have about 200 machines running it/Jessie) is that I tend to blame everything on systemd... DHCP problems on boot! Must be systemd! (It wasn't...)
Yup, let's face it, RedHat have bet the farm on systemd and the facilities/programs that go with it, so it will have to work and work well for 10+ years minimum.
RHEL7 and the clones have systemd v208. Debian Testing just had v215 arrive. Freeze is quite soon, so it looks like 215 will be the systemd version that ships, so the packagers will have to backport security fixes to it for the life of Jessie (say 2 years or so). Just wondering how much work that will be if systemd continues to iterate at the speed it is doing...
PS: why so many 'machines' (desktop clients, workstations?) running an unstable Debian I'm idly wondering...
It's worth mentioning both that Debian Testing is not called 'stable' because, in their own words, there are occasional significant breakages. It's also worth mentioning that right now, Debian Testing is being prepped for a freeze to become Debian Stable in less than a month, and it's absolutely chaotic as everyone is applying patches all over the place to get them in before the freeze.
After the freeze there'll be two months to sort out critical bugs like the ones in the article, but this possibly wasn't the best time to do a big upgrade of an edge-case system. Well, unless you want to find and report bugs :)
> it's absolutely chaotic as everyone is applying patches all over the place to get them in before the freeze.
No, it's not. That said, if you want to run Debian testing, do yourself a favour and also install apt-listbugs -- it'll hook into apt, and alert if the update has any bugs filed against it (the update you are installing, that is).
I recently moved my laptop from Debian/stable w/backports to Jessie (testing) -- and it's been fine (had to do a reinstall anyway due to shifting around partitions, because I needed more space for windows so I could create an installer for windows 8.1 -- didn't have enough free space to ... don't get me started).
I don't understand why people would choose a non rolling-release distro (e.g. Debian) over a rolling release distro (e.g. Arch). Because the former come with the implicit guarantee that every few months, you have to update EVERYTHING all at once. Seems like a needless risk and pain.
Genuinely curious here. There must be an upside to this model, can someone tell me what it is?
I switched from a rolling release distribution (Chakra) to Debian because I got tired of the rolling release system constantly breaking things. If you fall too far behind on updates in Chakra, the maintainers basically tell you you're screwed. I finally complained about it in their forums and was politely told to go away.
My computer is just a tool. I just want it to work so I can get things done. Debian has been good about that in the past and enjoys the support of a very large, dedicated, active developer community.
Maybe Arch is better than Chakra at managing their updates (although Chakra was basically a modified Arch distribution), but my experience with Chakra left a really bad taste in my mouth for the rolling release approach.
FWIW, Debian's "update EVERYTHING" every few months never broke things as badly for me as Chakra's frequent smaller updates did.
Yeah, Debian stable is basically what I'm on now. (Sort of. It's complicated.)
If I can be unabashedly honest for a moment, I think the problem with Linux advocacy isn't the software any more, it's the advocates. The advocacy seems to be coming from a lot of people that are always after the latest, newest, greatest thing, people who are actually uncomfortable when features aren't changing all the time. They tend to be very vocal about recommending anything that comes with frequent updates, whether it's browsers or Linux distributions.
But that's not what the larger untapped home market wants. They just want things that work, and they want things not to change on them all the time. They only have enough space in their daily lives to learn new features every once in a while; any more than that, and they get frustrated. I hear this from our customers all the time, and as I get busier, I'm really starting to understand their point of view better. (As an example, we've recently been seeing more of our customers switching back to Internet Explorer on Windows 8 systems.)
So as far as signposting goes, I think boring, stable, slow-moving distributions like Debian stable should be the default recommendation, and then Arch or similar recommended as the really cool cutting edge thing for hobbyists and early adopters. That is, Linux by default should be marketed as stable and safe and not changing all the time, with the option of making it a bit more fun.
We almost opted to recommend Linux to a bunch of our customers who were moving off of Windows XP this Summer, but the feedback wasn't great on a couple of experiments and finally we decided we were too small to really afford the support costs. It was a big missed opportunity though and I'm going to take another look at doing it next year.
Well, it's already pretty well signposted; the three most well known consumer distros are redhat, suse and ubuntu, all of which have this stable release model.
Arch suffers from the same problems, or at least it used to. I had a system that was out of date, and it became essentially impossible (or at least very, very difficult) to get it up to date.
I run a rolling distro (Debian Testing) on personal-use machines (laptop; work development desktop; personal server).
I run fixed release distros on production servers/devices. It's important to be able to install a new package on a machine and not have a hundred dependencies need upgrade, break or conflict. And lots of packages are available in backports when you really do need a newer kernel on Wheezy (https://packages.debian.org/wheezy-backports/kernel-image-3....).
> I run fixed release distros on production servers/devices. It's important to be able to install a new package on a machine and not have a hundred dependencies need upgrade, break or conflict.
Isn't that an argument for rolling release? I don't see how this would support using fixed release distros.
Not when you have a few hundred servers. Then, you want consistency. Consistency between servers, between datacenters and between pre-production and production environments.
Now, you can get there with your own repos and a rolling distro. Or you can accept a test/upgrade cycle for a fixed release distro every half year or every year. I personally think the latter is less error prone.
That is what distros like Manjaro / Chakra are effectively. They just pick a date, grab all of Arch, test it for a few weeks, and push it to end users in batches.
I am unfamiliar with Manjaro and Chakra. My experience with Arch is limited to my Cubox-i. It's a nice distro, with an excellent community. I liken it to Gentoo in its glory days.
Having defined my limitation in the observation, I'd just like to point out that, when it comes to distribution stability, for large server deployments, there is safety in numbers. Should a problem occur in a "stable" package, the odds that you are the first one to find the error are smaller with popular server distros (RHEL, Centos, Debian) than with less popular distributions. It's not a statement regarding the intrinsic quality of the distribution. It is a statement regarding the overall quality of the distribution + installed base.
All in all, for a distribution to dislodge entrenched players, for this use case, it will have to be an order of magnitude more stable.
Using a fixed release only requires that you apply the update once every few years, so you can plan for it. You can do it at a quiet time when you are available to fix any problems. With a rolling release you have no idea if something is going to break at an inconvenient time.
Debian Testing branch is always rolling except for a couple weeks/months before a freeze for a release.
I suppose if you really needed to keep rolling you could temporarily switch to unstable.
"Seems like a needless risk and pain."
And the purpose of the freeze is to iron all that out so it doesn't happen.
(edited to add, I'm sad seeing OP get downvoted. His post history shows hes an Arch guy and likely genuinely doesn't understand release tagging. As a Debian user since '97 I am not surprised that there exist both people who don't know the peculiar arcana of release tagging and there are people who are experts at it, so his confusion adds a little value to the conversation. Down arrow should mean a decrement of net worth of the conversation. People are learning things from his mistake, a down arrow is not an assessment that OP got a technical test question wrong. And I had to edit this about ten times to phrase it correctly.)
I don't know much about Debian. I may have slightly mischaracterized things because I don't understand. That's why I asked.
I am getting the impression that Debian Testing isn't really used in the sense of, "Be a nice volunteer and run this thing that has problems so we can fix them," which is what (to me) is implied by the name "testing." Rather, it seems to be "here is a (mostly) rolling release version of Debian, if that's what you want." In other words, it almost seems like "Debian Rolling" might be a more apt name, in a sense.
The thing is, as a user of Arch Linux (without a ton of other experience), I just don't have problems with a rolling release. For me, things don't break, and it just works. So it feels to me like Debian's whole release philosophy is based around the idea that things have to break all the time and be fixed carefully, and that just doesn't sync up with my experience. So I'm trying to figure out what is missing from my worldview.
Thanks for defending me. I think might be somebody (maybe multiple) who doesn't like me and just downvotes all my comments, but I don't have any hard evidence.
I think (could be wrong) that Debian Sid is the nearest to your idea of Debian Rolling. Debian Testing is a testbed for the next release, especially when the freeze happens (and just before freeze when people are trying to get patches in).
You generally want to test changes to your environment before rolling them out, and you generally want to keep disruptions in production as minimal as possible. That's why. It's a need that makes Red Hat alone a billion dollar company.
If you're going to wait a few months to update, you are much better off on an actual non-rolling release distro than Arch. Arch frequently does not test upgrades for packages more than several versions out of date, whereas distros with proper releases will test upgrading from the previous stable version.
Debian testing, however, isn't a proper released distro -- it's sort of a mishmash between a perpetual beta and a rolling release. Debian stable has proper releases, and Ubuntu was started largely as a result of people who wanted more frequent Debian releases.
> If you're going to wait a few months to update, you are much better off on an actual non-rolling release distro than Arch.
I think that's a good point and that makes sense. And I agree with that from my experience. However, I generally think that one should not wait months to do updates.
To be clear, the whole idea with Debian stable is that you get security patches all the time, but no new (or depricated) features/apis. So you update every night, but you upgrade only when a new release comes out.
Compared to that testing does both: typically similar frequency of security-related patches (but not guaranteed!) as stable -- and also migrations of new packages from unstable as soon as they "settle down" (and in "reasonable" sets, so that dependencies work).
So, you want a backported fix for the bash bug, in bash 4.2, but not upgrading to bash 4.3 -- possibly breaking somehting depending on 4.2 behaviour (something other than an exploit for shellshock, that is).
(Now, bash is pretty stable, so may not be the best example -- but the point remains).
If you're running testing, in addition to apt-listbugs, you want to have a look at "aptitude safe-upgrade/upgrade" vs "aptitude dist-upgrade" (or apt-get upgrade vs dist-upgrade). A dist-upgrade can be a little bit more invasive, and typically warrants some more vigilance than a mere "safe-upgrade". I don't think I can remember a "safe-upgrade" ever breaking anything in my ~14 years of using Debian. It's pretty safe to script to run automatically, unless you have very strict policies on uptime/predictability.
I run Debian sid, and I upgrade almost every day, with a few exceptions (like "during or immediately prior to travel"). Doing so lets me catch issues before others hit them, file bug reports, obtain the latest fixes and improvements, and participate in the community that shapes future architectural decisions for the distribution.
> implicit guarantee of "you have to change EVERYTHING
> all at once every few months"
This is a strange conclusion to draw. What makes you think that debian/stable requires updating everything all at once every few months? To be honest this does not even sound like a fair characterization of running unstable.
Funny. I upgraded my ZFS laptop to Utopic with systemd, and everything is more or less fine. There's a few residual issues - I need to install the ZFS targets since zfs-mount doesn't quite have the right mount order (but that's been a problem since well before ZFS hence using zfs-mountall).
But otherwise everything else basically just worked.
TL;DR: systemd breaks badly (at the moment) when having to deal with non-standard setups (ZFS, cryptsetup)...
As for the eternal systemd debate... if someone as knowledgeable as the author (his publications: http://www.complete.org/Publications) can't debug it... we'll have fun times ahead.
> TL;DR: systemd breaks badly (at the moment) when having to deal with non-standard setups (ZFS, cryptsetup)...
Come on, it's a little more specific than that. I have machines with zfs, machines with cryptsetup, all of them run arch and systemd, never had any problem. But of course, I don't have sysvinit scripts.
> TL;DR: systemd breaks badly (at the moment) when having to deal with non-standard setups (ZFS, cryptsetup)...
So you're saying that upgrading a custom tweaked configuration grown over the years to be just perfect for that user is going to break when converted to non-customized stock systemd packages?
The configuration in question has grown over the months and likely used multiple iterations to get right. You can't expect a stock systemd package to be a perfect drop in replacement.
What we really need in order to form an opinion of how badly or how well systemd is going to work, we need real-world examples of systems being upgraded into which only minimal tweaking of the init system has gone.
Like all the Arch and Fedora users who are not really complaining about systemd making their systems unstable or causing issues with day to day usage.
I should be able to expect exactly that, because systemd's supposed ability to work as a drop-in replacement for sysvinit[0] was touted. But suddenly it's unreasonable to expect it to live up to its promises. Convenient.
No. I think the linked article is pretty clear where the problems are. It's basically assumptions made by systemd that are not always true and from the looks of it a possible bug regarding the ordering of service files and reaction to events. Besides that broken or unusable console output without plymouth (I can't really tell from the article if systemd is responsible for that).
That's all fine and possible expected.. my only concern is just that this appears to be very difficult to debug. The author is as knowledgeable as you can hope for in a Linux user and is unable to get meaningful debug output.
All of these things are IMHO legitimate concerns that should be addressed by the systemd community or the Debian systemd packagers.
> we need real-world examples of systems being upgraded into which only minimal tweaking of the init system has gone
I am in that group (having a bunch of debian servers, ranging from "~5 years old upgraded one release at a time" to "~6 months old and installed from scratch"); converted them all to systemd; have found it to work fine and I am enjoying the extra features :)
You are not alone.. after playing around for some hours with CentOS7/RHEL7: https://i.imgur.com/sCiqS2O.png - The bug is fixed now (nmtui: changing the hostname crashed all of NetworkManager...) now I'm always a little scared when using a freedesktop.org project ;)
Look for the names Lennart Poettering, Kay Seivers, and Ulrich Drepper amongst the fd.o project's developers. If you see them, avoid the project. As 90 percent of freedesktop.org projects do not include these developers, you're probably safe, but always good to be sure. The majority of developers on fd.o will happily accept and action bug reports, so don't let these bad apples spoil you on the rest of the bunch.
I'm tired of that name calling. These are just bugs. All these projects fix bugs and honestly I'm personally glad for the work they've done. I don't like their attitude or some design choices but I also lack knowledge. I wouldn't be able to write these software and my complain is mainly a lack of thorough testing. Using C for all these things makes debugging not exactly easier.
Just curious, but what names did I call them? Is using their names not allowed?
I recognize your nick from systemd's forums and IRC channel. Nice to see you here defending your favorite project.
> And make sure to reopen them when they're inevitably marked WONTFIX
Bugzilla is public, the bug changes are also public. Show me that this is a trend. You said it is "inevitably marked WONTFIX", I say you're making stuff up. You raise this, you prove it. Good luck!
You're clearly not paying attention to the "debug" issue in the kernel. Bug was created on FD.o's bugzilla, Kay Seivers labelled it WONTFIX and closed the bug in systemd.
This wasn't the first time, either, but I'll leave that as an exercise for you. Simply look at what bugs in systemd/pulseaudio are closed as WONTFIX. It's a simple query.
Also, I recognize your nick from the SystemD IRC and forums. Nice to see you stopping by to defend your favorite project.
PC-BSD[0] (the desktop-focused fork of FreeBSD) does look like quite a promising systemd-free alternative to Debian. One potential problem is, as always, that the graphics drivers[1] lag behind Linux, so the latest stuff might not be supported.
And until bhyve[2] adds support for Windows guests (which they plan to do) I still need Linux's KVM.
I noticed the other day that KDE 4 (my current favorite WM) is now running on OpenBSD -- I'm due for a new desktop next year, I'm pretty sure it'll be OpenBSD. I've been looking forward to running that as a daily desktop for years.
More eyes and hands means better faster security patches and similar issues. That's why I don't use "three guys and a github" specialized linux distros. If you follow a similar outlook (and what works for me might not work for you) then a crude google search result indicates the community for freebsd is about 30 times more talkative (30 times bigger?) and some more wikipedia searches reveal freebsd is five times the age WRT stability.
I have never heard of Illumos which also implies I've never heard anything bad about it. Although I do personally know people who swear by freebsd, so ...
I would probably use a *bsd to avoid all possibility of systemd issues, and
implies the big "thing" for freebsd is "usable for any purpose" which is the old Debian "universal OS" concept I like so much.
I would strongly consider openbsd for something bare out on the internet not behind a stateful firewall and my limited experience with netbsd is its pretty awesome but being able to boot on a PDP11 necessitates some excessive old code. I'm astounded by retroBSD on PIC32 arch.
Thank you for the explanation--I'm looking at different high-reliability OS's for long-time embedded deployments, and xBSD flavors are looking pretty promising.
For me, package management on OpenIndiana sucks. OpenCSW doesn't support IPS yet, the Oracle IPS repo is way outdated, and the SmartOS/pkgsrc packages don't install/build correctly. Compare this with my FreeBSD setup, where I'm building my own package repository with poudriere (which also handles my build customizations) and easily pushing packages and configuration data with Salt. I was especially disappointed with pkgsrc, since I figured that would be easy to get running.
That said, I haven't had a chance to try other Illumos distributions like SmartOS or OpenSXCE. Maybe the sysadmin experience is better on them.
Volkerding is taking a pragmatic view of all this (as you might have expected). May have to go systemd at some point in the future due to mudballing tendency. [1]
Indeed - for all folks disillusioned by systemd, Gentoo is a source based rolling release distro whose fundamental tenet is choice - so it shouldn't be a surprise that it is possible to use alternate init systems on Gentoo, in fact Gentoo liveCD/handbook defaults to OpenRC while providing the choice to run other init systems (I run systemd with a somewhat complicated RAID setup with no problems).
As a fellow gentoo-user, I am extremely happy with it.... until it's kernel upgrade time. I've managed to forget the set of flags needed to make my system work as desired. So it's a pretty painful process.
run `zcat /proc/config.gz > ~/kernelconfig` (i believe that's the right filename for the average Gentoo kernel; haven't booted Gentoo in a long time) and you'll end up with a full list of all the options you chose for your currently running kernel.
This shows all the config options. Options change from kernel to kernel so doing a simple diff does not always reveal the user's intentional config changes. Otherwise gentoo is painless.. in the long term.
I never used genkernel. When I compile a new kernel version I just eselect it, cd into /usr/src/linux, "zcat /proc/config.gz > .config; make oldconfig" and go through the new options manually. I also have scripts for the modules/kernel installation and the recompilation of packages with kernel modules.
For the specific zfs problem, I can't say for sure (I use xfs), but a simple script you wrote yourself is usually the best. I also had fstab related errors for swap (I use luks) until I disabled the automatic stuff and wrote a replacement script.
systemd is well done and such fixes don't take long, once you understand the logic of it. (usually a symlink to /dev/null to disable the script and a few lines of ascii text for the replacement)
My results: on a thinkpad x60 (2006 era laptop), after coreboot, I boot in about 3 seconds, cf http://www.coreboot.org/pipermail/coreboot/2014-July/078215.... (link to a youtube video of another person doing the same thing). Suspend works very well out of the box, and even better (ex: renewing dhcp leases) after adding a few scripts.
systemd makes many things possible and easy, but with debian at least, the out-of-the-box experience has to be improved. Nothing impossible if you have say a day to spare to learn how systemd works and write your fixes.
Time and again proponents of systemd belittled those of us with compatibility concerns and assured us of its compatibility[0] with existing scripts and setups. Now that it's been adopted, using that supposed compatibility is "doing it wrong". Imagine my surprise.
Now that it's been adopted, using that supposed
compatibility is "doing it wrong".
It has only been adopted in non-Stable distributions of Debian... I haven't seen anyone belittle anyone over "compatibility concerns"; I have seen folks belittled (especially in this thread) when they complain about imperfect stability in distributions which are not Stable.
At the risk of appealing to authority, I'd be interest to see posts from Debian/Ubuntu maintainers in which they express frustration at the various problems systemd is causing them. I haven't seen any and would appreciate hearing from them or getting some links to their posts.
It seems you're rather upset with people who like systemd. That's fine. However, you're putting a lot of words into my mouth. I like systemd.
According to you because I like systemd:
- my opinion is "screw users"
- belittled you with compatibility concerns
- arguments are nothing more than rhetoric.
I find such discussion style rather poor. Instead of diving people into pro/con or black/white (with us or against us), try expressing things a bit more granular. Further, it seems you're out to win arguments by selectively quoting and changing the meaning of what you've quoted.
If there are bugs in systemd, they have to be fixed. I'm one of the biggest systemd advocates. This combined proves you are incorrect on what people liking systemd say or suggest.
If there are bugs in systemd, they have to be fixed.
Philosophically, yes. Realistically? The behavior of some of the people on the systemd bug tracker[1] is the single most convincing argument against adopting it.
It might break things? Sure, it's a pretty massive change.
People are going to have to re-learn how to do a lot? Again, it's a pretty fundamental shift in a lot of ways. This is expected.
The developers will fucking backtalk you[2] and make political issues out of clearcut things? Uh.. no. That's forkworthy behavior. That's dysfunctional enterprise behavior, which is something I'd imagine the average hacker hates instinctually.
And sure, in the end, those bugs were addressed, but it took an outburst from Linus and widespread, public coverage (and I'm sure no small amount of offline discussion among the systemd folks) for it to happen.
Poettering is on record as wondering why systemd attracts so much hate. He only need look in the mirror.
That bugreport is often misunderstood. You only gave one example, and that example is flawed. The bugreport was to change systemd behaviour to avoid a kernel bug. The bug should be fixed at kernel, that is why the notabug was there. Further, it is ack'ed that this should've been handled much better.
I have seen loads and loads of examples where "no" is said in some email conversation based on technical arguments.
> And sure, in the end, those bugs were addressed,
The bug was fixed in systemd git before Linus got involved.
> Poettering is on record as wondering why systemd attracts so much hate. He only need look in the mirror.
Here you're being an ass. This bugreport was not handled by Lennart. Lennart is also not talking about disliking systemd, he's talking about people taking things way too far towards him. He talked about people raising money to hire an assassin.
> It was a bug in the kernel, as Torvalds has already admitted.
Where?
> It's obvious you're not a programmer.
I certainly am, and he's right. Stop trying to deflect blame. The kernel isn't responsible for preventing init from doing stupid things, init shouldn't do stupid things.
And most importantly, when your code does stupid things that breaks users, it is your responsibility to fix it, no matter what you think anybody else's code should be doing. If you can't accept that, you're a menace, and need to stop writing code.
"I guess this does mean that we have to apply my patch to rate-limit messages into the kernel."
......
"+Greg Kroah-Hartman: I really hoped that you could push the trivial and obvious one-liner through. What’s going on here?"
From LKML
"Steven, Borislav, one thing that strikes me might be a good idea is to
limit the amount of non-kernel noise in dmesg. We already have the
concept of rate-limiting various spammy internal kernel messages for
when device drivers misbehave etc. Maybe we can just add rate-limiting
to the interfaces that add messages to the kernel buffers, and work
around this problem that way instead while waiting for Gregs fix to
percolate?"
He even goes so far as to say:
"I don't think we should try to protect against willful bad behavior
unless that is shown to be necessary. Yeah, if it turns out that
systemd really does that just to mess with us, we'd need to extend it,
but in the absence of proof to the contrary, maybe this simple
attached patch works?"
Showing that even if systemd tried to get around the rate-limiting just to piss off the kernel developers, that it's a bug they would fix (which is the wrong stance for whatever it's worth; you should be using defensive programming, not waiting for someone to abuse it first, but at the very least shows that it's a kernel issue).
> I certainly am
Certainly not a very good one.
When a library has an error, is it your fault for triggering that problem, or do you log a bug with the library?
This isn't rocket science.
> And most importantly, when your code does stupid things that breaks users, it is your responsibility to fix it, no matter what you think anybody else's code should be doing. If you can't accept that, you're a menace, and need to stop writing code.
Greg Kroah-Hartman himself (on G+) said "the original bug that was causing the syslog spew to stop the box has been fixed in systemd.".
Please don't contribute to a conversation if you know absolutely fuck-all about it. Spreading misinformation is one of the world's worst problems, and you're simply contributing to it.
No mention of a bug in the kernel anywhere. I see systemd proponents just can't stop lying. Please don't contribute to a conversation when you can't be at all truthful.
Your comment begins with bald-faced lie. I have said nothing about "people who like systemd". You follow it up with another lie, that "according to [me]" your opinion is "screw users".
It is sad but totally unsurprising that you can't find ways to advocate for systemd without stating obvious lies about its detractors.
I find it rather obvious you have something against anyone liking systemd. After above quote, you followup with "screw users". I like systemd, you react in the same silly way to me. Overly aggressive, pretending to be a saint, while painting me with a broad brush, while saying I'm doing that. I find you a little bit crazy.
> I find it rather obvious you have something against anyone liking systemd.
I find it rather obvious that you either don't know what "proponent" means, or you can't distinguish between one who likes something and one who advocates for something. Either way, you are being extremely offensive. Unlike systemd proponents, I don't care what other people like. I care what they try to force me to use.
> After above quote, you followup with "screw users".
I also find it rather obvious you don't have any concept of context. "screw users" was in response to someone who clearly stated they cared about maintainers, not users.
I'm not sure to what extent this is systemd, the author's setup, or the integration work on part of Debian.
To be honest, I wouldn't be surprised if it's the latter. It turns out that a lot of Linux distros really don't have very adequate systemd bootstraps at this point, which you'll find the systemd developers admitting (including, and in my personal experience, especially for Fedora): http://freedesktop.org/wiki/Software/systemd/Optimizations/
When you factor in the fact that Debian and other distributions have some absolutely strange packaging policies concerning when and to where to include systemd as a dependency, coupled with the fact that most distros still don't have good mechanisms for separating hard and soft dependencies, and you get some pretty confused default configurations.
I once tried compiling and booting from systemd on a Lubuntu box, out of my own curiosity. It worked, and there were already some predefined unit files from systemd-shim and various system packages. Performance was good. I tried upgrading my system... the process is absolutely broken, due to preinst/postinst scripts assuming Upstart. Sometimes a package will get successfully unpacked, sometimes it won't. Error messages galore.
Considering systemd's extensively rapid evolution, I don't see integration work in most distros stabilizing or being optimized very soon.
I have been half-following this whole saga and as far as I can tell, it's a shit-show on a scale that runs the risk of turning entire distro major releases into complete write-offs. I just hope they only go one release cycle before deciding to abandon it. I'll get on the last pre-systemd LTS and hold my breath.
They don't see the benefit of keeping critical system components simple, they don't listen to criticism, they make promises they can't keep, and they've bitten off way more than they're capable of chewing. The only part I really can't figure out is why major distros have actually decided to make the switch way before any of the tech is ready.
One of the things they're planning next is to replace the current kernel VT implementation with a userland terminal emulator that depends on systemd, so you won't even be able to boot with init=/bin/bash in a pinch because without systemd there won't be any terminal to run it on anymore. No idea how people will be expected to diagnose and fix issues that break boot at that point.
Pick a 'usb stick' distro of choice (my choice is usually sysrescuecd) and boot that image off of media or virtual media if you have one of those remotely managed systems. Though it sure would be nice if said systems could pull the image off of a mirror local to the DC.
Alternatively, build a similar rescue option in to your boot menu; if that's broken you've got no choice anyway.
USB rescue distros are nice, but that assumes you have a USB stick to use and already had the rescue distro installed. That doesn't help when you are trying to fix some failed update late at night with the computer in question being the only hardware available. Contrived example? Yes, but I've been in more or less that situation a couple times in the past and was very thankful that init=/bin/bash worked as a fallback.
// yes, most of the time I just put fallback/rescue options in the boot menu
Cryptsetup was known to require a bit more work. Debian has some specific stuff for that AFAIK. Same thing with ZFS. Just things that need fixing.
I find the entire blogpost rather encouraging. Let's get those bugs fixed as well! Doom and gloom is rather weird attitude IMO, it's on loads of distributions and for various releases of those distributions. It is not without bugs, first release within any distribution is going to have a few issues. Fairly logic that with aim of Debian (everything has to work), it'll take a bit longer.
systemd intention is great but aftermath is that of talented albeit inexperieced developer writing it. Fact that i have misconfigured one of the services, should not lock me out of machine to the point of needed to boot from usb flash stick. I am not as advanced but that was what I got when put coreOS with systemd on one of our servers - thankfully not production.
I think it was hello world service but after tooling around starting and killing it , rebooted my machine and it machine froze solidly on reboot said something like unable to find mountable systems. but the systems were there with flash drive everything was intact. I've just picked up coreos and I think stuff like while(1); echo "hello world"; sleep 1; maybe it was not that or something else but setup was stock stable 431.
I don't use Linux on the desktop so my experience with systemd is strictly server-side but it just feels like too much magic. I'm used to (and like) Solaris/OI's SMF so I'm certantly not in the "init scripts for life" camp.
Just saw the all-hands call on the IRC channels and systemd fanbois groups for an epic show of whiteknighting required yet again on HN where rational folks are trying to discuss the actual and real downsides of systemd - quick, post the rebuttals, choose from:
1. You're doing it wrong
2. Of course, if you use <insert something here>, then it won't work!
3. If you spend a month, it is simple! What, are you lazy?
4. If you hack these bits, disable these bits, run some pointless commands here, then presto it works!
