With systemd (and afaik other service managers) you don't need/should to do all that daemonization dance. The application can run in the foreground, keep any fds open, don't worry about tty etc because the service manager will take care of them for you. As a cherry on top, anything you write on stdout gets automatically forwarded to journald or wherever.
The more I use systemd the more I love it. It simplifies so much, and adds all the little features I see so many projects using custom implementations for.
The people who keep complaining about it seem more and more like children who just refuse to learn something new and better because they've already stumbled through all the holes it fills.
> The people who keep complaining about it seem more and more like children who just refuse to learn something new and better because they've already stumbled through all the holes it fills.
Systemd apologists always seems to think that anyone not completely in love with systemd, is just refusing to learn something new. Please stop with the stupidity, people have real reasons to hate that shit.
I had no problem with switching to WireGuard, after learning about it, because this is so much better in both use and design than OpenVPN that I have used for 20 years. I had no problem with learning about systemd either, but that is just badly designed software to me.
Just disable the parts you don't like on your system. I disable resolved everywhere I can and don't feel the need to go complaining about it. Where are you being labeled a systemd hater? Possibly it's more tone of voice than what you're complaining about.
I do, where I can. But with our clients It's often impossible (due to politics) to control all of the clients on the network, so I can't disable parts I don't like.
So I have to find workarounds, like intercepting DNS queries on network level, to fix what used to work before.
And if dnssec ever gets implemented, that wont work either.
Sure if I have access to the host, which I often don't[1].
I wrote whole wall of text but at the end of the day, its not that big of a deal,
It's more annoying explaing to customers that they have to talk to their other vendors to fix their configs, so sometimes just network level hack is easier.
[1] In enterprise environment and even some SMB environment its somewhat common (at least here in South EU) that big vendors just drop black boxes to you (usually in a form of vmware image, lately sometimes Docker containers). A lot of them are just stock RH, or Ubuntu with their software. ANd that comes with default fallback to goolge or cloudfalre DNS's
What has systemd-resolved got to do with the fact that you have to use black-boxes? If they had used no DNS management at all and instead just dumped entries in /etc/resolv.conf, you'd be just as SoL as you are now with systemd-resolved. If you can write to /etc/systemd, you can set whatever DNS config you want in the config file for systemd-resolved. I fail to see how our woes with bad software products have anything to do with systemd-resolved.
Most black boxes are just default linux distro with their vendor software on top.
Before systemd, you just needed to set correct DHCP config and it would all just work.
But nowdays distros come with systemd-resolved which usualy has (by deafult) fallback public DNS servers.
That means that boxes suddenly can switch from your DHCP network provided DNS servers (or even static DNS servers) to goolge (or cloudflare,) public DNS server.
Bottom line is, it used to be enough to set DNS server through DHCP (or static ones) that is no longer enough in some situations.
I'm not entirely certain that it is systemd-resolved's fault that the distro maintainers are setting default servers in the config. I'm not disputing that RHEL might, but debian certainly does not, nor any of the distros I've used recently, but I probably have a completely different perspective as I have to deal with more end-user focused distros. I do feel your pain though, DNS config management seems to be incredibly hard. And this fuckery where a OS default setting overrides the DHCP config you're pushing would be incredibly painful.
> I like the init part, i could do without everything else.
Then, do exactly that? None of the components you're complaining about are mandatory (except for journald, but you can easily forward log messages to a regular syslog daemon).
I actually thought that systemd is the init part.. Regarding DNS, is the systemd resolver just not advanced/flexible enough? I.e as a desktop user I can't complain about anything, but I don't configure it much either.
As a nitpick, I think "cohesion" and "coupling" are terms coming more from software engineering, I'm not sure they are objects of study in the computer science branch of mathematics.
Which isn't even correct because systemd consists of many individual parts that do their thing and are replacable. But because they all share the "systemd-*" naming pattern people think they're the same piece of software.
Would be funny if people made the same complaint about the various GNU core utils because those are all installed from the same package on most (all?) distros.
Please show independent working implementations of those "replaceable individual parts". AFAIK people who tried to do it found too many obstacles like undocumented or inconvenient API.
Sure, but that's like saying my coffee machine also makes latte and hot chocolate. I guess what I'm asking is: do any of these taste bad, and how? Perhaps the parent had a few of these on his/her mind that could have been listed.
Unix philosophy has a reason. It allows to easily replace components and simplifies testing. AFAIK it's not easy to replace various parts of systemd, it's very monolithic and it does too many things.
> AFAIK it's not easy to replace various parts of systemd, it's very monolithic and it does too many things.
You can easily replace timesyncd with any NTP daemon. You can easily replace networkd with any other networking manager. You can easily replace timer units with cron. You can easily forward log messages to a syslog daemon. You can easily replace systemd-boot with grub or any other boot manager.
It should be pretty obvious how baseless the monolithic arguments against systemd are.
I know the philosophy. What I'm trying to get at is that perhaps the thing that came before was a set of things that didn't quite work together, so now they made something that does.
(And was just wondering if there was any specific thing the original parent wanted to replace.)
I was a fan of upstart. It was simple, it knew how to perform the task it did it, it didn't try to be everything plus the kitchen sink, it worked well and it didn't get its tendrils in everything like systemd does (something that concerns me as it increases the attack surface).
It was a technically superior solution murdered by politics.
What's worse was all of the people who were going "why are you moaning? what's wrong? systemd is still an improvement on sysvinit!" as if being better than a bunch of bash scripts held together with bits of sticky tape and string was all the qualification needed to be PID 1.
And yes, people do erroneously paint it as only a choice between van Smoorenburg init+rc and systemd, as if they were the only two possibilities, when in fact in the Debian Hoo-Hah (for one) it was effectively a choice between systemd, upstart, and OpenRC, nearly all parties agreeing at one point in the proceedings that van Smoorenburg init+rc was last choice, either before or after "further discussion".
The criticisms levied at upstart there are more about lack of features, not really about overall approach. It's easier to add features than it is to fix a deficient design.
Moreover, I'm not really so sure a couple of those features even belong in an init system.
Debian's decision effectively doomed upstart and led to an init system monoculture. The practical upshot is that no competition for systemd means no pressure to improve. Even for systemd it was a bad decision.
Actually it was about overall approach. Read M. Allberry's further discussions in that bug with Colin Watson, Steve Langasek, and others. Josh Triplett critiques it as well. Cameron Norman's discussion with Nikolaus Rath is also relevant. There were actually quite a lot of technical discussions about the fundamentals of the various architectures. Ian Jackson's and others' discussions about readiness protocols is worth reading, for example.
One cannot truly portray this as either "politics" or only about lack of features.
Why do you stumble up on holes, when all you need to know to make a daemon is described in APUE in detail and in hundreds of books and articles? Making a daemon, either via systemd or a classical way, is learning something new.
"We need a process without a process group, session, controlling terminal and mount point. How to do that? Well, fork two times, setsid, chdir, close all descriptors. Let's call that daemon for short."
If one does not understand what all of this really means, or why all of this is part of a question, who do they blame against learning something new?
And why, pray tell, must this logic be duplicated (poorly) in every single "service" application? Isn't the UNIX way "do one thing, and do it well"? Isn't the whole "I, the author of the application, will forcefully disregard the environment the user provided to it" against the UNIX spirit of composability? No UNIX application that I know of re-opens the controlling tty as its stdout — that breaks pipes, so is heavily discouraged and nobody does that. Well, "daemonizing" breaks job control/child process control/service managers, and yet people insist on doing it.
Not everyone insists on daemonizing, and not everyone nails their services to whatever popular environment there is. Make it a standard then complain. My point is that doing that also is "learning something new", as opposed to "being a luddite disrespecting our systemd". I'm using systemd and other service managers on all linux instances that I have and with all self-written services, btw, it's just this cool kid stance that bothers.
Also I just downloaded the recent sources of sshd and it does daemon(0, 0) under the hood, unless -D is passed. ssh.service just runs it with -D, so what's the deal? Does one function call for the case when systemd is not there break some religion?
Forking however many times you want is not going to guarantee that your are spawning a daemon. The process will end up being owned by its nearest sub-reaper, which could in theory still be owned by the initial session.
That implies that daemons always start from a login session and under a process which does this specific trick. That's not true, unless you explicitly wanted them to malfunction.
My point was not that this happens often, but that it is not guaranteed to work (whereas adding your daemon as a systemd service unit is, on systems using systemd).
Also, if you're not starting from a login session, do you really need to double fork?
It is rather sad to see something that was out of date in the 1990s still being circulated today. Even the part about process #1 inheriting orphaned processes is 8 or so years out of date.
Has POSIX ever defined “service manager” as a concept? https://news.ycombinator.com/item?id=11798328 points out that glibc’s daemon(3) function isn’t standard but might be a good place to abstract out “do whatever is appropriate for this distro.”
> Simply put, a daemon (pronounced dee-mon) is an intentionally orphaned background process. Additionally, daemons are detached from the terminal in which they’re spawned, operate without user interaction, and are descendants of the system’s init process.
A memorable way to think about this is that a daemon is a child process whose parent has literally disowned it, forked off, and died.
... except that that's most definitely not the defining characteristic, and the defining characteristic is rather that the process isn't part of a login session, which ironically on modern operating systems with all of the one-way trapdoors that now exist one cannot reach by forking.
Thinking that forking makes a dæmon is much the same error as thinking that su drops privileges, it conflates what happend to be a mechanism from years ago with the actual purpose, totally missing that things have changed since (long since, decades ago in fact).
Of all the great content in APUE, I think W. Richard Stevens’ (RIP) coverage of daemonization is the only piece I’ve ever used.
Fortunately I recently replaced my lost 1st edition with a 2nd edition, and the coverage looks similar. From what I can see there are two details left out of this article:
* Why changing the working directory is important (so the daemon doesn’t accidentally prevent a file system from being unmounted in the future).
* WRS emphasized logging.
Some of the details are different (umask as the first function instead of the last), but the highlights are comparable.
I was hoping this would be an article for beginners to understand daemons, as in the Ancient Greek daimon (δαίμων: "god", "godlike", "power", "fate") :-)
Growing up on a (quasi fundamentalist) christian household, daemons were for a long while a very real threat, and my mom would not allow me to watch horror movies, "receive things from strangers", an other procedures to protect us from "inviting the daemons home".
My rational mind leaves no room for such creatures these days, but even so under the right circumstances, sometimes my lizard brain begins to think maybe the Bible (and my mom for that matter) was right :-). Hard to erase some emotive memories from childhood I guess.
A while ago I read this book [1] which is pretty cool, covers the mind mechanisms behind things like the placebo effect, "daemonic possession", alien abductions, mass hysteria, out of body experiences, etc...
King James is more famous for the Bible he commissioned. And his treatise on True Law and the divine right of kings. This work of his is somewhat less well-known, I think.
Usually I see δαίμων latinized as "demon" not "daemon". You got me looking and I found an interesting discussion on these words over on stack exchange [1]. TL;DR, "daemon" sometimes means "demon", but without the negative connotations, other times it is just an alternative spelling.
Oh! The good daemons! "benevolent or benign nature spirits...". Not in the Judeo-Christian tradition, for sure :-)
Also got me curious about the C.S. ethymology of the word. According to the venerable Jargon File (which has entries for both "daemon" and "demon"), it is named in honor of one of Maxwell's thought experiments! [1]
Sorry you don't share my sense of humor! I've seen countless conversations about psychology subjects here at HN so I thought someone would find this subject interesting.
This function may do mostly the right thing in a process dedicated to it, but it is not safe to call as part of a more complex process - especially a multi-threaded one.
For example, malloc() may be holding on to a mutex in the parent process (on another thread), so subsequent malloc() calls in the child process will block waiting for the mutex to be released. Closing the file handles may help if this was a kernel mutex, but it may not help if this was a user-space mutex.
In general, fork() without exec() is only safe as the first thing done in a process.
Also, it looks like there is a minor race condition in the implementation: since the original process exit()s immediately after the fork(), control will return to its parent, which may close the session - this could happen before the child had a chance to call setsid() and detach from the parent session.
A good practice for a classical daemon and any program in general after closing all descriptors is to reopen first three (0,1,2) to /dev/null. After that, any forgotten or unexpected diagnostics (rare printf in third party library) would not corrupt your open files.
No, that's a bad practice, because since the late 1990s service management subsystems have been as the norm setting up standard output and standard error so that they are connected to loggers. daemontools did it, once it gained svscan; and so did the other toolsets in the daemontools family such as runit, perp, s6, and others. systemd also does it, but is very much a latecomer here.
Furthermore, this bad practice lands one in hot water with library functions, including ironically syslog client libraries, that open and retain file descriptors internally for their own purposes. Yet another case of this was just on Hacker News this week, in fact, at https://news.ycombinator.com/item?id=24329540 .
That's still a good practice, if you close descriptors. Such criticism and trouble comes from environments that are unable to follow the routine because reasons, e.g. they openlog() and construct other objects before daemonizing. Given how much can happen before python's main() to be called, it is obvious that daemonizing python-anything is a bad idea to start with. It only works for few controlled steps at int main(), after that you're screwed. All these pitfalls are discussed in literature, the real problem is that barely anyone bothers to read it and think for a second on their options. One of them being to use a proper service manager instead, like systemd.
> Conventionally, daemon process names end with the letter “d”. If you’re interested in seeing all daemons installed on your Linux machine, use this command:
> service --status-all
$ service --status-all
bash: service: command not found
Perhaps you're trying to make a point, but the command should work with /usr/sbin/service or prefaced by sudo. If you run `whereis service` you can see that the binary is located in /usr/sbin (at least on Debian/Ubuntu systems) which is typically placed in the PATH of root but not normal users. However --status-all doesn't require elevated permissions so the command works if you provide the full path.
A bit of both. My computer does not have the "service" command at all. While it's true that many systems still do, it's typically there as a legacy compatibility layer, to translate SysV style commands to systemd.
So while the command might work for many readers, I don't think it's a good idea to send beginners that way, and its inclusion (without any footnotes or other qualifications) suggests that the author's knowledge is out of date.
So how does this work in windows?
Edit, as for some reason it's getting downvoted:
In windows, do you have to do the double fork etc? Do you have to close FDs etc?
Services are also regularly run on different account especially the ones under "NT AUTHORITY" like "SYSTEM" and "NETWORK SERVICE".
As I recall Visual Studio contains some pretty good templates for writing services with C# but the underlying requirements are abstracted by the .net Framework.
There’s some Win32 stuff you can hook into with your code for service management. These are things like service startup, shutdown, pause and resume. I can’t speak to the Win32 stuff, but .NET abstracts this and you just override a base class to hook into the service stuff. Windows then has a command `sc` to register your EXE as a service and then configure things like auto-start and service restarts. Alternatively, MSI’s can be crafted to configure your EXE as a service at install time.
IMO a lot of parts of Windows are a pain to develop on compared to Linux, but creating and managing services are one of the better/smoother parts.
Windows doesn't have fork() or any direct equivalent (instead, Windows has the CreateProcess() familiy of functions, that work essentially like fork()+exec()).
Now depending on what you actually want to do, you have some options:
- Use the Windows Services API to ask Windows to start and manage your process as a system service - this is closest to a SystemD demon
- Create a GUI process with WindowStyle=hidden (this will be part of the current user's session)
- Create a Console process with the DETACHED_PROCESS flag, which should work similarly to the above
- Use one of the CreateProcessAsUser() family of functions to create a process for another user - this should allow the process to run in the background, and allow it to persist even if the current user signs off
In all cases above, you can simply tell CreateProcess* that the child process should not inherit any FDs. You should probably also use the PROC_THREAD_ATTRIBUTE_PARENT_PROCESS flag to spawn this process as the child of some other long-running process, to ensure that the child process is not part of the same Job that the current process may be (usually, when a Job is finished, all processes in that job and any child they spawned, recursively, are killed).
Jobs, more precisely, nested jobs, are one of the best things (IMHO) that were added to Windows 8 from a technical point of view (then again, there ain't many good things added in Windows 8). I had to write a work-manager sort of application that would need to start worker subprocesses (which may also start some other subprocesses, which may or may not be programs under my control) and one of the requirements was that if the main process crashed, all the worker processes should be terminated (imagine if a part of what a worker process is supposed to do is to listen on some well-known TCP ports and provide an HTTP API on it: you can't restart the main process without killing the workers off first).
On Linux, it turned out to be surprisingly tricky. Process groups and sessions are not nested, you can easily break out of them and in fact there were some programs we needed to use that insisted on doing daemonization. Thankfully the most important one had a "--foreground" flag but it still was buggy: this program sends SIGTERM to its process group as part of a shutdown — so it kills its parent, too. Amazing. In the end I believe we ended running each worker process as a separate Docker container.
Then there is a dance of waiting (or not waiting? SIGCHILD is truly horrible) on children PIDs to detect the crashes and unexpected hang in the shutdown (the worker process sends "I am done" notification to the parent, but then for some reason doesn't exit). The main process had to be made a sub-reaper to do all of that more or less reliably, and even then there were some strange sightings (signal handling is fun!).
On Windows, we just shoved each worker process into its own unbreakable-from job, and that was it. The main process dies for whatever reasons, its children just disappear. A worker process dies, its children just disappear, without affecting any other worker processes and their children. Detecting termination of the child processes too was a breeze: on Windows, when you spawn a child process you get not only its PID, but also a handle (think "file descriptor") to it. This handle refers exactly to this process, and you can (interruptibly) wait on it, and when the last open descriptor to a process is closed, the process is deleted from the process table. That's why BTW there is no PID 1 that is constantly reaping its children on Windows: each process implicitly has its own handle alway open, so if the parent exits before the child does, nothing bad happens: The child process had 2 open handles on it, now it has 1. When it itself exits, the last handle is dropped, and the slot in the process table is cleaned.
TL;DR: Process management is horrible, use Erlang.
The Service Control Manager (SCM) has a list of settings for registered services: the parameters of a service are its name, executable path, arguments, environment variables, user to run it under, other services it depends on, restarting options, etc. When the system is started, the SCM starts those services. When the system is being shut down, the SCM stops them.
How is a service started? The SCM launches the specified executable, in the way it's specified in the service's settings and... that's pretty much it.
However, the service is supposed to support a certain protocol of interaction with SCM. First of all, the service must quickly (within half a minute from the start, IIRC) call the WinAPI function StartServiceCtrlDispatcher() on its main thread. That's basically a pre-written main(): it returns only after the service has been stopped. It takes several function pointers: to the ServiceMain where the service proper is supposed to be, and various event handlers (pause/resume/stop requests, system is shutting down, etc). It creates a separate thread to run the ServiceMain on, and then basically loops processing requests/events from the SCM (event handlers are run on the main thread).
That's basically it. Nothing prevents one to write an application that can run both as a service and stand-alone: put your logic in ServiceMain, call StartServiceCtrlDispatcher with it, if it returns ERROR_FAILED_SERVICE_CONTROLLER_CONNECT (that means you're not being launched by the SCM), call ServiceMain by yourself. Ctrl-C handler and "stop the service" handler are easily unified because on Windows, Ctrl-C handler always runs on a different thread. You just need make sure that your ServiceMain function can be safely signaled that it needs to stop from a different thread, but that's what thread-communication primitives are for.
They have a double fork - forked tongue and forked tail, and often cannot be killed through normal means, re-spawning with the aid of an even greater evil known as SystemD.
further reading https://www.man7.org/linux/man-pages/man7/daemon.7.html