Anatomy of a typical Unix-like/Linux daemon

This short blog post should provide a short introduction into the anatomy of the typical Unix or Linux daemon. It describes the typical steps taken by a daemon to demonize itself.

• What is a daemon process anyways?
• Which steps does a daemon undertake?
  • Alternative flow
• A basic skeleton

What is a daemon process anyways?

A daemon process is a process that doesn’t directly interact with a terminal or graphical user interface, runs in the background and provides some service. Typical examples are webservers, mail servers, chronological job schedulers (cron), your audio server (like jack, etc.), the syslog server, etc. These processes normally expose a service and one or more methods of interaction. They are of course different to batch jobs that often also run inside demonized containers.

Which steps does a daemon undertake?

• First of most daemons do sanity checking on configuration files.
• If a PID file is used open if using pidfile_open. This opens or creates a PID file and tries to lock it. In case the file has already been present and the file could not be locked the function will store the PID of an already running daemon (or -1 if this daemon hasn’t written it’s own PID yet) into the supplied pidptr. In case no path is specified the pidfile will be put in /var/run/<progname>.pid. The file will be opened with O_CLOEXEC so the handle will survive calls to fork but not exec.
• Then they fork for the first time. If this is successful the parent process will simply terminate. Note that the child is now running in the background but is still attached to the same process group (i.e. when the controlling terminal terminates, the process will be terminated too but the foreground process now again accepts user input - the same way as when one sends a process into background that one can pull into foreground again)
• Now the process will create a new session using setsid. This now detaches the process from the process group - itself will be the new session leader so it won’t terminate in case the previous parent process terminates.
• To ensure detachment from all previous session the process will fork again since only the new session leader could have been attached to the previous session in any way. The parent process will - as usual - simply terminate.
• Then the process normally registers some signal handlers. These normally include:
  • SIGHUP which is mostly used to signal daemons to restart themselves or re-read configuration. Under normal (non daemon) circumstances this signal would have signaled the termination of the controlling terminal - the name originates from hangup.
  • SIGTERM is used in the usual way. It’s a termination request that should normally lead to a graceful termination of the service.
• Now many daemons open resources that won’t be accessible later on (for example because they limit themselves). This includes configuration files, privileged sockets (that is necessary if one has the concept of sockets that can only be bound to by the root user for example - this is of course not needed in case one has a correct portacl rule for the user one’s running under). One might also open configuration files that are existing outside the later chroot
• In case daemons have to modify mandatory access control properties this has to be done now (if launched as root, if not this would have to be done by a startup script or any other administrative measure). This includes (note: most of the listed stuff is FreeBSD specific):
  • Modifying firewall rules (this is normally done by the administrator if the application isn’t some kind of wiretapping or NAT daemon)
  • Setting mac_portacl rules to allow access by non-root daemon users to privileged ports (this is normally done by the administrator of a given system and not the application)
  • Loading of mandatory access control modules like NTPDs mac_ntpd. This allows one to grant specific privileges (like PRIV_ADJTIME or PRIV_NTP_ADJTIME, etc.) to a specific process. This way a normal unprivileged process is capable of performing some specific actions that won’t be possible with a traditional Unix-like systems. Note that policies can do way more (implement new syscalls without global syscall registration or wrapping syscalls, controlling access to bpf sockets, controlling access to labels and labeling objects with MAC labels, perform privilege checking, wrapping socket operations, etc. - if one’s interested in developing a MAC module one should look at the easy example of mac_ntpd, some module like the mac_portacl or the other currently available mandatory access control modules of the base system. But please be ware that an compromised MAC module will totally compromise your system and any MAC module is a really deep and extensive, powerful modification of normal system behavior.
• The daemon should close handles to stdout, stderr and stdin and re-open them referencing /dev/null or some debug target stream like a logfile, etc.
• Write one own’s PID into the PID-file
• The last step most daemon initialization code contains is limiting it’s own privileges. This is done using a combination of the following calls (in the listed order):
  • jail (FreeBSD specific) will create an lightweight virtual machine or container the application runs in. It gets it’s own set of virtual network adapters, it’s own hostname, it’s own IP addresses, etc.
  • setgid sets the real and effective group IDs and the saved set-group-ID of the current process. Again this is most of the time used in case the process has been launched by root. If possible in any way one should refrain from this method and launch the process directly as the required user.
  • setuid changes the real and effective user IDs and the saved set-user-ID of the current process. Note that the process has to have the privilege to to that. This is most of the time used to impersonate a given daemon user after startup in case the daemon is launched by root. If possible in any way one should refrain from this method and launch the process directly as the required user.
  • chroot changes the filesystem root to a given filesystem. Direct access to files outside the chroot won’t be possible any more.

Alternative flow

An alternative flow might substitute the first fork and the call to the setsid function as well as closing stdint as well as stdout and stderr pipes by calling the daemon(3) function. This function allows one to move into background, detach from the controlling terminal and optionally do a chdir into the root directory / and optionally redirection all standard input and output pipes to /dev/null. The following example won’t use that approach.

How to provide logging from inside a daemon process

There are a few different approaches one might take:

• Writing into a logfile (one should re-open them when being signaled via the SIGHUP signal or any other configurable signal or be prepared that logfiles simply vanish when newsyslog rotates them). This approach is often taken by modern daemons like Java servlet containers, webservers, etc.
• Writing into a syslog socket. This approach allows a centralized logging infrastructure. One should allow customization of logging facilities though. This is the approach taken by most traditional daemons like cron, sshd and other basic system services.
• Send log- and statistics information via AMQP or MQTT to a message broker that routes them to logging and analysis microservices. Note that one should of course provide some other means of logging until this service connection has been established.

It’s a good idea to provide a way to run the process for debugging purposes in foreground (and of course also allow termination using the SIGINT signal - that’s the signal that gets transmitted when one uses CTRL+C on the controlling terminal). This allows easy debugging under many circumstances during development and manual deployment scenarios.

A basic skeleton

With all that in mind we now can develop a simple basic daemon (the code is provided as a GitHub GIST):

This article is tagged:

• Programming