Docker Tip: Run Daemon Service In Foreground

Running daemon services in foreground helps for trouble shooting. Also a very useful trick for container environments.

Here is the general practice. And some real examples of popular services, like Apache, MySQL and Jenkins.

How To Run Daemon Service In Foreground



Update Per Audience Feedback:


1.1 Why Run Process In Foreground?

Usually we don’t. Running process in Daemon mode gives us a lot of benefits. Service can be easily managed. Remember “service XXX start”, “chkconfig XXX on”?. When terminal disconnects, the process remains.

However sometimes services just fail to start. We may want to run the process in foreground mode. To understand why? Say duplicated global envs, incorrect conf files, missing folders, access denied, etc[1]. Related reading: How To Check Linux Process Deeply With Common Sense.

In container envs, each container usually host one and only one process. One trick to lower the operation cost. When containers start, the service automatically starts. When service stops, the container stops. To achieve this, we’d like find a way running process in foreground. Then change the entrypoint to the start command.

Take redis container for example.

From:

docker run -t -d --name my-nginx nginx /bin/sh
# Start containers, then start process
docker exec -it my-nginx bash
service nginx start

To:

docker stop my-nginx; docker rm my-nginx
# When container starts, start process inside
docker run -t -d --name my-nginx nginx \
  /usr/sbin/nginx -g "daemon off;"

docker exec -it my-nginx bash
ps -ef | grep nginx

1.2 Figure Out The Start Command To Run in Foreground

Daemon Service = Binary + Configuration + Initscript

To run process in foreground, we just need to the first 2 ingredients.

  • Binary: bash -x /etc/init.d/XXX start. Here “-x” instructs shell to display verbose output. Should be easy to figure out binary path. Another trick: “ps -ef | grep $PROCESSPATTERN
  • Configuration: Typically speaking, we can dig out from console output of “bash -x”. Sometimes it’s just too long to read. Then figure out the process id. Then list environment from cat /proc/$pid/environ

Lots of decent services refuses to run as root, for security concerns. Here is how to impersonate other users with root.

su -l elasticsearch -c "java XXX"

  • Sample output of “bash -x /etc/init.d/jenkins start”
root@denny:/# bash -x /etc/init.d/jenkins start
bash -x /etc/init.d/jenkins start
+ PATH=/bin:/usr/bin:/sbin:/usr/sbin
+ DESC='Jenkins Continuous Integration Server'
+ NAME=jenkins
+ SCRIPTNAME=/etc/init.d/jenkins
+ '[' -r /etc/default/jenkins ']'
+ . /etc/default/jenkins
++ NAME=jenkins
++ JAVA=/usr/bin/java
++ JAVA_ARGS=-Djava.awt.headless=true
...
...
+ '[' no '!=' no ']'
+ ulimit -n 8192
+ '[' -n '' ']'
+ /bin/su -l jenkins --shell=/bin/bash -c
'/usr/bin/daemon --name=jenkins --inherit
--env=JENKINS_HOME=/var/lib/jenkins
--output=/var/log/jenkins/jenkins.log
--pidfile=/var/run/jenkins/jenkins.pid
-- /usr/bin/java  -Dhudson.diyChunking=false
-Djenkins.install.runSetupWizard=false
-jar /usr/share/jenkins/jenkins.war
--webroot=/var/cache/jenkins/war
--httpPort=18080'
+ START_STATUS=0
+ do_check_started_ok 0
+ sleep 1
+ '[' 0 -ne 0 ']'
+ get_running
++ grep -c .
++ egrep -e '(java|daemon)'
++ ps -U jenkins --no-headers -f
...
...
+ return 0
+ exit 0

1.3 Run Apache/Jenkins/MySQL In Foreground

Using above technique, we can figure start command running popular services in foreground.

  • Run apache in foreground [2]
cd /etc/apache2/ && \
apachectl -d /etc/apache2 -e info -DFOREGROUND

  • Run Jenkins in foreground
/bin/su -l jenkins --shell=/bin/bash \
-c '/usr/bin/daemon --name=jenkins \
--inherit --env=JENKINS_HOME=/var/lib/jenkins \
--output=/var/log/jenkins/jenkins.log \
--pidfile=/var/run/jenkins/jenkins.pid -- \
/usr/bin/java  -Dhudson.diyChunking=false \
-Djenkins.install.runSetupWizard=false -jar \
/usr/share/jenkins/jenkins.war \
--webroot=/var/cache/jenkins/war \
--httpPort=18080'

  • Run Mysql in foreground
/bin/sh /usr/bin/mysqld_safe

  • Run Nginx in foreground [3]
/usr/sbin/nginx -g "daemon off;"

[1] ma.ttias.be/varnish-running-in-foreground-but-fails-to-run-as-servicedaemon/
[2] http://zroger.com/blog/apache-in-the-foreground/
[3] http://honeyco.nyc/blog/running-nginx-in-the-foreground/

More Reading:

linkedin
github
slack

PRs Welcome

Blog URL: https://www.dennyzhang.com/process_foreground


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.