Get Alerts, When Containers Run Into Issues

I’m running docker containers for all side projects. Usually one single container. Sometimes multiple containers started by docker-compose[1].

If containers get issues, I want to get alerts.
What a typical monitoring requirement! But if you do some research, you will find the information is just overwhelming. cAdvisor, Prometheus, InfluxDB, etc. Excuse me? Can’t we have a simple solution for this simple requirement? Here is my answer. Try it and discuss with me.

---

---

This solution works for docker-compose deployment. Please check Prometheus[2], for more complex docker environments.
BASIC IDEA
Firstly I admit: Prometheus can do the job. Yes, it’s capable, but …
Hint: Want to use prometheus in a relatively easy way, try dockprom [3]

Frankly speaking, learning Prometheus took much longer than what I had expected. And it’s a bit over-killing for my scenario. Prometheus solution itself will start multiple containers. But my envs only have very few containers to monitor.

Ironically “docker ps -a” can easily answer most of my questions.

root@denny-blog:~# docker ps -a
CONTAINER ID    IMAGE             COMMAND                  CREATED         STATUS                     PORTS                   NAMES
2936e1d7ec34    denny_wordpress   "/bin/sh -c /docke..."   3 weeks ago     Up 13 days (healthy)       0.0.0.0:443->443/tcp    blog-wordpress
87665b7a76df    mysql:5.7         "docker-entrypoint..."   3 weeks ago     Up 3 weeks                 3306/tcp                blog-mysql
4aa821fa9d37    denny/ss:v2       "ssserver -c /etc/..."   3 weeks ago     Up 3 weeks (healthy)       0.0.0.0:6187->6187/tcp  shadowsock
ea96f4dbefaa    ubuntu:14.04      "/bin/bash"              14 seconds ago  Exited (0) 6 seconds ago                           nginx-test

1. To get stopped container: just check “STATUS” of each container.
2. To get unhealthy containers: check status for “unhealthy” output.
   Hint: To enable docker healthcheck[4], you need update your Dockerfiles properly
3. Send alerts, whenever any of above occurrences have happened. Should be easy, Right?

So how about we start a container, which keeps polling status of all containers. Then send out alerts, if necessary?

You may think: we won’t be able to run “docker ps” in containers. It only runs in docker host.

Yes, you’re right. But there is one way out. You can query status via docker unix socket file. Try below locally, you will understand.

docker run -v /var/run/docker.sock:/var/run/docker.sock \
    -t -d --privileged --name test-socket \
    --entrypoint=/bin/sh byrnedo/alpine-curl

docker exec -it test-socket \
    curl -XGET --unix-socket /var/run/docker.sock \
    http://localhost/containers/json

HOW TO USE
Check code in Github. GitHub

Try it, if you find it’s useful. And discuss with me.

[1] https://docs.docker.com/compose/
[2] https://prometheus.io
[3] https://github.com/stefanprodan/dockprom
[4] https://goo.gl/pCiDWc

More Reading:

• Get Slack Notifications For New Active Sprint
• Avoid Unnecessary Communication Of TOI
• Demo Jenkins: How I Enforce CI/CD

Blog URL: https://www.dennyzhang.com/docker_monitor

---