Detect Suspicious Linux Processes

Usually kernel processes are safe and clean. For kernel processes, either PID(process id) is 2 or PPID(parent process id) is 2. Here is how to get all non-kernel processes.

# rss(resident set size): real RAM usage
# -deselect: rule out matched processes
root@denny:~# ps --ppid 2 -p 2 -p 1 \
   --deselect -o uid,pid,rss,%cpu,command
UID   PID   RSS %CPU COMMAND
   0   411  1848  0.0 /lib/systemd/systemd-
   0   572  2904  0.0 dhclient -1 -v -pf /r
 102   902  1244  0.0 dbus-daemon --system
   0   912  1948  0.0 /lib/systemd/systemd-
   0  5869   388  0.0 upstart-socket-bridge
 200  1953   904  0.0 /usr/sbin/apache2 -k
 200  3463  3700  0.0 /usr/sbin/apache2 -k
  ...  ...
  ...  ...
   0  5098  4224  0.0 sshd: ubuntu [priv]
   0  5139  1748  0.0 /usr/bin/python /usr/
 200  5140  3484  0.0 /usr/bin/python /usr/
 200  5176  1904  0.0 sshd: ubuntu@pts/3
 200  5177  3860  0.0 -bash
 200  5193  1200  0.0 tmux attach -t denny
   0  5297  4224  0.0 sshd: ubuntu [priv]
  ...  ...
  ...  ...

We may have many processes running, which are expected and trusted. e.g apache2, tomcat7, mysqld, etc. To avoid distraction, build a white list especial for your project.

We’re more concerned about suspicious processes using noticeable resource.

# Sort by memory first, then cpu
ps --ppid 2 -p 2 -p 1 --deselect \
  -o uid,pid,rss,%cpu,command, \
  --sort -rss,-cpu

Here comes a python script in Github (detect_{suspiciousprocess.py}).

You can use it to capture all unknown process.

1. Create a crontab or scheduled jenkins job to check the count of suspicous processes.
2. Raise alert, if the count is not 0 or it changes.

It might take a while to build a suitable white list. Once it’s done, your servers are always more secured and managed!

More Reading:

• Automate Insecure Ports Check By Nmap
• 9 Useful Tips For Linux Server Security
• docker-bench-security: Audit Well-Known Docker Vulnerabilities

Blog URL: https://www.dennyzhang.com/suspicious_process