Avoid Using Same SSH Private Key For All Your Servers

The more projects you handle, the more servers you manage. But when you ssh to servers of different projects, are you using the same private key?
And how secured you feel about this? Let’s imagine. One day, your powerful private key gets compromised somehow. Boom! All your servers, and all your projects are in danger.

Check out this post. And get improved for all your projects, in just five minutes!

Avoid Using Same SSH Private Key For All Your Servers

Update Per Audience Feedback:

Thanks to Pankaj Agarwal. He has raised a good question: If I use multiple ssh key files and one gets compromised, the rest of key files won’t be secured neither.

Well, most likely. But not necessarily.

  1. Some people might upload their ssh key files into jump box. This is certainly not a good practice, but it happens. If we use different keys, it helps.
  2. We can add passphrase protection for ssh key files. What’s better, use different passphrases for different key files. Thus even if hackers take them all, we might still survive.

There’re many reasons to avoid using one ssh key for all. We need to reduce the attack surface.

Usually I use one ssh key for one project. Meanwhile I don’t want to keep switching ssh key, when I work across projects. Here is what I usually do.
Step1: Generate Different SSH Key Pairs For Different Projects.

Using ssh-keygen, we can easily generate as many ssh key pairs as we need.

Let’s say we already have two key pairs for two projects: project1idrsa and project2idrsa.
Step2: Use Different Private Keys Selectively. But In An Easy Way!

Version 1.0: Manually specify a private key, when ssh to different servers.

# ssh to server in project1
ssh -i project1_id_rsa user1@server1

# ssh to server in project2
ssh -i project2_id_rsa user2@server2

It works. But…

Typing those extra characters thousands of time, it’s not fun. And pointless.

Version 2.0: Create alias in ~/.ssh/config, then ssh with alias.

# Server in project1
host server1
     StrictHostKeyChecking no
     Port 22
     User user1
     IdentityFile /data/project1_id_rsa

# Server in project2
host server2
     StrictHostKeyChecking no
     Port 22
     User user2
     IdentityFile /data/project2_id_rsa

SSH with alias is quite easy and straight-forward.

# ssh to server in project1
ssh server1

# ssh to server in project2
ssh server2

So are we good now? Hang on, my friend. Not yet.

Let’s say you have tens of, or hundreds of servers. You don’t want to configure them one by one. Right?

Version 3.0: Update ~/.ssh/config to load all ssh private keys. Simply put something like below to the top of your ~/.ssh/config.

# Load private key of project1
IdentityFile /data//project1_id_rsa

# Load private key of project2
IdentityFile /data/project2_id_rsa

Now you can ssh like normal: “ssh user1@server1”.
SSH will try with all your private keys one by one. To confirm this, ssh with -vvv option.

# $ ssh -vvv user1@server1 date 2>&1 | grep "debug1: Offering RSA public key"
#   debug1: Offering RSA public key: /data/.ssh/project1_id_rsa
#   debug1: Offering RSA public key: /data/.ssh/project2_id_rsa

You can argue it will waste some time for the retry. Yes, it does. But it’s fast enough, before we can even notice the difference.

And ssh try the keys from top to bottom. We can put frequently used keys to the top. It will speed up a little bit.
Step3: [Optional] Secure Your SSH Private Key With Passphrase.

To make it better, add passphrase protection for your ssh private keys. Check this: Manage SSH Key File With Passphrase.

So now, why don’t you have a try for the tips shared in this post?

Please leave me comments, if you have any questions or feedbacks.

And don’t forget to share this post, if you find it might be useful for your friends or colleagues.

Related Reading:

Image Credit: blakesmith.me


PRs Welcome

Blog URL: https://www.dennyzhang.com/ssh_key_projects

4 Responses to Avoid Using Same SSH Private Key For All Your Servers

  1. what are the ways you think of sacrificing your private key file ? if one is sacrificed then the rest are too. i think passphrase protecting will be a much safer option other then using a configuration management tool to manage user keys which then allows you an option to instantly change the same on all nodes in no time.

    • Yes, Pankaj. Definitely passphrase protection will help.

      The main motivation of this post is to remind people don’t use one ssh key for all projects. If ssh key per project, it would lower the attack surface. Do you think so?

      To achieve that, we don’t want to make it hard to use. So there is a trick for this.

      About your original question. If one key is compromised, would other keys be unsafe as well. Most likely. But not necessarily.

      People may upload their ssh key into jump box. This is not a good practice, but it happens. Right?

  2. Enabling HashKnownHosts in the ssh config file is a good idea as well. This prevents an attacker (who

    presumably just got your private keys by gaining access to your account) from getting a lengthy list of all the machines that accept those keys.

    It’s the difference between seeing something like this: ecdsa-sha2-nistp256 (hostkey)

    and like this:

    |1|Nk3TYKyx/Q0oPwaioNCBO28ISQY=|/koKGn+mddMWak6WqrH5qPacrLA= ecdsa-sha2-nistp256 (hostkey)

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.