Home Server How to add SSH public key to server in Linux

How to add SSH public key to server in Linux

Adding an SSH public key to a server is a crucial step for secure and convenient remote access in Linux. This guide explains how to generate an SSH key pair, copy the public key to your server, and configure SSH for key-based authentication, streamlining your remote login process.

by Arun Kumar
public key addition to linux servers

Welcome to the dynamic world of Linux server management! Today, we’re zeroing in on a critical skill that stands at the crossroads of security and accessibility – adding an SSH public key to a Linux server. As we delve into this essential task, we’ll unveil both the automatic and manual methods, using Ubuntu as our guiding example.

This blog is crafted for everyone, from curious beginners to seasoned sysadmins, ensuring that secure server access is not just a necessity but also an achievable goal for all.

Understanding SSH keys

Before we start, a bit of context. SSH keys are a pair of cryptographic keys that can be used to authenticate to an SSH server as an alternative to password-based logins. A public key, which you can share with anyone, and a private key, which you should guard like your most embarrassing secret.

Automatic vs manual way: Choose your adventure

You can add your SSH public key to a server in two main ways: the automatic way, using a simple command, and the manual way, which involves a bit more tinkering. I personally enjoy the manual method – it feels like I’m a locksmith, working my magic. But let’s explore both!

Automatic way: Using ssh-copy-id

  1. Generate your SSH key pair (if you haven’t already)
    Open your terminal and type:

    Follow the prompts, and remember, it’s a good idea to secure your key with a passphrase.

    Example Output:

    Generating public/private rsa key pair.
    Enter file in which to save the key (/your/home/.ssh/id_rsa):
  2. Copy your SSH public key to the server
    Use ssh-copy-id for this. It’s like mailing your key to the server – super simple!
    ssh-copy-id username@server-address

    Replace username with your actual username and server-address with the server’s IP address or hostname.

    Example Output:

    Number of keys added: 1
    Now try logging into the machine.

    And that’s it! Your key should now be added to the server’s authorized keys list.

Manual way: The good old copy-paste

If you’re like me and enjoy the hands-on approach, here’s how to do it manually.

  1. Locate your SSH public key
    Typically, your public key will be in ~/.ssh/id_rsa.pub. Use cat to display it.
    cat ~/.ssh/id_rsa.pub

    Copy the output. This is your public key.

  2. Log into your server
    Use SSH to connect to your server:
    ssh username@server-address
  3. Edit the authorized_keys file
    Navigate to ~/.ssh/ on your server. If the directory doesn’t exist, create it with mkdir ~/.ssh.Open (or create) the authorized_keys file with your favorite editor. I have a soft spot for nano:
    nano ~/.ssh/authorized_keys

Paste your public key at the end of the file. Save and close the file. In nano, it’s CTRL+O to write and CTRL+X to exit.

Example Output:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3... user@local-machine

This line is your public key, now safely residing in your server’s authorized_keys.

Setting the right permissions

Whether you chose the automatic or manual path, ensuring the correct permissions is crucial for security. Here’s what you need to do:

  1. Set permissions for .ssh and authorized_keys
    On the server, run:
    chmod 700 ~/.ssh
    chmod 600 ~/.ssh/authorized_keys

    This makes sure that only you can read, write, and execute in your .ssh directory, and only read and write the authorized_keys file.

Testing your setup

Now for the moment of truth. Disconnect from your server:


And then try logging in again with SSH:

ssh username@server-address

If everything went smoothly, you should be logged in without needing to enter a password. Magic, isn’t it?

Why I prefer SSH keys over passwords

Personal anecdote time! I’ve always favored SSH keys over passwords for a few reasons. First, they’re far more secure.

The chances of someone brute-forcing your SSH key are astronomically lower than guessing a password. Second, as someone who juggles multiple servers, SSH keys save me the hassle of remembering different passwords or, worse, using the same password everywhere (a big no-no in security!).

Troubleshooting common issues

Even with a guide, things can sometimes go awry. Here are a few common issues you might encounter and how to resolve them:

Permission denied error

If you see a Permission denied (publickey) error, double-check the permissions for your .ssh directory and the authorized_keys file on the server. They should be set to 700 and 600 respectively.

Key not recognized

Make sure you copied the entire key correctly. The public key is generally quite long, and missing even a single character can cause issues.

Connection timed out

If you can’t connect to your server at all, it might be a network issue, or SSH might not be installed or running on your server. Verify your network connection and check your server’s SSH setup.

FAQ: Frequently Asked Questions about adding SSH public keys to Linux servers

What is an SSH key, and why should I use it instead of a password?

Answer: An SSH key is a cryptographic key used for secure access to a server using SSH. Unlike passwords, SSH keys are almost impossible to decipher through brute force attacks, making them a much safer method for authentication.

Can I use the same SSH key for multiple servers?

Answer: Absolutely! You can use the same public key to connect to multiple servers. This makes managing your keys simpler, especially if you regularly connect to many servers.

What if I lose my private key?

Answer: If you lose your private key, you cannot access the server using that key pair. You’ll need to generate a new SSH key pair and add the new public key to the server. Always keep your private key secure and backed up!

How do I change the passphrase of my SSH key?

Answer: To change the passphrase, use the command ssh-keygen -p. It’ll prompt you for the old passphrase and then ask for a new one. Remember, a strong passphrase adds an extra layer of security.

Can I add multiple SSH keys to a single server?

Answer: Yes, you can. Each key needs to be added to the authorized_keys file on the server. This is useful if multiple users need access to the server, each with their own key.

Is it safe to share my public key?

Answer: Sharing your public key is safe. It’s designed to be distributed and cannot be used to derive the private key. However, your private key should never be shared.

What should I do if my SSH key is compromised?

Answer: If you believe your SSH key is compromised, generate a new key pair immediately and replace the old public key on any servers or services where it was used.

Do SSH keys expire?

Answer: By default, SSH keys do not expire. However, it’s a good security practice to regularly update your keys.

Can I use SSH keys for things other than server authentication?

Answer: Yes! SSH keys can be used for other purposes, like signing Git commits, authenticating to various services that support SSH-based authentication, and more.

How do I know if my server is set up to accept SSH key authentication?

Answer: Check the SSH configuration file (/etc/ssh/sshd_config) on your server. Look for the PubkeyAuthentication directive. It should be set to yes to allow SSH key authentication.


And there we have it – a comprehensive walkthrough of adding an SSH public key to a Linux server. From the simplicity of the ssh-copy-id command to the hands-on approach of manual key addition, we’ve covered the spectrum, ensuring your entry into the realm of secure Linux administration is both smooth and informed. Your questions and experiences are always welcome here, where learning and sharing go hand in hand.

You may also like

Leave a Comment



FOSS Linux is a leading resource for Linux enthusiasts and professionals alike. With a focus on providing the best Linux tutorials, open-source apps, news, and reviews written by team of expert authors. FOSS Linux is the go-to source for all things Linux.

Whether you’re a beginner or an experienced user, FOSS Linux has something for everyone.

Follow Us


©2016-2023 FOSS LINUX



“Linux” is the registered trademark by Linus Torvalds in the U.S. and other countries.