Set Up and Use an SSH Jump Server

Using SSH (Secure Shell) to connect to remote hosts or servers is a very common practice today (and has been for a long time). What this means is basically allowing you to interact with a remote computer via a command line interface. While SSH enables use-cases like SSHFS and SCP, operating a command line is its main purpose. The host you would like to connect to may not be directly reachable from where you connect from, though. Sensitive servers or residential computers may be hidden behind NAT, firewalls, or gateway servers. You would have to connect to a different computer first, before you can “jump” to your final destination host. SSH specifically suppors this through a feature called Jump Servers. In this article I will show you how you can easily set up and use an SSH Jump Server!

What is an SSH Jump Server?

An SSH jump server (or bastion host) acts as an intermediary between your local machine and your target servers. By funneling your SSH connections through this jump server, you add an extra layer of security, minimizing the direct exposure of your servers to the internet.

The jump server acts as an isolation between the origin and the destination networks. These jump servers often reside in the so-called DMZ (Demilitarized Zone) where servers are reachable from the internet, but don’t hold actual sensitive information.

Jump servers serve two purposes really:

• Protecting your sensitive server infrastructure: By not allowing direct access to your sensitive networks, you protect other resources in the same network (or even other ports on the same server). If the actual server were accessible directly from the cloud, simple configuration errors could violate its entire security model.
  
• Obscuring your own origin IP: The destination host “sees” the IP addresses of hosts connecting to it directly. This means that, when using a jump server, the destination host sees the jump server’s IP, not the origin IP. This is a great way to ensure online IP privacy by hiding behind a (potentially anonymous) jump server for added privacy. Anonymous VPS services like OrangeWebsite are perfect for this.

In the following, I will show you how to set up an SSH Jump Server, and how to use it.

Setting up an SSH Jump Server

First, you need a machine that will act as your jump server. This machine should be secured and have SSH access enabled. You can use any Linux-based system for this purpose. If you don’t have a publicly accessible network infrastructure to host this in yourself, you can also use any public VPS hoster. Some common and cheap ones are mentioned below:

• DigitalOcean (starts at ~$5 per month)
• Scaleway (starts at ~$4 per month)
• OrangeWebsite (anonymous, ~$30 per month)
• IONOS VPS (starts at ~$1 per month)

I will assume a Ubuntu or Debian-based setup in the following steps. Should you choose any other operating system/distribution, please adjust the below advice accordingly.

Configuring SSH on the Jump Server

Since I’m assuming you’re using a remote instance, SSH will likely already be installed and enabled. I will cover installation of SSH here anyway for the sake of completeness. First, install the SSH server package:

sudo apt-get update
sudo apt-get install openssh-server

Once this is installed, edit the SSHd config file:

sudo nano /etc/ssh/sshd_config

Here, you will need to ensure these settings are set:

PermitRootLogin no
PasswordAuthentication no
AllowTcpForwarding yes

These settings have a few implications that you should be aware of:

PermitRootLogin no

• Explanation: This setting disallows direct SSH access to the root user.
• Purpose: Disabling root login via SSH increases security by requiring users to log in as a non-root user and then escalate privileges if necessary. This minimizes the risk of brute-force attacks targeting the root account.
• Implication: You can still SSH into the host, but you need to use a regular user account. Once logged in, you can switch to the root user if necessary using sudo or su.

PasswordAuthentication no

• Explanation: This setting disables password-based authentication for SSH.
• Purpose: Disabling password authentication forces users to use SSH keys for authentication, which is more secure. SSH keys are much harder to brute-force than passwords.
• Implication: You can still SSH into the host, but you must use an SSH key pair. This means you need to generate an SSH key on your local machine and copy the public key to the authorized keys on the server. With PasswordAuthentication set to no, even if someone knows your username, they can’t log in without the corresponding private key. Find out how to generate your own SSH key pair here!

AllowTcpForwarding yes

• Explanation: This setting allows TCP forwarding, which is necessary for features like SSH tunneling and proxying.
• Purpose: Enabling TCP forwarding allows the jump server to forward connections to the target servers. This is essential for an SSH jump server, which needs to relay connections from your local machine to the final destination.
• Implication: This setting allows SSH to create tunnels that forward network traffic, enabling the jump server to act as an intermediary. This is critical for the jump server to function correctly.

How These Settings Work Together

• PermitRootLogin no ensures that the root account is not directly accessible via SSH, forcing users to log in as non-root users.
• PasswordAuthentication no enhances security by requiring SSH key-based authentication, which mitigates the risk of password-based attacks.
• AllowTcpForwarding yes enables the jump server to forward traffic, facilitating the role of the jump server in relaying SSH connections.

These settings follow a best-practices approach for configuring your SSH server. You can customize them the way you want but should keep the AllowTcpForwarding yes setting intact for the jump server to work.

Set Up User Accounts

Create user accounts on the jump server that will be used for the SSH connections. For example, to create a user called jumpuser:

sudo adduser jumpuser

Secure the Jump Server

It’s crucial to secure your jump server. Here are some basic security steps:

• Use SSH Keys: Disable password authentication and use SSH keys for authentication.
  • Generate an SSH key pair on your local machine using ssh-keygen.
  • Copy the public key to the jump server using ssh-copy-id user@jump-server.
• Enable Firewall: Use a firewall to restrict access to the jump server.
  • On a Debian-based system, you can use ufw to enable and configure the firewall:

sudo ufw allow OpenSSH
sudo ufw enable

• Keep the System Updated: Regularly update the jump server to ensure all security patches are applied:

sudo apt-get update && sudo apt-get upgrade

Test the Jump Server

Test the setup by connecting to the jump server from your local machine:

ssh jumpuser@jump-server

SSH Key Access

With PasswordAuthentication no (configured above), you need to set up SSH key authentication. Here’s how to do it:

Generate an SSH Key Pair on your local machine (if you don’t already have one):

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

This command creates a public/private key pair in the ~/.ssh directory.

Copy the Public Key to the Jump Server:

ssh-copy-id jumpuser@jump-server

This command copies your public key to the ~/.ssh/authorized_keys file on the jump server, allowing you to authenticate using your private key.

Connect Using Your SSH Key:

ssh jumpuser@jump-server

By using SSH keys, you enhance security while still being able to SSH into the host. This method ensures that only users with the correct private key can access the server, reducing the risk of unauthorized access.

Should you now be able to connect to your jump server using the above command, without entering your user password, then you’re all set up for the next step: Actually using your jump server.

Connecting to a Destination Server Using the Newly Configured Jump Server

Now that your jump server is set up and secured, you can use it to connect to your final destination server. This involves configuring your SSH client to route through the jump server. There are multiple ways to achieve the jump, which I’ll desribe below.

Direct Connection Using ssh -J

You can connect to your destination server through the jump server directly from the command line using the -J option (for ProxyJump). Here’s how:

ssh -J jumpuser@jump-server targetuser@target-server

• jumpuser@jump-server: The user and host for the jump server.
• targetuser@target-server: The user and host for the destination server.

This command establishes an SSH session to the target server by first connecting to the jump server and then forwarding the connection.

Configuring SSH for Easier Access

To make connecting easier, you can configure your SSH client by editing the ~/.ssh/config file. This way, you can set up aliases and avoid typing the full command each time.

Open (or create) the ~/.ssh/config file:

nano ~/.ssh/config

Add the following configuration:

Host jump-server
  HostName your.jump.server.ip
  User jumpuser
  IdentityFile ~/.ssh/id_rsa

Host target-server
  HostName your.target.server.ip
  User targetuser
  ProxyJump jumpuser@jump-server

Replace your.jump.server.ip with the IP address or hostname of your jump server, and your.target.server.ip with the IP address or hostname of your destination server. Also, replace jumpuser and targetuser with the appropriate usernames for the jump server and destination server, respectively.

With the SSH configuration file set up, you can now connect to your destination server using a simple alias. For example using this command:

ssh target-server

This command will automatically use the jump server as an intermediary, thanks to the configuration in ~/.ssh/config.

Verifying the Connection

To ensure everything is set up correctly, try connecting to your target server using the alias or direct command:

ssh target-server

If the connection is successful, you should see the prompt for your destination server. If you encounter any issues, double-check the following items:

• Ensure your SSH keys are correctly set up and copied to both the jump server and the destination server.
• Verify the IP addresses and usernames in your SSH configuration file.
• Make sure the jump server’s SSH service is running and accessible from your local machine.

Conclusion

Using an SSH jump server provides an added layer of security and flexibility for accessing remote servers. By configuring your jump server and setting up your SSH client properly, you can securely and efficiently manage your remote connections. The setup explained in this article is particularly useful for protecting sensitive server infrastructures and maintaining IP privacy.

To stay extra secure and private, be sure to check out my articles on securely using the net with a VPN and how to set up your own encrypted private network! For even more added security, consider using a hardware security dongle like a YubiKey!

If you liked this article, or want to share your own thoughts and experiences, comment below to get the conversation started!