Locking Down SSH: Keys, Configs, and Fail2Ban

What you need: a Linux server you can reach over SSH with a sudo user, a local machine with the OpenSSH client (ssh and ssh-keygen), and a way back in if SSH stops working, such as the provider's web console.

Versions: written for OpenSSH as shipped with Ubuntu 24.04 and 26.04 LTS, Debian 12 and 13, and RHEL-family 9 and 10. Directive names and defaults are from the OpenBSD manual pages for sshd_config(5) and ssh_config(5); ssh -V prints your version and man sshd_config on the server documents exactly what it supports. The SSH service is called ssh on Debian and Ubuntu and sshd on Fedora and RHEL; Ubuntu 26.04 (OpenSSH 10.2) also accepts sshd as an alias, according to its release notes.

The Hardened Config

Put these settings in a drop-in file, /etc/ssh/sshd_config.d/10-hardening.conf, rather than editing the main file. Debian, Ubuntu and recent RHEL-family releases include /etc/ssh/sshd_config.d/*.conf at the top of /etc/ssh/sshd_config, and for most directives sshd uses the first value it reads. A low-numbered file therefore wins over both the main file and later drop-ins (for example a cloud image's own file that re-enables passwords). If your system has no such Include line, edit /etc/ssh/sshd_config instead, after backing it up with sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak.

Port 47892
PermitRootLogin no
PasswordAuthentication no
PermitEmptyPasswords no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
UsePAM yes
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
MaxAuthTries 3
LoginGraceTime 20
ClientAliveInterval 300
ClientAliveCountMax 2

Don't apply it yet. Do the key setup below first, then follow Apply the Config Safely: check the file with sudo sshd -t, open the new port in your firewall, restart, and test from a second terminal. Changing the port on Ubuntu also needs the socket activation steps.

The rest of this page explains what each directive does and walks through key setup if you haven't done that yet.

What Each Line Does

One line at a time:

• Port 47892 — Moves SSH off port 22. Bulk scanners that only probe port 22 stop reaching it, so your logs get much quieter. Not real security: a targeted port scan still finds it.
• PermitRootLogin no — Nobody logs in as root over SSH. Use a normal user and sudo.
• PasswordAuthentication no — Keys only. Password-guessing attacks stop working once this applies.
• PermitEmptyPasswords no — Already the default; stating it guards against an included file that changed it.
• KbdInteractiveAuthentication no — Turns off keyboard-interactive authentication, the route by which PAM can still ask for a password even with PasswordAuthentication no. Older guides call it ChallengeResponseAuthentication, which the manual now lists as a deprecated alias.
• PubkeyAuthentication yes — Enables key-based login. On by default, but be explicit.
• AuthenticationMethods publickey — Only allow public key. No keyboard-interactive, no GSSAPI, nothing else.
• UsePAM yes — Keep PAM on for account and session management (limits, environment, login records); with the two lines above it no longer asks for passwords.
• X11Forwarding no — Unless you're forwarding GUI apps (you're probably not), turn it off.
• AllowTcpForwarding no — Prevents your server from being used as a tunnel. This also blocks ssh -L dashboard tunnels used in the monitoring and Nextcloud guides, as well as ProxyJump. If an operations account needs local tunnels, use a deliberate per-user exception with AllowTcpForwarding local and PermitOpen 127.0.0.1:3000 127.0.0.1:9090 127.0.0.1:9093 127.0.0.1:8080, allowing only the destinations that account needs. Validate with sshd -t and inspect the effective configuration for that user with sshd -T -C user=operations,host=localhost,addr=127.0.0.1 before reloading. A bastion needs its own restricted forwarding policy (see the 3-tier VPC guide for that setup).
• AllowAgentForwarding no — Same idea. A compromised server can't use a forwarded agent to log in elsewhere as you.
• MaxAuthTries 3 — Three failed attempts per connection, then disconnect (default is 6). Slows down brute-force scripts. If you have many keys loaded in your agent, select the right one with IdentityFile and IdentitiesOnly yes on the client, or you may hit the limit before the right key is tried.
• LoginGraceTime 20 — 20 seconds to authenticate or get dropped. The default is 120.
• ClientAliveInterval 300 — Ping idle clients every 5 minutes.
• ClientAliveCountMax 2 — Two unanswered pings and the connection is closed. Cleans up dead sessions.

Back up first. Before you create the file, save copies of what you might need to restore: sudo cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config.bak, and on your own computer keep a copy of your key pair somewhere safe, because a key-only server can't be reached without it.

Generating an SSH Key

Run this on your local machine, not the server:

# Modern systems - use Ed25519
ssh-keygen -t ed25519 -C "you@laptop"

# Older systems that don't support Ed25519
ssh-keygen -t rsa -b 4096 -C "you@laptop"

Set a passphrase when prompted. If your laptop gets stolen, the passphrase is what stands between the thief and your servers. ssh-agent (or your OS keychain) remembers it for the session so you don't type it on every login; a password manager is a good place to store it.

You get two files:

• ~/.ssh/id_ed25519 — Your private key. Never share this. Never copy it to servers.
• ~/.ssh/id_ed25519.pub — Your public key. This goes on servers you want to access.

Copying Your Key to the Server

Easiest method (while password auth is still enabled):

ssh-copy-id username@yourserver.com

Appends your public key to ~/.ssh/authorized_keys on the server.

Manual method if ssh-copy-id isn't available:

# Display your public key
cat ~/.ssh/id_ed25519.pub

# Copy the output, then on the server:
mkdir -p ~/.ssh
echo "paste-your-public-key-here" >> ~/.ssh/authorized_keys
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

With the default StrictModes yes, sshd ignores authorized_keys if the file or its directories can be modified by other users, so the chmod commands aren't optional. Understanding Linux Permissions explains what these modes mean. On EC2, the key you chose at launch is already in authorized_keys for the default user (see Launch Your First EC2 Instance).

Test Before You Disable Passwords

Do not skip this. Open a new terminal and confirm key-based login works:

ssh username@yourserver.com

If it asks for your key passphrase (or nothing, if your agent already has the key), you're good. If it asks for the server password, the key isn't being accepted. Fix that before touching sshd_config; ssh -v (see Troubleshooting) shows why.

Apply the Config Safely

If you're making changes incrementally instead of using the full file, these are the critical lines:

PasswordAuthentication no
PermitEmptyPasswords no
KbdInteractiveAuthentication no
PubkeyAuthentication yes

Then, in this order:

1. Keep your current session open for the whole procedure. Existing sessions survive an sshd restart, so it's your way back in if the new config locks you out.
2. Check the config: sudo sshd -t. No output means the syntax is valid; otherwise it names the bad line. sshd won't start with an invalid config, so never restart without this.
3. If you changed the port, open it in the host firewall and any cloud security group first (see Change the Port), and on Ubuntu follow the socket activation steps.
4. Restart:

sudo sshd -t && sudo systemctl restart ssh     # Debian/Ubuntu
sudo sshd -t && sudo systemctl restart sshd    # Fedora/RHEL

5. Test from a second terminal with a brand-new connection. Only when that works, close the first session.

Disable Root Login

"root" is the first username automated attacks try. Turn it off. Make sure you have a sudo-capable user first:

adduser alice
usermod -aG sudo alice      # Debian/Ubuntu
# usermod -aG wheel alice   # Fedora/RHEL

Copy your public key to that user, then confirm sudo -v works in a session logged in as them. PermitRootLogin no is already in the config above.

Change the Port

People will tell you this is "security through obscurity" and therefore useless. They're half right: a port scan of your address finds it quickly. But much of the bulk password spraying across the IPv4 range only checks port 22, so moving off it cuts most of the log noise, and that makes real problems easier to spot.

The config above uses 47892. Pick any unused high port. Avoid well-known alternatives such as 2222, which scanners commonly try as well.

Open the port in your firewall before restarting sshd or you'll lock yourself out:

# For ufw
sudo ufw allow 47892/tcp

# For firewalld
sudo firewall-cmd --permanent --add-port=47892/tcp
sudo firewall-cmd --reload

# Fedora/RHEL with SELinux enforcing: allow sshd to bind the new port
sudo semanage port -a -t ssh_port_t -p tcp 47892

On a cloud server, also add the port to the security group or provider firewall, as shown in Launch Your First EC2 Instance. Remove the old port 22 rules only after you've logged in on the new port.

Ubuntu: Socket Activation

Since Ubuntu 22.10, OpenSSH is socket-activated by default: systemd listens on the port through ssh.socket and only starts sshd when a connection arrives, as described in Ubuntu's announcement sshd now uses socket-based activation. That means restarting ssh.service alone doesn't move the listening port:

• Ubuntu 24.04 LTS and later: a systemd generator reads Port and ListenAddress from your sshd configuration. After changing them, run sudo systemctl daemon-reload so the generator runs again, then sudo systemctl restart ssh.socket.
• Ubuntu 22.10 to 23.10 upgraded systems: existing Port/ListenAddress settings were migrated to /etc/systemd/system/ssh.socket.d/addresses.conf; that file controls the listening port there.
• Ubuntu 22.04 LTS and Debian: not socket-activated by default; restarting ssh is enough.

sudo sshd -t
sudo systemctl daemon-reload
sudo systemctl restart ssh.socket
sudo ss -tlnp | grep 47892     # confirm systemd is listening on the new port

Then connect with the new port:

ssh -p 47892 username@yourserver.com

Save yourself from typing -p 47892 every time — add it to your local SSH config:

Host myserver
 HostName yourserver.com
 User alice
 Port 47892
 IdentityFile ~/.ssh/id_ed25519
 IdentitiesOnly yes

Now ssh myserver does the right thing. All client options are listed in ssh_config(5).

Fail2Ban

Fail2Ban watches logs for repeated failures and temporarily blocks the source address in the firewall. With key-only authentication, those password guesses can't succeed anyway, so Fail2Ban is a second line rather than the main defence: it cuts log noise and slows down anything that keeps hammering the port. It's worth adding, as long as you don't treat it as a replacement for the settings above.

sudo apt update && sudo apt install fail2ban

# Rocky Linux / AlmaLinux 10: enable CRB, then EPEL (Fedora skips both)
sudo dnf config-manager --set-enabled crb
sudo dnf install epel-release

sudo dnf install fail2ban
sudo systemctl enable --now fail2ban

On RHEL itself the EPEL release package isn't in Red Hat's repositories. The EPEL getting started page gives the commands per release: enable the CodeReady Builder repository with subscription-manager, then install epel-release-latest-10.noarch.rpm (or -9) from dl.fedoraproject.org. Debian and Ubuntu start the service as soon as the package is installed.

Customize it by creating a local config (the shipped jail.conf gets overwritten on package updates):

[DEFAULT]
# Ban for 1 hour
bantime = 3600

# An IP gets banned if it fails 5 times...
maxretry = 5

# ...within a 10-minute window
findtime = 600

# Never ban these: localhost, plus your own static IP if you have one
# (203.0.113.10 is a documentation address; replace it or delete it)
ignoreip = 127.0.0.1/8 ::1 203.0.113.10

[sshd]
enabled = true
port = ssh,47892
filter = sshd
# Read sshd's messages from the systemd journal
backend = systemd
maxretry = 3

Journal or log file? Debian 12 and later no longer install rsyslog by default (see the Debian 12 release notes), so there is no /var/log/auth.log and SSH messages only go to the systemd journal. A jail pointed at logpath = /var/log/auth.log then fails to start with "Have not found any log file for sshd jail", as reported in the fail2ban issue tracker; backend = systemd is the fix. If your system does write /var/log/auth.log (Ubuntu installs rsyslog, for example), either setting works; to read the file instead, use backend = auto with logpath = /var/log/auth.log.

• bantime = 3600 — Block for 1 hour. Permanent bans grow the firewall rule set without much benefit, because the sources of automated attacks change constantly.
• maxretry = 3 — Three failures for SSH. With key authentication, legitimate users rarely fail at all.
• findtime = 600 — Counting window. 3 failures in 10 minutes = banned.
• ignoreip — Your home/office IP goes here, if it's static, so you can't ban yourself.
• port — Include your custom SSH port if you changed it, or the ban won't cover it.

Restart Fail2Ban:

sudo systemctl restart fail2ban

Checking Fail2Ban Status

# Check status of SSH jail (failure counts and the banned IP list)
sudo fail2ban-client status sshd

# Unban an address (if you banned yourself)
sudo fail2ban-client set sshd unbanip 203.0.113.10

# Check that the sshd filter matches what is in the journal
sudo fail2ban-regex systemd-journal sshd

status sshd shows the number of currently failed and banned addresses and the list of banned IPs. If the counters stay at zero while the journal shows failed logins, the jail is reading the wrong source: the fail2ban-regex line tells you whether the filter matches the journal at all.

Extra Hardening

Worth considering on servers that matter more than average:

Limit Who Can SSH

Whitelist who can SSH in:

AllowUsers alice bob

Logins by any other user are refused, even with a valid key. AllowGroups does the same by group, which is easier to maintain.

Use Two-Factor Authentication

A FIDO2 security key gives you two factors with plain OpenSSH: generate the key with ssh-keygen -t ed25519-sk, and every login needs the physical key as well as the key file. TOTP codes are also possible through a PAM module, with AuthenticationMethods publickey,keyboard-interactive, but that means re-enabling keyboard-interactive authentication, so set it up carefully and test it from a second session.

Set Up Login Notifications

A simple script that emails on every login:

# Run by sshd with /bin/sh, so keep it to plain sh syntax
IP=$(echo "$SSH_CONNECTION" | awk '{print $1}')
echo "SSH login: $USER from $IP on $(hostname) at $(date)" | mail -s "SSH login on $(hostname)" admin@example.com

Replace admin@example.com with your own address. sshd runs /etc/ssh/sshrc with /bin/sh at login, as the user who logged in, unless that user has their own ~/.ssh/rc (see sshd(8)). It needs a mail command and a working mail setup on the server. Treat it as a convenience: someone who gets in as that user can edit or bypass their own rc file, so it's no substitute for central logging.

Verify It Worked

sshd -T (extended test mode) checks the configuration and prints the effective settings, one lowercase keyword value line each, after all included files have been read:

sudo sshd -T | grep -E '^(port|permitrootlogin|passwordauthentication|kbdinteractiveauthentication) '

With the config above, you're looking for port 47892 and no for the other three keywords. If a value is different, another file set it first: with the "first value wins" rule, an earlier drop-in or a line above the Include takes precedence. Match blocks only show up when you describe a connection with -C, for example sudo sshd -T -C user=alice,host=laptop,addr=203.0.113.10; that's how you check per-user or per-address overrides.

Troubleshooting

• See what the client is doing: ssh -v myserver (add more vs for more detail). Look for the lines that say which keys it offers, whether the server accepts or rejects each one, and which authentication methods the server says can continue. If your key is never offered, the client is using a different file (check IdentityFile); if it's offered and rejected, the problem is on the server side.
• Server side: sudo journalctl -u ssh -f (or -u sshd) while you try to log in. Messages such as "Authentication refused: bad ownership or modes" point straight at ~/.ssh permissions.
• "Connection refused" after changing the port: nothing listens on the new port. On Ubuntu 24.04, you probably restarted ssh.service but not ssh.socket (see above). sudo ss -tlnp shows the real listening ports. On RHEL, check the SELinux port label.
• Connection times out: a firewall or security group drops the packets. Check ufw/firewalld and the cloud firewall for the new port.
• sshd won't start: sudo sshd -t prints the error; systemctl status ssh and the journal show it too. Fix it from the session you kept open, or from the provider's console.
• Locked out completely: use the provider's web or serial console (or physical access), restore the backup or remove the drop-in, run sshd -t, and restart.
• Fail2Ban jail won't start: sudo fail2ban-client -d dumps the parsed configuration and shows errors; on Debian 12+, a logpath pointing at a missing /var/log/auth.log is the usual cause.
• "WARNING: connection is not using a post-quantum key exchange algorithm": recent OpenSSH clients, including the one in Ubuntu 26.04, print this when the server is too old to offer a post-quantum key exchange. It's a warning, not a failure; the login still works. Updating OpenSSH on the server makes it go away.

Undoing the Changes

To go back to the distribution defaults, or to clean up a test server, reverse the steps in this order, from a session that is already logged in:

# Remove the drop-in (and restore the main file if you edited it)
sudo rm /etc/ssh/sshd_config.d/10-hardening.conf
# sudo cp -a /etc/ssh/sshd_config.bak /etc/ssh/sshd_config

# Check, then restart (Ubuntu 24.04+: daemon-reload and restart ssh.socket too)
sudo sshd -t && sudo systemctl restart ssh

# Remove the firewall rule for the custom port once port 22 works again
sudo ufw delete allow 47892/tcp
# firewalld: sudo firewall-cmd --permanent --remove-port=47892/tcp && sudo firewall-cmd --reload
# SELinux:   sudo semanage port -d -t ssh_port_t -p tcp 47892

# Remove Fail2Ban and its local config if you no longer want it
sudo apt purge fail2ban       # or: sudo dnf remove fail2ban
sudo rm -f /etc/fail2ban/jail.local /etc/ssh/sshrc

Log in again on port 22 from a second terminal before closing the first. On your own computer, remove the Port 47892 line from ~/.ssh/config, and remove any keys you generated only for testing from the server's ~/.ssh/authorized_keys. Close the custom port in the cloud security group as well.

Next Steps

• Understanding Linux Permissions: why sshd is strict about the modes on ~/.ssh.
• Running Your Scripts as Real Services with Systemd: more on units, sockets and the journal that sshd and Fail2Ban rely on.
• Tailscale Mesh VPN: reach servers over a private network so the SSH port doesn't need to face the internet at all.
• AWS VPC 3-Tier Architecture: a bastion host setup, where TCP forwarding has to stay enabled.