Using an SSH tunnel with Bacula

1. Introduction

1.1 Copyright And License

This document, Using an SSH tunnel with Bacula is copyrighted © 2009 by Kevin Keane. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html .

1.2 Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies which could damage to your system. Though this is highly unlikely, proceed with caution. The author(s) do not accept responsibility for your actions.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements.

1.3 Credits / Contributors

Credit goes to Stephan Holl and Joshua Kugler, who wrote earlier documents and scripts that document the basic idea of an SSH tunnel.

1.4 Overview

This is a similar approach, but different in some subtle but important details. Most importantly, this approach allows you to use the same Storage daemon for both local clients and clients through an SSH tunnel. This is important because you can use the same pool and schedule for both.

I also changed the way SSH is invoked to make it self-closing. The tunnel will simply disappear as soon as the backup is completed.

Any questions? Please post to the bacula-users mailing list, or visit http://www.4nettech.com and use the Contact Us form.

2. What you need

On the bacula-director side:

  • bacula director running as user “bacula”, and fully configured and working using the standard ports 9101-9103. You can only have one Storage Daemon in your configuration. Make sure the Storage resource uses an FQDN; do not use IP addresses. Also make sure that the FQDN for the Storage does not resolve from the client machine!
  • openssh
  • for each client, an available port to listen on, in addition to bacula's standard 9101-9103. I chose 9112. You may want to add this port to /etc/services, see below for details.
  • the firewall must be open on localhost, ports 9101 and 9103.

On the bacula client side:

  • bacula-fd installed.
  • a user account with minimal permissions. Let's call it “tunneluser”. This account will only serve as end point for the SSH tunnel. For security reasons, DO NOT use the root account.
  • The firewall must be open on localhost, port 9101, 9102 and 9103. It is not necessary to open these ports on the network (eth0 port), just to the lo port
  • The firewall must be open on port 22 to allow inbound SSH connections

3. Setting up OpenSSH on the server side

First, we need to set up ssh for the bacula user, since this is the user name the director uses:

  • Log on as user “bacula”
 You may have to log on as root, and then use the "su - bacula" command to actually become user bacula. 
 Notice that on ubuntu system, login is disabled for bacula user. 
 To enable it, do 'usermod -s /bin/bash bacula' as root before trying to login.
 (the user is disabled for login if /bin/false is at the end of : cat /etc/passwd | grep bacula)
 Easier alternative: if "su - bacula" does not work, try "sudo -u bacula bash"

You should be in bacula's home directory - probably /var/lib/bacula.

mkdir -p ~/.ssh
chmod 700 ~/.ssh

Also verify that .ssh is owned by user bacula, group bacula.

Now we need to create an SSH key pair into this new directory. Make sure that you do not use a pass phrase. If you set a passphrase, this last will be asked upon connection, which we don't want.

We use the file names bacula_tunnel and bacula_tunnel.pub for the private and public keys, respectively. The private key file, bacula_tunnel, should remain secret, so avoid backup of this key (it's easily regenerated) or do secure backup of this key as it gives access to your client computer.

cd ~/.ssh
ssh-keygen -b 2048 -C "SSH Tunnel for bacula" -t rsa -f bacula_tunnel -N ""

Create a file “config” in the ~/.ssh directory with the following content (<yourclientFQDN> is the actual FQDN that SSH will use to connect to the client. yourclientname should simply be the same as <yourclientFQDN>, although in some cases you could choose a different value for it). Do not add the angled brackets.

Host <yourclientFQDN>
      AddressFamily           inet
      ConnectionAttempts      10
      ForwardAgent            no
      ForwardX11              no
      ForwardX11Trusted       no
      GatewayPorts            yes
      HostBasedAuthentication no
      HostKeyAlias            <yourclientname>
      HostName                <yourclientFQDN>
      IdentityFile            ~/.ssh/bacula_tunnel
      PasswordAuthentication  no
      Protocol                2
      Compression             yes
      CompressionLevel        9
      ServerAliveCountMax     3
      ServerAliveInterval     15
      TCPKeepAlive            no
      User                    tunneluser

Just in case permissions got messed up on the way, fix them in the .ssh directory

chmod 400 ~/.ssh/*

Transfer the public key (the file bacula_tunnel.pub) to your client machine, using whatever method you like (USB, FTP, scp, file shares, copy & paste from bacula_tunnel to authorized_keys also works …).

IMPORTANT: DO NOT TRANSFER THE PRIVATE KEY!

4. Setting up OpenSSH on the client side.

install the SSH public key into the tunneluser account on the client machine:

cd /home/tunneluser
mkdir -p /home/tunneluser/.ssh
touch /home/tunneluser/.ssh/authorized_keys
# Make sure you use the double >> redirection, or you will destroy any
# previous authorized keys if they existed.
cat <wherever>/bacula_tunnel.pub >>/home/tunneluser/.ssh/authorized_keys
chmod 700 /home/tunneluser/.ssh
chmod 400 /home/tunneluser/.ssh/*
chown -R tunneluser:tunneluser /home/tunneluser/.ssh

5. Complete OpenSSH setup

Before you can use OpenSSH, you need to complete one more step: store the host's fingerprint. On the server side, log on as user “bacula” (or use “su - bacula”). Then use ssh to connect to your client. Note that you MUST use the same FQDN you used in the “config” file earlier.

ssh <yourclientFQDN>

ssh will now prompt you to accept the fingerprint. Once you are at the command prompt, you can type “exit” to leave ssh.

6. Bacula setup, client side.

On the server, find the Address= lines for the Storage resource. Make sure the address is a FQDN on the private network, and do not resolve from the client. On the client, verify that the FQDN of the Storage daemon does not resolve:

nslookup <storageFQDN>

This should return an error. If it returns a valid IP address: STOP! Proceeding would cause errors in other services. Instead, create a new FQDN for the Storage daemon as an alias (DNS CNAME record) for the actual FQDN, and use that.

On the client, edit your /etc/hosts file by adding the following line:

127.0.0.1	<storageFQDN>

There should be a tab, not just spaces, between the IP address and the FQDN.

Note: what this does is allow the file daemon (FD) to find the storage daemon (SD). Normally, the SD is of course hidden behind a firewall and inaccessible - that is why you are setting up an SSH tunnel, after all. This entry tricks your client system into believing that the SD is located at 127.0.0.1 - which actually will be the end of the tunnel that we are going to establish soon.

FileDaemon {
  Name = <yourclientFQDN>-fd
  FDAddress = 127.0.0.1
  FDport = 9102                  # where we listen for the director
  WorkingDirectory = (use the standard values)
  Pid Directory = (use the standard values)
  Maximum Concurrent Jobs = (use the standard values)
}

7. Bacula setup, server side

Simply set up a client and job resource for the new client as usual, using the standard schedule and pool settings. Use <yourclientFQDN-fd> as the client name.

Now change the client resource as follows:

Name=<yourclientFQDN-fd
Address=localhost
FDPort=9112

In the job resource, add a Run Before Job script:

Run Before Job = "/usr/local/sbin/sshbacula <yourclientFQDN>"

Note : some people consider it a good practice to keep track of each port used on the machine by putting it /etc/services. It helps avoid accidentally reusing the same port, which can cause hard-to-track-down errors. However, other people feel that /etc/services should be left pristine.

Bacula regular port are already there :

bacula-dir  9101/tcp      # Bacula Director
bacula-dir  9101/udp
bacula-fd   9102/tcp      # Bacula File Daemon
bacula-fd   9102/udp
bacula-sd   9103/tcp      # Bacula Storage Daemon
bacula-sd   9103/udp

Look up the port number you are planning to use. If it is already assigned to a service that you are not using on this server, comment out the line in question.

Then add your port number at the end of the file :

# Local services
bacula_fd_tunnel1 9112/tcp #bacula ssh tunnel port for clientFQDN

If you should backup several computer at the same time, you'll either need a port per machine, or you must back up one machine at a time.

Another tool to track down the duplicate use of ports is netstat. Use the command:

netstat -ltunp | grep LISTEN

to get a listing of all the ports that are currently in use for listening. The p option will also report which program is using which port.

8. The SSH tunnel script

Use the following script and save it as /usr/local/sbin/sshbacula. Remember to change the permission as needed:

chmod +x /usr/local/sbin/sshbacula

Hint: you may want to change the line

CLIENT=$1

to

CLIENT=${1:-<yourclientFQDN>}

This will provide a default value to connect to

#!/bin/bash
# Establishes a self-killing SSH tunnel to the
# given SSH server, and forwards the correct
# ports for bacula usage.

# Bacula-dir runs as user bacula, but with root's
# environment. We need to trick it into running
# actually looking for the .ssh directory in
# the right place.
USER=bacula
HOME=$(grep "^$USER:" /etc/passwd | cut -d : -f 6)
CLIENT=$1
LOCAL=$(hostname -f)
SSH=/usr/bin/ssh

echo "Starting SSH-tunnel to $CLIENT..."
# -f means: go into background
# -C means: use compression
# -2 means: only use SSH2
# -R 9101:$LOCAL:9101 means: when client connects to remote port 9101 (bacula-dir), it will be
#    forwarded to this machine.
# -R 9103:$LOCAL:9103 means: when client connects to remote port 9101 (bacula-sd), it will be
#    forwarded to this machine.
# -L 9112:localhost:9102 means: when bacula-dir connects to port 9112 (instead of the normal 9102),
#    it will be forwarded to the client's FD. The client will think the connection was to port
#    9102 as usual
# sleep 60 is a simple command that will execute on the server and does nothing for 60 seconds,
# then it exits. This keeps ssh running for 60 seconds. Once we connect to the FD, that
# connection will keep ssh running even beyond the 60 seconds.
# Using this approach, we do not need to tear down the tunnel later, it disconnects itself
# automagically.
# It is important to redirect stdout and stderr, or else bacula will not realize that the
# script has terminated.
$SSH -fC2 -R 9101:$LOCAL:9101 -R 9103:$LOCAL:9103 -L 9112:localhost:9102 $CLIENT sleep 60 >/dev/null 2>/dev/null
# give ssh a little time to establish the connection.
sleep 10

Important You must make sure that the client's SSH fingerprint is added to the known_hosts file on the server. Otherwise, the SSH connection, and thus the backup, will fail. You can add it either to the global known_hosts file, or to the known_hosts file for the bacula user. The easiest way to do it:

su bacula
/usr/local/sbin/sshbacula <yourclientFQDN>
# verify the fingerprint, and then answer Yes when prompted.
# repeat the above command for each client
exit

You only need to do this once; SSH remembers the fingerprint.

9. Hints

I found that sometimes, the connection to the SD could not be established reliably. It helped to allow more time. Edit the bacula-sd.conf file and add the following line to the Storage resource:

Client Connect Wait = 120

Even with that, remote backups are inherently somewhat less reliable than local backups; a full backup sometimes takes several attempts. I added the following line to my JobDefs resource to automate that:

Rerun Failed Levels = yes
sshtunnel.txt · Last modified: 2012/10/29 12:40 by sebastian_held_sim-gmbh.de
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki