Bacula And Removable Disk HOWTO

Josh Fisher

Revision History

Revision 0.9.1 2009-10-27

revision 0.9 2009-01-26

Revision 0.8 2008-12-06

  • Converted to XHTML.
  • Hosted by Brian DeRocher on derocher.org because this document could not be found on the Internet. Also i was unable to contact Josh at jfisher@jaybus.com.

Revision 0.7.4 2006-12-12

  • Updated parameter checks in vchanger script.
  • Fixed bug in vchanger script causing abort when no magazines have yet been created
  • Add 'baculasd_user' config file parameter to maintain correct ownership and permissions when vchanger is invoked by root
  • Added 'purge' command to allow purging all volumes in magazine from the Bacula catalog

Revision 0.7 2006-11-13

  • Initial version.

This document describes how to utilize removable disk drives as backup media for a backup solution using Bacula.
In addition to reading this guide you could check out "disk-changer" script which is part of bacula distribution.

1. Introduction

This document describes how to utilize removable disk drives as backup media for a backup solution using Bacula. Having seen many people ask about the use of USB disk drives as a backup media on the Bacula User's e-mail list, I felt that producing this document based on my experience with using USB disk drives as backup media might be beneficial to some bacula users.

What follows is the description of a method to use several USB disk drives as the backup media for a backup solution using Bacula and is based on my experience with setting up a backup system for a small company with three servers and around 10 workstations. This method involves emulating a traditional magazine-based tape library device by using partitions on the USB disk drives as the “magazines”. As such, the method would work equally well for disk drives in Firewire, eSATA, or any other hot-swappable enclosures supported by the OS.

1.1 Copyright And License

This document, Bacula And Removable Disk HOWTO, is copyrighted © 2006 by Josh Fisher. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is available at http://www.gnu.org/copyleft/fdl.html.

1.2 Disclaimer

No liability for the contents of this document can be accepted. Use the concepts, examples and information at your own risk. There may be errors and inaccuracies which could damage to your system. Though this is highly unlikely, proceed with caution. The author(s) do not accept responsibility for your actions.

All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements.

1.3 Credits / Contributors

Thanks to all those who frequent the Bacula User's e-mail list, and of course to Kern Sibbald and the other Bacula developers. Thanks to Jay Fisher jfisher [at] jaybus [dot] com for the initial version.

1.4 Feedback

The most up to date version of this HOWTO can be found at http://derocher.org/Docs/Bacula_And_Removable_Disk_HOWTO.html. Send your comments and corrections to Brian DeRocher brian [at] derocher [dot] org. Please send unified diffs if you make any changes. And please make one change at a time.

2. Definitions

Bacula is an open source network backup solution. Much more information about Bacula is available at the Bacula website.

A removable disk storage device is a random access disk storage device that is external to, or easily removable from, a computer system. Examples are external drive enclosures that connect via USB, Firewire, eSATA, or other hot-swappable interfaces.

vchanger is a shell script that emulates a traditional tape library device and acts as an interface to Bacula.

A USB disk drive is an external enclosure that connects to a computer system via a USB interface and usually houses a hard disk drive designed for either desktop or laptop computers. Those that use laptop hard drives are more shock tolerant and better suited to transporting.

The term hot-swappable refers to hardware devices that may be attached to and removed from a running computer system.

3. Requirements

Briefly, the requirements are:

  • Two or more removable disk storage devices
  • A server running bacula-dir, bacula-sd, udev, and autofs
  • A network connecting the servers and workstations to be backed up
  • The vchanger script and a vchanger configuration file for each virtual autochanger

4. How It Works

USB disk drives are used to emulate a multi-drive magazine-based tape library. The heart of the emulation is the vchanger shell script. This script understands the Bacula Autochanger Interface, accepting commands from the bacula storage daemon, performing the command, and then returning a result. When vchanger is used as the Autochanger Command in an Autochanger resource in Bacula Storage Daemon configuration file, (bacula-sd.conf), the Storage Daemon will invoke vchanger to load and unload virtual tapes, list the barcodes of the “tapes”, etc.

Vchanger treats a partition on a USB disk drive as a virtual magazine. Each “magazine” filesystem is given a filesystem label that links it to a particular virtual autochanger. In other words, all magazines belonging to a particular autochanger are given the same filesystem label. An autochanger's magazines are distinguished by a magazine index number in the range 1-99. The magazine index number is stored as a line of text in a file named 'index' in the magazine partition's root directory.

Each autochanger is configured for a fixed number of virtual slots per magazine, and of course, all magazines belonging to an autochanger have the same number of slots. Each slot is “occupied” by a regular file which acts as a virtual tape that will contain a bacula volume. These files are named using the pattern 'mIIsJJJ', where II is replaced by the (one-based) two digit magazine index number and JJJ is replaced by the (one-based) three digit slot number. This name is used as the “barcode” of the “tape”, and Bacula will also use this as the volume label name when the volumes in the magazine are labeled using Bacula's 'label barcodes' command.

Bacula is configured to invoke vchanger with the following parameters:

vchanger "config_file" "command" "slot_number" "archive_device" "drive_index"

The first parameter gives the location of the vchanger configuration file to use. Every virtual autochanger has its own configuration file, (defined in install section below).

The second parameter is the autochanger command to be performed and is either one of the standard Bacula autochanger API commands:

  • load Load a “tape” from a magazine “slot” into a “drive”. Vchanger creates a symlink with the name passed in parameter 4 (archive_device) pointing to the file mIIsJJJ, where II is the two digit magazine index and JJJ is the slot number passed to Vchanger as parameter 3.
  • unload Unload a “tape” from the “drive” back into the magazine “slot”. Vchanger deletes the symlink that was created by a previous load command for the drive.
  • loaded Return the “slot” currently loaded into the “drive”.
  • list List the barcodes of the “tapes” in each of the “slots” in this magazine.
  • slots Return the number of slots per magazine.

… or one of the vchanger extended commands:

  • purge Delete all volumes in a magazine and re-create them using the bconsole 'label barcodes' command. Volumes are deleted from the Bacula catalog using the bconsole 'delete volume' command, and the files acting as virtual tapes containing those volumes are deleted from the virtual magazine filesystem.

The third parameter is the slot number to use for the current command.

The fourth parameter, the Archive Device, will be the path where vchanger should create a symlink when loading a drive.

The fifth parameter is the drive index of the drive this command affects.

5. Mounting USB Disk Drives

5.1 udev And Hot-swappable Drives

Udev is the subsystem that dynamically assigns device nodes to hot-swappable hardware devices whenever they are attached to a running system and frees those device nodes whenever they are detached. There is no guarantee that a particular piece of hardware will be assigned a known device node when it is plugged in. The device node that udev assigns will depend on how many other devices are already attached and could change every time the hot-swappable device is plugged in.

Fortunately, udev provides a way to create aliases to the device nodes it assigns so that the alias is always at a known path. On most modern *nix systems, aliases to hot-swappable disk storage devices are placed under /dev/disk. Of particular interest for the virtual autochanger we are creating from USB disk drives, udev uses the filesystem label of any mountable partitions on the USB disk drive to create a symlink under /dev/disk/by-label that points to the real device node that udev assigned to the disk partition. By labeling the filesystems on the disk partitions that will be used as our backup media, we then gain access to the dynamically assigned device node via the alias (symlink). For example, if we set the filesystem label to 'usbchanger1' on all of the partitions we will be using for the virtual autochanger, then we can access the device node through the alias at /dev/disk/by-label/usbchanger1, where /dev/disk/by-label/usbchanger1 is simply a symlink pointing to the real device node that udev assigned to the partition when the USB drive was plugged in.

Note that while it is possible to write udev rules that create an alias at some other path, writing udev rules is beyond the scope of this document. If you want (or need) to do this, then a good place to start might be http://www.kernel.org/pub/linux/utils/kernel/hotplug/udev.html.

5.2 Using autofs To Mount Magazine Partitions

Udev provides the means to access device nodes for the USB drive's partitions, but we still must somehow get them mounted so that Bacula can read and write volumes on them. There are many ways to do this, and in fact newer releases of Bacula provide configuration variables for just this purpose. It is also possible to create entries in /etc/fstab using the alias described above. The method described here, however, makes use of autofs to mount the USB drive's partition(s).

The autofs daemon provides a way to mount and unmount the partitions on-the-fly as they are accessed. Since we know the device node (alias) will be at /dev/disk/by-label/usbchanger1 for our 'usbchanger1' autochanger, we can select a mountpoint of /mnt/usbchanger1/magazine and create a file named /etc/auto.usbchanger1 as follows:

# /etc/auto.usbchanger1
magazine  -fstype=auto,rw  :/dev/disk/by-label/usbchanger1
# eof

Add an entry to /etc/auto.master to tell autofs to pull in the new auto.usbchanger1 configuration:

# /etc/auto.master
# ...
/mnt/usbchanger1    /etc/auto.usbchanger1    --timeout=30
# eof

Now restart the autofs daemon and when a drive that has a partition with a filesystem label of 'usbchanger1' is plugged in, it can be accessed at /mnt/usbchanger1/magazine without any need for an explicit mount command. Autofs will mount the partition on-the-fly as needed.

6. Preparing USB Disk Drives

Preparing the USB drives for use with the virtual autochanger is fairly simple. All that is needed is a partition on the drive with a filesystem that has a filesystem label named appropriately for the virtual autochanger it will be used with. Most USB drives come from the factory with a single large FAT32 partition. While that could be used by simply giving it a filesystem label and adjusting the autofs configuration from section 5.2 above appropriately, in this example we will be using a ext3 filesystem on a single partition encompassing the entire disk. The preparation procedure detailed in section 6.1 and section 6.2 below is used to prepare each drive to be used.

6.1 Partitioning

To partition a drive we need to know the drive's device node. This can usually be determined from the output of dmesg a few seconds after the drive is plugged in. After finding the drive's device node, say for example /dev/sdc, use fdisk /dev/sdc (or some other partitioning tool) to create a single Linux partition as the drive's primary partition number 1.

6.2 Formatting / Labeling Magazine Partitions

Assuming the drive was assigned node /dev/sdX, the partition can now be formatted and labeled using:

mke2fs -j -T largefile -L "usbchanger1" -m 0 /dev/sdX1

This will create a new ext3 filesystem on the partition with filesystem label 'usbchanger1'. Now unplug the USB drive, wait a few seconds, then plug it back in. A few seconds after plugging it in you should see a symlink /dev/disk/by-label/usbchanger1 pointing to whatever device node udev decided to assign to the partition. Now try:

ls /mnt/usbchanger1/magazine

This should cause autofs to mount the partition, and the output from ls should show an empty filesystem (or possibly a single 'lost+found' directory). The output from the 'df' command should show that the partition is mounted at /mnt/usbchanger1/magazine.

Now there is one thing left to do to prepare the drive. Unless you will be running the Bacula Storage Daemon bacula-sd as root, (not recommended), you will need to set the owner, group, and permissions for the filesystem on the new partition using:

chown -R bacula.disk /mnt/usbchanger1/magazine chmod 0770 /mnt/usbchanger1/magazine

7. Installing vchanger

Place a copy of the vchanger shell script in the /etc/bacula directory, than change its owner and permissions as follows:

chown root.disk /etc/bacula/vchanger
chmod 750 /etc/bacula/vchanger

The first positional parameter passed to vchanger from Bacula specifies the autochanger configuration file. Each virtual autochanger will have its own configuration file. An autochanger configuration file defines the following values:

  • baculasd” [Required] (Default: none) The name of the autochanger, as defined in bacula-dir.conf by the Name parameter of the Storage resource for the autochanger device.
  • baculasd_user” (Default: bacula) Specifies the user that the Bacula Storage Daemon bacula-sd runs as.
  • bconsole” (Default: /etc/bacula/bconsole) Path to the bconsole executable.
  • magslots” (Default: 10) The number of slots in each magazine.
  • maxdrive” (Default: 0 (which defines 1 virtual drive)) The maximum zero-based drive index number to use. This defines the number of virtual drives the autochanger uses. Multiple drives allow Bacula to concurrently access multiple volumes. The number of virtual drives used must be less than or equal to the number of slots in a magazine.
  • mountpoint” [Required] (Default: none) Path to where the magazine partitions get mounted by autofs.
  • purgepool” (Default: Scratch) Specifies the pool that all volumes will be placed in following a purge command that deletes and then re-creates a magazine's volumes.
  • statedir” [Required] (Default: none) Path to the working directory for this autochanger.

For example, let's say we have two USB disk drives, each of which has been initialized as in section 6 above to contain a single large ext3 filesystem with filesystem label 'usbchanger1' that gets mounted at /mnt/usbchanger1/magazine. The following vchanger configuration file defines an autochanger that has 20 slots in a magazine, two virtual drives, and its state information stored in /var/bacula/usbchanger1:

# /etc/bacula/usbchanger1.conf
baculasd="usbchanger1"
baculasd_user=bacula
bconsole=/etc/bacula/bconsole
magslots=20
maxdrive=1
mountpoint=/mnt/usbchanger1/magazine
statedir=/var/bacula/usbchanger1
# eof

The owner and permissions for /etc/bacula/usbchanger1.conf should be set using:

chown root.disk /etc/bacula/usbchanger1.conf
chmod 0640 /etc/bacula/usbchanger1.conf

Note that the mountpoint is set to the the mountpoint we defined when setting up autofs in the section above.

8. Testing vchanger

The vchanger script may be tested by running it from the command line as the user that bacula-sd runs as. For the autochanger with config file /etc/bacula/usbchanger1.conf, issue (as root):

su bacula /etc/bacula/vchanger /etc/bacula/usbchanger1.conf list

The above command should list the barcodes for the virtual volumes in each of the magazine's slots. If the working directory for the autochanger does not exist it will be created. New magazine partitions will be initialized and assigned the next available magazine index. The magazine partition filesystem should have a file named 'index' and one file for each slot, all of which should be owned by the bacula-sd user and have mode 0640. The index file should contain the magazine's index number as a single line of text.

The above list command will fail if there is no magazine loaded, or in other words, no removable drive with a correctly labeled filesystem is plugged in or could not be mounted. It will also fail if there is a permissions problem. You should correct any problems now before continuing to section 9.

9. Configuring Bacula To Use The Autochanger

The virtual autochanger must be defined by adding Autochanger and Device resources to Bacula's configuration files as described in section 9.1 and section 9.2. After making changes to Bacula's configuration, the Storage Daemon and Director must be restarted for these changes to take effect.

9.1 Configuring The Storage Daemon

To configure the Bacula storage daemon (bacula-sd) we add an Autochanger resource and associated Device resource(es) to bacula-sd.conf. For the example 2 drive, 20 slot autochanger we created in section 7 above, we define:

# /etc/bacula/bacula-sd.conf
# ...
#----  local virtual autochanger with USB drive "magazines"
Autochanger {
  Name = usb-changer-1
  Device = usb-changer-1-drive-0
  Device = usb-changer-1-drive-1
  Changer Command = "/etc/bacula/vchanger %c %o %S %a %d"
  Changer Device = "/etc/bacula/usbchanger1.conf"
}
#---  drive 0 of the usb-changer-1 autochanger
Device {
  Name = usb-changer-1-drive-0
  DriveIndex = 0
  Autochanger = yes;
  DeviceType = File
  MediaType = File
  ArchiveDevice = /var/bacula/usbchanger1/drive0
  RemovableMedia = no;
  RandomAccess = yes;
}
#---  drive 1 of the usb-changer-1 autochanger
Device {
  Name = usb-changer-1-drive-1
  DriveIndex = 1
  Autochanger = yes;
  DeviceType = File
  MediaType = File
  ArchiveDevice = /var/bacula/usbchanger1/drive1
  RemovableMedia = no;
  RandomAccess = yes;
}# ...
# eof

In the Autochanger resource, the Changer Device value is the path to the vchanger configuration file for this autochanger. The Changer Command value is the command Bacula will execute when it needs the autochanger to perform some function, (like loading a “tape”). Here, we have installed the vchanger script at /etc/bacula/vchanger, and bacula is going to pass the Changer Device value in parameter 1, the command (load, unload, etc) in parameter 2, the slot number for the command in parameter 3, the Archive Device value from the Device resource that has a Drive Index value equal to the drive index for the command in parameter 4, and the drive index for the command in parameter 5.

In the Device resource(es) it is important to set Device Type = File so that Bacula understands it is reading/writing a File Storage type volume. The ArchiveDevice value is set to a path where vchanger will dynamically create a symlink to the requested slot's file when a “tape” is loaded. The vchanger script requires that this path be STATEDIR/driveN, where STATEDIR is the working directory for the autochanger that was defined in the vchanger config file and N is the drive index. In the example, the statedir was defined in /etc/bacula/usbchanger1.conf as /var/bacula/usbchanger1, so we used ArchiveDevice=/var/bacula/usbchanger1/drive0 for drive index 0 and ArchiveDevice=/ var/bacula/usbchanger1/drive1 for drive index 1.

9.2 Configuring The Director

The Bacula Director is configured to use the autochanger more or less as it would be configured for a traditional tape autochanger.

# /etc/bacula/bacula-dir.conf
# ...
# Local USB drive virtual autochanger
Storage {
  Name = usbchanger1   # same as defined by 'baculasd' in vchanger config file
  Address = 192.168.1.3
  SDPort = 9103
  Password = "secret_password"
  Device = usb-changer-1  # name of the Autochanger resource defined in bacula-sd.conf
  Media Type = File
  Autochanger = yes;
}
# ...
Pool {
  Name = Scratch
  Pool Type = Backup
}
# ...
# eof

Note that we have a Scratch pool defined in bacula-dir.conf. Bacula treats this pool differently than other pools in that if a job requires a new volume from a pool that has no available volumes, then Bacula will automatically move a volume from the Scratch pool into the pool that needs a new volume. When the volumes in new virtual magazines are labeled, (usually using the 'label barcodes' command from bconsole), we will initially place all of the new volumes into the Scratch pool so that they can be moved into other pools when and as needed.

10. Using The Autochanger

After restarting Bacula Storage Daemon and Director to load the new configuration, the Storage device 'usbchanger1', the virtual autochanger that has been created in the examples above, is ready to use.

10.1 Labeling Volumes In New Magazines

Plug in one of the newly prepared USB drives from section 6 above, then from bconsole execute the command 'label barcodes'. When prompted for the pool to place the newly labeled volumes into, select the Scratch pool. Bacula will ask the vchanger script for the barcodes of the “tapes” in each of the magazine's “slots” and use those barcodes will be used as the volume label names for the virtual tapes. The first magazine added to the autochanger will be given magazine index 1. The next will be given index 2, and so on.

10.2 Unloading Magazines

To unload a magazine, first check the status of the director with the command 'status dir' from within bconsole to make sure no currently running jobs are using a volume on the autochanger, then simply unplug the USB drive.

10.3 Loading Magazines

When Bacula requests a volume on another magazine, it will be necessary to load that magazine into the autochanger. Which magazine to load will be easily discovered from the volume label name that is being asked for. The two digit number following the 'm' at the beginning of the volume name is the magazine index number. To load the magazine, first unload any currently loaded magazine as described above, then plug in the requested magazine and from bconsole issue the command 'update slots'. If prompted for a drive index, just select zero. Bacula will ask the vchanger script for the barcodes of the volumes in the magazine and update the catalog database accordingly.

10.4 Adding New Magazines

You can add up to 99 magazines to an existing autochanger. Just prepare a new USB drive as described in section 6 above and then label its volumes as in section 10.1 above. A new magazine will be created using the next available magazine index number.

10.5 Purging All Volumes In a Magazine

In addition to the standard Bacula Autochanger API commands, vchanger defines an extended purge command. The purge command deletes all volumes in the current magazine and then re-creates them, placing the re-created volumes in the Scratch pool (or in the pool defined by the purgepool keyword in the vchanger configuration file). Volumes are deleted by first issuing a 'delete volume' command to delete the volume from Bacula's catalog database and then deleting the file representing the virtual tape for the volume from the virtual magazine filesystem. The volumes are then re-created with the 'label barcodes' command and placed in the Scratch pool.

The purge command must be invoked as root using:

vchanger /path/to/vchanger/config/file purge

CAUTION! The purge command re-creates the current magazine's volumes regardless of retention time or volume status. Data contained in those volumes is lost following a purge command and cannot be restored. Do not use the purge command unless you are sure those volumes are no longer needed.

11. Scripts

11.1 vchanger

#!/bin/sh
#
#  Bacula interface to virtual autochanger using removable disk drives
#
#  Based (somewhat) on the "disk-changer" script from bacula 1.39.26
#
#  Vchanger is a Bacula autochanger script that emulates a conventional
#  magazine-based tape library device using removable disk drives.
#  Partitions on the removable drives are used as virtual magazines,
#  where each "magazine" contains the same number of virtual slots. Each
#  "slot" holds one virtual tape, where a "tape" is a regular file that
#  Bacula treats as a "Device Type = File" volume.
#
#  This script will be invoked by Bacula using the Bacula Autochanger
#  Interface and will be passed the following arguments:
#
#  vchanger "changer-device" "command" "slot" "archive-device" "drive-index"
#                 $1            $2       $3          #4             #5
#
#  See the Bacula documentation and the Bacula Removable Disk Howto for
#  further details.
#
#  Copyright (C) 2006 Josh Fisher
#
#  Permission to use, copy, modify, distribute, and sell this software
#  and its documentation for any purpose is hereby granted without fee,
#  provided that the above copyright notice appear in all copies.  This
#  software is provided "as is" without express or implied warranty.
#
#  This software is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# $Id: vchanger,v 0.7.4 2006/12/01 09:29:04 jfisher Exp $

#
# log whats done
#
# to turn on logging, uncomment the following line
#touch $wd/vchanger.log
#

#
# Write to a log file
#    To log debugging info, create file /var/bacula/vchanger.log
#    with write permission for bacula-sd user. To stop logging,
#    delete file /var/bacula/vchanger.log
#
dbgfile="/var/bacula/vchanger.log"
function debug()
{
    if test -e $dbgfile; then
	echo "`date +\"%Y%m%d-%H:%M:%S\"` $*" >> $dbgfile
    fi
}

#
# Return length of string $1
#
if [ `uname` = "FreeBSD" ]
then
        function strlen ()
        {
                expr -- "$1" : ".*"
        }
else
        function strlen ()
        {
                expr length $1
        }
fi


#
# Prepend zeros to $1 and return a string that is $2 characters long
#
function mklen ()
{
   o1=$1
   while [ `eval strlen ${o1}` -lt ${2} ]; do
      o1="0${o1}"
   done
   echo $o1
}

#
# Get uid of $1 (or current user if $1 empty)
#
function get_uid() {
   id $1 2>/dev/null | cut -d ' ' -f 1 | sed "s/uid=//" | cut -d '(' -f 1
}


#
# Initialize autochanger's state directory if not already initialized
#
function init_statedir() {
   debug "Initializing $statedir"
   # Create state directory if needed
   if [ ! -d "${statedir}" ]; then
      if [ "$su_uid" != "" ]; then
         su -c "mkdir ${statedir} &>/dev/null" $su_uid
      else
         mkdir ${statedir} &>/dev/null
      fi
      if [ $? -ne 0 ]; then
         echo "Could not create ${statedir}"
         exit 1
      fi
      chmod 770 ${statedir} &>/dev/null
      if [ $? -ne 0 ]; then
         echo "Could not chmod ${statedir}"
         exit 1
      fi
   fi
   # Create nextmag file to hold max magazine index used
   if [ ! -f "${statedir}/nextmag" ]; then
      if [ "$su_uid" != "" ]; then
         su -c "echo 0 >${statedir}/nextmag" $su_uid
      else
	     echo 0 >${statedir}/nextmag
      fi
      if [ $? -ne 0 ]; then
         echo "Could not create ${statedir}/nextmag"
         exit 1
      fi
      chmod 660 ${statedir}/nextmag
      if [ $? -ne 0 ]; then
         echo "Could not chmod ${statedir}/nextmag"
         exit 1
      fi
   fi
   # Check nextmag value
   nextmag=`cat "${statedir}/nextmag"`
   if [ $? -ne 0 -o "${nextmag}" == "" -o $nextmag -lt 0 -o $nextmag -gt 99 ]; then
      echo "${statedir}/nextmag has invalid value"
      return 1
   fi
   # Create 'loaded' files for each virtual drive that hold the slot
   # number currently loaded in that 'drive'
   i=0
   while [ $i -le $maxdrive ]; do
      if [ ! -f "${statedir}/loaded${i}" ]; then
         if [ "$su_uid" != "" ]; then
            su -c "echo 0 >${statedir}/loaded${i}" $su_uid
         else
            echo 0 >${statedir}/loaded${i}
         fi
         if [ $? -ne 0 ]; then
            echo "Could not create ${statedir}/loaded${i}"
            exit 1
         fi
         if [ "$su_uid" != "" ]; then
            su -c "chmod 660 ${statedir}/loaded${i}" $su_uid
         else
            chmod 660 ${statedir}/loaded${i}
         fi
         if [ $? -ne 0 ]; then
            echo "Could not chmod ${statedir}/loaded${i}"
            exit 1
         fi
      fi
      i=`expr ${i} + 1`
   done
}


#
# Get magazine index of currently loaded magazine
#
function get_magazine() {
   debug "Get magazine index"
   # Check for mountpoint dir
   if [ ! -d ${mountpoint} ]; then
      echo "No magazine loaded at ${mountpoint}"
      exit 1
   fi
   # Check magazine for existing index
   if [ ! -f "${mountpoint}/index" ]; then
      echo "00"
      return 1
   fi
   mi=`cat "${mountpoint}/index"`
   if [ $? -ne 0 ]; then
      echo "Failed to read ${mountpoint}/index"
      exit 1
   fi
   # must be 1-99
   if [ $mi -lt 1 -o $mi -gt 99 ]; then
      echo "Magazine has invalid index ${mi}"
      exit 1
   fi
   # make magazine index 2 digits
   eval mklen ${mi} 2
   return 0
}

#
# Initialize magazine if not already initialized
#
function init_magazine() {
   debug "Initializing magazine"
   # Get max magazine index that has been used
   nextmag=`cat "${statedir}/nextmag"`
   if [ $? -ne 0 -o "${nextmag}" == "" ]; then
      echo "Failed to read ${statedir}/nextmag"
      exit 1
   fi
   # Check magazine for existing index
   if [ -f "${mountpoint}/index" ]; then
      # retrieve existing magazine index
      mi=`cat "${mountpoint}/index"`
      if [ $? -ne 0 ]; then
         echo "Failed to read ${mountpoint}/index"
         exit 1
      fi
      # must be 1-99
      if [ $mi -lt 1 -o $mi -gt 99 ]; then
         echo "Magazine has invalid index ${mi}"
         exit 1
      fi
   else
      # new magazine, so assign it the next avail index
      mi=`expr ${nextmag} + 1`
      if [ $mi -lt 0 -o $mi -gt 99 ]; then
         echo "Max magazines exceeded"
         exit 1
      fi
      if [ "$su_uid" != "" ]; then
         su -c "echo ${mi} >${mountpoint}/index" $su_uid
      else
         echo ${mi} >${mountpoint}/index
      fi
      if [ $? -ne 0 ]; then
         echo "Failed to write ${mountpoint}/index"
         exit 1
      fi
      chmod 640 ${mountpoint}/index 2>/dev/null
   fi
   # make sure max index used is up to date
   if [ $mi -gt $nextmag ]; then
      echo $mi 2>/dev/null >"${statedir}/nextmag"
      if [ $? -ne 0 ]; then
         echo "Failed to update ${statedir}/nextmag"
         exit 1
      fi
   fi
   # make magazine index 2 digits
   magindex=`eval mklen ${mi} 2`
   # setup slot files (ie. virtual tapes)
   i=1
   while [ $i -le $magslots ]; do
      s=`eval mklen ${i} 3`
      f="${mountpoint}/m${magindex}s${s}"
      if [ ! -f "${f}" ]; then
         if [ "$su_uid" != "" ]; then
            su -c "touch ${f} &>/dev/null" $su_uid
         else
            touch ${f} &>/dev/null
         fi
         if [ $? -ne 0 ]; then
            echo "Failed to create ${f}"
            exit 1
         fi
      fi
      i=`expr ${i} + 1`
   done
   return 0
}


#
# Get sd status with bconsole
#
function bconsole_sd_status() {
   debug "Doing 'status storage' with bconsole"
   $bconsole <<EOD_SDSTAT
status storage=$baculasd
quit
EOD_SDSTAT
}


#
# Delete volume with bconsole  param1 = slot
#
function bconsole_delete_volume() {
   debug "Doing 'delete volume' with bconsole"
   s=`eval mklen $1 3`
   $bconsole <<EOD_DELVOL
delete volume=m${magindex}s${s}
yes
quit
EOD_DELVOL
}


#
# Label volumes from barcodes with bconsole
#
function bconsole_label_barcodes() {
   debug "Doing 'label barcodes' with bconsole"
   $bconsole <<EOD_SDSTAT
label storage=$baculasd pool=$purgepool drive=0 barcodes
yes
quit
EOD_SDSTAT
}

#
# Checks if any of magazine's volumes are in use
#
function volumes_in_use() {
   debug "Checking for in use volumes"
   inuse=""
   bconsole_sd_status | while read f; do
      #echo $f
      a=`echo ${f} | grep ^${magindex}`
      if [ "$a" != "" -a "$inuse" == "" ]; then
         inuse=`echo $f | cut -d ' ' -f 1`
      fi
   done
   if [ "$inuse" != "" ]; then
      echo $inuse
   fi
}


#
# Unload all drives
#
function unload_drives() {
   debug "Unloading all drives"
   vuse=`eval volumes_in_use`
   if [ "${vuse}" != "" ]; then
      return 1
   fi
   i=0
   while [ $i -le $maxdrive ]; do
      unlink "${statedir}/drive${i}" 2>/dev/null >/dev/null
      echo "0" >"${statedir}/loaded${i}"
      i=`expr ${i} + 1`
   done
   return 0
}


#
# check parameter count on commandline
#
function check_parm_count() {
    pCount=$1
    pCountNeed=$2
    if test $pCount -lt $pCountNeed; then
	echo "usage: vchanger ctl-device command [slot archive-device drive-index]"
	echo "	Insufficient number of arguments arguments given."
	if test $pCount -lt 2; then
	    echo "  Mimimum usage is first two arguments ..."
	else
	    echo "  Command expected $pCountNeed arguments"
	fi
	exit 1
    fi
}

# Setup arguments
check_parm_count $# 2
ctl=$1
cmd=$2
slot=$3
device=$4
drive=$5

# Setup default config values
baculasd=
baculasd_user=bacula
bconsole="/etc/bacula/bconsole"
magslots=10
maxdrive=0
mountpoint=
purgepool="Scratch"
statedir=

# Pull in conf file
if [ -f $ctl ]; then
   . $ctl
else
   echo "Config file ${ctl} not found"
   exit 1
fi

# When invoked by root, create files/dirs as bacula-sd user
su_uid=
myuid=`eval get_uid`
if [ "$myuid" == "0" -a "$baculasd_user" != "" ]; then
   buid=`eval get_uid $baculasd_user`
   if [ "$buid" == "" ]; then
      echo "bacula-sd user $baculasd_user not found"
      exit 1
   fi
   if [ "$buid" != "0" ]; then
      su_uid=baculasd_user
   fi
fi

# check for required config values
if [ "${mountpoint}" == "" ]; then
   echo "Required variable 'mountpoint' not defined in ${ctl}"
   exit 1
fi
if [ "${baculasd}" == "" ]; then
   echo "Required variable 'baculasd' not defined in ${ctl}"
   exit 1
fi
if [ "${statedir}" == "" ]; then
   echo "Required variable 'statedir' not defined in ${ctl}"
   exit 1
fi
if [ "${magslots}" == "" -o $magslots -lt 1 -o $magslots -gt 999 ]; then
   echo "Ivalid value for 'magslots' in ${ctl}"
   exit 1
fi
if [ "${maxdrive}" == "" -o $maxdrive -lt 0 -o $maxdrive -ge $magslots ]; then
   echo "Invalid value for 'maxdrive' in ${ctl}"
   exit 1
fi
if [ "${bconsole}" == "" -o ! -f "${bconsole}" ]; then
   echo "Ivalid value for 'bconsole' in ${ctl}"
   exit 1
fi
if [ "${purgepool}" == "" ]; then
   echo "Invalid value for 'purgepool' in ${ctl}"
   exit 1
fi

# attempt to set defaults for params not specified on command line
if [ "${drive}" == "" ]; then
   drive=0
fi
if [ "${device}" == "" ]; then
   device="${statedir}/drive${drive}"
fi

# make sure archive device makes sense
if [ "${device}" != "${statedir}/drive${drive}" ]; then
   echo "Param 4 (archive device) must be ${statedir}/driveN,"
   echo "   where N is the drive number passed as param 5"
   exit 1
fi

# Initialize state directory for this autochanger
init_statedir

debug "Parms: $ctl $cmd $slot $device $drive"

#
#  Process command
#
case $cmd in
   list)
      check_parm_count $# 2
      debug "Doing list command"
      init_magazine
      if [ $? -ne 0 ]; then
         echo "Magazine Not Loaded"
         exit 1
      fi
      i=1
      while [ $i -le $magslots ]; do
         s=`eval mklen ${i} 3`
         echo "${i}:m${magindex}s${s}"
         i=`expr ${i} + 1`
      done
      ;;
   slots)
      check_parm_count $# 2
      debug "Doing slots command"
      echo $magslots
      ;;
   load)
      check_parm_count $# 5
      debug "Doing load slot $slot into drive $drive"
      if [ $drive -gt $maxdrive ]; then
         echo "Drive ($drive) out of range (0-${maxdrive})"
         exit 1
      fi
      if [ $slot -gt $magslots ]; then
         echo "Slot ($slot) out of range (1-$magslots)"
         exit 1
      fi
      ld=`cat "${statedir}/loaded${drive}"`
      if [ $? -ne 0 ]; then
         echo "Failed to read ${statedir}/loaded${drive}"
         exit 1
      fi
      if [ $ld -eq 0 ]; then
         unlink "${device}" &>/dev/null
         # make sure slot is not loaded in another drive
         i=0
         while [ $i -le $maxdrive ]; do
            if [ $i -ne $drive ]; then
               ldi=`cat "${statedir}/loaded${i}"`
               if [ $ldi -eq $slot ]; then
                  echo "Storage Element ${slot} Empty (loaded in drive ${i})"
                  exit 1
               fi
            fi
            i=`expr ${i} + 1`
         done
         init_magazine
         if [ $? -ne 0 ]; then
            echo "Magazine Not Loaded"
            exit 1
         fi
         s=`eval mklen ${slot} 3`
         if [ "$su_uid" != "" ]; then
            su -c "ln -s '${mountpoint}/m${magindex}s${s}' '${device}'" $su_uid
         else
            ln -s "${mountpoint}/m${magindex}s${s}" "${device}"
         fi
         echo $slot >"${statedir}/loaded${drive}"
      else
         echo "Drive ${drive} Full (Storage element ${ld} loaded)"
         exit 1
      fi
      ;;
   unload)
      check_parm_count $# 5
      debug "Doing unload drive $drive into slot $slot"
      if [ $drive -gt $maxdrive ]; then
         echo "Drive ($drive) out of range (0-${maxdrive})"
         exit 1
      fi
      if [ $slot -gt $magslots ]; then
         echo "Slot ($slot) out of range (1-$magslots)"
         exit 1
      fi
      ld=`cat "${statedir}/loaded${drive}"`
      if [ $? -ne 0 ]; then
         echo "Failed to read ${statedir}/loaded${drive}"
         exit 1
      fi
      if [ $slot -eq $ld ]; then
         echo "0" >"${statedir}/loaded${drive}"
         if [ $? -ne 0 ]; then
            echo "Failed to write ${statedir}/loaded${drive}"
            exit 1
         fi
         unlink "${device}" 2>/dev/null >/dev/null
         exit 0
      fi
      if [ $ld -eq 0 ]; then
         echo "Drive ${drive} Is Empty"
         exit 1
      else
         echo "Storage Element ${slot} is Already Full"
         exit 1
      fi
      ;;
   loaded)
      check_parm_count $# 5
      debug "Doing loaded command for drive $drive"
      if [ $drive -gt $maxdrive ]; then
         echo "Drive ($drive) out of range (0-${maxdrive})"
         exit 1
      fi
      if [ $slot -gt $magslots ]; then
         echo "Slot ($slot) out of range (1-$magslots)"
         exit 1
      fi
      cat "${statedir}/loaded${drive}"
      ;;
   purge)
      check_parm_count $# 2
      debug "Doing purge command"
      magindex=`eval get_magazine`
      if [ "$magazine" == "00" ]; then
         echo No magazine loaded
      fi
      cm=""
      while [ "$cm" != "y" -a "$cm" != "n" ]; do
         echo -n "Purge all volumes on magazine $magindex (y/n)? "
         read cm
         if [ "$cm" == "Y" ]; then
            cm="y"
         fi
         if [ "$cm" == "N" ]; then
            cm="n"
         fi
      done
      if [ "$cm" == "n" ]; then
         exit 0
      fi
      echo "Unloading $baculasd drives"
      unload_drives
      if [ $? -ne 0 ]; then
         echo Volume `eval volumes_in_use` in use...aborting
         exit 1
      fi
      echo "Deleting magazine $magindex volumes"
      i=1
      while [ $i -le $magslots ]; do
         bconsole_delete_volume $i &>/tmp/vclog
         rm -f "${mountpoint}/m${magindex}s${s}"
         echo "deleted volume m${magindex}s${s}"
         i=`expr ${i} + 1`
      done
      echo "Doing 'label barcodes' to re-create magazine's volumes"
      bconsole_label_barcodes | while read f; do
         echo $f
      done
      ;;
   *)
      echo "Command not recognized"
      exit 1
      ;;
esac

exit 0
# eof

11.2 Example Autochanger Config File

# Example 2-drive autochanger config file for vchanger 0.7.4
#
# baculasd
#    Specifies the bacula 'storage device' name for the autochanger
#    Default: none (but required)
#baculasd=
baculasd="usbchanger1"
#
# baculasd_user
#    Specifies the user that bacula-sd runs as. When vchanger is
#    invoked as root, files and directories will be created as this
#    user. Set this to root ONLY if you run bacula-sd as root.
#    Default: bacula
#baculasd_user=bacula
#
# bconsole
#    Path to the bconsole executable
#    Default: /etc/bacula/bconsole
#bconsole=/etc/bacula/bconsole
#
# magslots
#    Number of virtual slots per virtual magazine, where a virtual
#    magazine is a disk partition and slots are numbered 1 - n.
#    Default: magslots=10
#magslots=10
magslots=20
#
# maxdrive
#    Specify the maximum zero-based drive index. (ie. 0 for 1 drive,
#    1 for two drives, etc.)
#    Default: maxdrive=0
#maxdrive=0
maxdrive=1
#
# mountpoint
#    Path to where a virtual magazine gets mounted
#    Default: none (but required)
#mountpoint=
mountpoint=/mnt/usbchanger1/magazine
#
# purgepool
#    Specifies the pool a magazine's volumes will be assigned to
#    when a purge command is issued to delete and then re-create
#    a magazine's volumes.
#    Default: Scratch
#purgepool=Scratch
#
# statedir
#    Working directory for this autochanger
#    Default: none (but required)
#statedir=
statedir=/var/bacula/usbchanger1
removable_disk.txt · Last modified: 2010/11/12 21:08 by darrick
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki