GNU tar 1.35: 9.6 Using Multiple Tapes

Often you might want to write a large archive, one larger than will fit on the actual tape you are using. In such a case, you can run multiple tar commands, but this can be inconvenient, particularly if you are using options like ‘--exclude=pattern’ or dumping entire file systems. Therefore, tar provides a special mode for creating multi-volume archives.

Multi-volume archive is a single tar archive, stored on several media volumes of fixed size. Although in this section we will often call ‘volume’ a tape, there is absolutely no requirement for multi-volume archives to be stored on tapes. Instead, they can use whatever media type the user finds convenient, they can even be located on files.

When creating a multi-volume archive, GNU tar continues to fill current volume until it runs out of space, then it switches to next volume (usually the operator is queried to replace the tape on this point), and continues working on the new volume. This operation continues until all requested files are dumped. If GNU tar detects end of media while dumping a file, such a file is archived in split form. Some very big files can even be split across several volumes.

Each volume is itself a valid GNU tar archive, so it can be read without any special options. Consequently any file member residing entirely on one volume can be extracted or otherwise operated upon without needing the other volume. Sure enough, to extract a split member you would need all volumes its parts reside on.

Multi-volume archives suffer from several limitations. In particular, they cannot be compressed.

GNU tar is able to create multi-volume archives of two formats (see section Controlling the Archive Format): ‘GNU’ and ‘POSIX’.

9.6.1 Archives Longer than One Tape or Disk

To create an archive that is larger than will fit on a single unit of the media, use the ‘--multi-volume’ (‘-M’) option in conjunction with the ‘--create’ option (see section How to Create Archives). A multi-volume archive can be manipulated like any other archive (provided the ‘--multi-volume’ option is specified), but is stored on more than one tape or file.

When you specify ‘--multi-volume’, tar does not report an error when it comes to the end of an archive volume (when reading), or the end of the media (when writing). Instead, it prompts you to load a new storage volume. If the archive is on a magnetic tape, you should change tapes when you see the prompt; if the archive is on a floppy disk, you should change disks; etc.

The method tar uses to detect end of tape is not perfect, and fails on some operating systems or on some devices. If tar cannot detect the end of the tape itself, you can use ‘--tape-length’ option to inform it about the capacity of the tape:

When GNU tar comes to the end of a storage media, it asks you to change the volume. The built-in prompt for POSIX locale is(27):

Prepare volume #n for 'archive' and hit return:

where n is the ordinal number of the volume to be created and archive is archive file or device name.

(You should only type ‘y’ after you have changed the tape; otherwise tar will write over the volume it just finished.)

The volume number used by tar in its tape-changing prompt can be changed; if you give the ‘--volno-file=file-of-number’ option, then file-of-number should be an non-existing file to be created, or else, a file already containing a decimal number. That number will be used as the volume number of the first volume written. When tar is finished, it will rewrite the file with the now-current volume number. (This does not change the volume number written on a tape label, as per Including a Label in the Archive, it only affects the number used in the prompt.)

If you want more elaborate behavior than this, you can write a special new volume script, that will be responsible for changing the volume, and instruct tar to use it instead of its normal prompting procedure:

The command can contain additional options, if such are needed. See section Running External Commands, for a detailed discussion of the way GNU tar runs external commands. It inherits tar’s shell environment. Additional data is passed to it via the following environment variables:

These variables can be used in the command itself, provided that they are properly quoted to prevent them from being expanded by the shell that invokes tar.

The volume script can instruct tar to use new archive name, by writing in to file descriptor $TAR_FD (see below for an example).

If the info script fails, tar exits; otherwise, it begins writing the next volume.

If you want tar to cycle through a series of files or tape drives, there are three approaches to choose from. First of all, you can give tar multiple ‘--file’ options. In this case the specified files will be used, in sequence, as the successive volumes of the archive. Only when the first one in the sequence needs to be used again will tar prompt for a tape change (or run the info script). For example, suppose someone has two tape drives on a system named ‘/dev/tape0’ and ‘/dev/tape1’. For having GNU tar to switch to the second drive when it needs to write the second tape, and then back to the first tape, etc., just do either of:

$ tar --create --multi-volume --file=/dev/tape0 --file=/dev/tape1 files
$ tar -cM -f /dev/tape0 -f /dev/tape1 files

Finally, the most flexible approach is to use a volume script, that writes new archive name to the file descriptor $TAR_FD. For example, the following volume script will create a series of archive files, named ‘archive-vol’, where archive is the name of the archive being created (as given by ‘--file’ option) and vol is the ordinal number of the archive being created:

#! /bin/bash
# For this script it's advisable to use a shell, such as Bash,
# that supports a TAR_FD value greater than 9.

echo Preparing volume $TAR_VOLUME of $TAR_ARCHIVE.

name=`expr $TAR_ARCHIVE : '\(.*\)-.*'`
case $TAR_SUBCOMMAND in
-c)       ;;
-d|-x|-t) test -r ${name:-$TAR_ARCHIVE}-$TAR_VOLUME || exit 1
          ;;
*)        exit 1
esac

echo ${name:-$TAR_ARCHIVE}-$TAR_VOLUME >&$TAR_FD

The same script can be used while listing, comparing or extracting from the created archive. For example:

# Create a multi-volume archive:
$ tar -c -L1024 -f archive.tar -F new-volume .
# Extract from the created archive:
$ tar -x -f archive.tar -F new-volume .

Notice, that the first command had to use ‘-L’ option, since otherwise GNU tar will end up writing everything to file ‘archive.tar’.

You can read each individual volume of a multi-volume archive as if it were an archive by itself. For example, to list the contents of one volume, use ‘--list’, without ‘--multi-volume’ specified. To extract an archive member from one volume (assuming it is described that volume), use ‘--extract’, again without ‘--multi-volume’.

If an archive member is split across volumes (i.e., its entry begins on one volume of the media and ends on another), you need to specify ‘--multi-volume’ to extract it successfully. In this case, you should load the volume where the archive member starts, and use ‘tar --extract --multi-volume’—tar will prompt for later volumes as it needs them. See section Extracting an Entire Archive, for more information about extracting archives.

Multi-volume archives can be modified like any other archive. To add files to a multi-volume archive, you need to only mount the last volume of the archive media (and new volumes, if needed). For all other operations, you need to use the entire archive.

If a multi-volume archive was labeled using ‘--label=archive-label’ (see section Including a Label in the Archive) when it was created, tar will not automatically label volumes which are added later. To label subsequent volumes, specify ‘--label=archive-label’ again in conjunction with the ‘--append’, ‘--update’ or ‘--concatenate’ operation.

Notice that multi-volume support is a GNU extension and the archives created in this mode should be read only using GNU tar. If you absolutely have to process such archives using a third-party tar implementation, read Extracting Members Split Between Volumes.

9.6.2 Tape Files

To give the archive a name which will be recorded in it, use the ‘--label=volume-label’ (‘-V volume-label’) option. This will write a special block identifying volume-label as the name of the archive to the front of the archive which will be displayed when the archive is listed with ‘--list’. If you are creating a multi-volume archive with ‘--multi-volume’ (see section Using Multiple Tapes), then the volume label will have ‘Volume nnn’ appended to the name you give, where nnn is the number of the volume of the archive. If you use the ‘--label=volume-label’ option when reading an archive, it checks to make sure the label on the tape matches the one you gave. See section Including a Label in the Archive.

When tar writes an archive to tape, it creates a single tape file. If multiple archives are written to the same tape, one after the other, they each get written as separate tape files. When extracting, it is necessary to position the tape at the right place before running tar. To do this, use the mt command. For more information on the mt command and on the organization of tapes into a sequence of tape files, see The mt Utility.

--label="some-prefix `date +some-format`"

9.6.3 Concatenate Volumes into a Single Archive

Sometimes it is necessary to convert existing GNU tar multi-volume archive to a single tar archive. Simply concatenating all volumes into one will not work, since each volume carries an additional information at the beginning. GNU tar is shipped with the shell script tarcat designed for this purpose.

The script takes a list of files comprising a multi-volume archive and creates the resulting archive at the standard output. For example:

tarcat vol.1 vol.2 vol.3 | tar tf -

The script implements a simple heuristics to determine the format of the first volume file and to decide how to process the rest of the files. However, it makes no attempt to verify whether the files are given in order or even if they are valid tar archives. It uses dd and does not filter its standard error, so you will usually see lots of spurious messages.