Open-E logo
0 Liked

    A few words about hardware compression in a backup with tape drives.

    Most present tape drives include a function called hardware compression.
    This makes available data compression to the magnetic tape by a drive. In many
    cases this feature may prove very useful.

    The hardware compression is much faster than a software compression
    because as opposed to the software compression it does not use a
    computer processor which draws from resources. Also it is transparent to the operating
    system and data is compressed “on the fly”.

    Usually compression ratio for magnetic tape drives is 2:1. This means
    that if we would write 2MB data to a tape then the drive will compress
    the data and you will have saved only 1MB compressed data on the tape.
    Sounds good, yes? But not really!

    In fact compression ratio of 2:1 really may be equal of 1.2:1 or 1.6:1
    or another ratio. That depends the type of data which you are writing
    onto tape. For uncompressed data types (such as txt, bmp, etc.) real
    ratio may be near 2:1 but if you are going to write pre-compressed data
    (for example multimedia data type such as mpg, jpg, mp3, etc.) this
    ratio will be very poor and in some compression algorithms may cause the
    written data to tape to be larger than the original data.

    Magnetic tapes manufacturers provide two tape capacities. First – written
    a large font – a compressed capacity and second – usually written
    smaller font – a native data capacity. For example a tape that is LTO-4 (Linear
    Tape-Open 4 generation) has native capacity of 800GB. If the compression
    ratio is 2:1 then the compressed capacity is 1600GB (in theory). If you do
    not use hardware compression then with LTO-4 tape you can write 800GB
    maximum. When you use hardware compression you can write more than
    800GB. How much? It depends on your data.

    The compression ratio is also related to the speed obtained during
    operations on magnetic tape. This speed is greater when the compression
    ratio is better. For example, the tape drive has a native speed of
    120MB/s. With 2:1 compression ratio of this speed is doubled. Also in
    this case, the real speed is dependent on the extent to which data will
    be compressed.

    Typically default hardware compression is turned on but this is not a set
    rule. If a compression functionality for your drive is not enable as
    Default, you may turn on it in a few ways depending on the drive. Sometimes
    you can switch it on by an extra jumper on corpus or through a program via the
    operating system.

    Rating: / 5.

    No votes yet


    • Alan Brown

      November 16, 11 2017 05:00:58

      ” in some compression algorithms may cause the written data to tape to be larger than the original data.”


      All drives made in the last 20 years which have hardware compression will switch the compression off and write blocks directly if expansion occurs. There is _no_ good reason to disable hardware compression in any modern tape drive (The last drives which were guilty of the stated behaviour were DDS-2 DAT format)

      _Much_ greater gains in write speeds can be obtained by increasing the “blocking factor” if writing tarfiles, or using larger maximum block sizes and file sizes if using backup software such as Amanda or Bacula.

      In most cases write speed increases of 100-200% can be obtained simply by changing the maximum block size written to tape from the software default (usually 64kB) to 1MB. Increasing beyond this doesn’t give much speed increase even on LTO7.

      Further Job speed increases can be obtained by increasing the tape file size from the default (usually 1GB) to at least 10GB. This is because the tape drive must stop and write a file marker at the end of each file, which takes a few seconds on almost all modern half inch formats. (LTO, TS10000, TS1100)

      On top of all that, tape drives are FAST – much faster than mechanical disk drives, even when sequentially reading/writing. If you don’t keep their buffers full, LTO drives will drop to lower linear speeds and as a last resort start “shoe shining” if fed data at less than 1/4 their raw data rate (which is 160MB/s for LTO5 and over 250MB/s for LTO7). Once this happens, tape wear increases dramatically and throughput drops to less than 20MB/s. Therefore, unless your fileserver is idle and connected at 10Gb/s or faster to the backup server, it is good practice to use a large, fast SSD as an intermediate spool drive when writing to tape.

      VA:F [1.9.22_1171]
      Rating: +5 (from 5 votes)
    • Alan Dean

      April 12, 04 2018 12:52:16

      Thanks for providing a very useful information. Really Nice Post…

      VA:F [1.9.22_1171]
      Rating: 0 (from 0 votes)

    Leave a Reply