Archival Storage

Archival storage is optimized for very low cost and low footprint of media and a very long durability.

This means that raw media (magnetic tape, magnetic or optical discs) is usually separate from the drives that are used to read and write the media. That way media can also be stored in a different place, usually a darker, cooler, dryer and more secure environment that benefits durability. Media must be loaded into drives as cartridges or magazines which either requires human intervention or a robotic loader which is part of large libraries. Access latency is much higher than that of online storage where data rests on spinning disks.

HDDs for Cold Storage

A recent improvement in disc technology that allows for higher data density is Shingled Magnetic Recording (SMR). Seagate and HGST sell SMR drives with 8-10 TB capacity. Normal HDDs store data in non-overlapping parallel tracks (perpendicular recording) on magnetic platters. SMR drives overlap these tracks making each track narrower and increasing capacity. Manufacturers report an average increase of 25% compared to perpendicular drives. The downside of SMR is that write speed decreases because adjacent tracks must be overwritten when writing to a track. Hence SMR drives are best suited for backup and archival data that is written once. SMR drives are not useful in situations where writes are frequent and random, such as for databases or desktop computers.

LTO Tapes and Tape Libraries

Linear Tape-Open is an open standard for magnetic tape storage developed by HP, IBM and Quantum. Originally introduced in 2000 the recent generation LTO-7 was published in 2015. LTO defines physical dimensions of tape cartridges and the logical data format stored on tape which makes LTO interoperable between equipment vendors.

LTO-7 has a native data capacity of 6TB or 15TB with 2.5:1 compression while the ucompressed data rate is 300 MB/s. That means a full write of 6TB would take 5.5 h. LTO-7 tapes need about 11 sec (19 sec) to load (unload), average file access takes 56 sec and rewind of the 960m long tape takes 108 sec max and 58 sec on average.

LTO tape media is designed to be durable for 15 to 30 years of storage and in case of backup operations it can sustain up to 16000 full rewrite cycles. LTO drives may remain operational between 5-10 years, depending on usage pattern. Interoperability between LTO generations is an important concern because new drives must be able to read back old media in the future. LTO drives are downward compatible, which means they can read media from their own generation up to two prior generations and write media up to one prior generation. As an example, an LTO-7 drive can read LTO-5, LTO-6 and LTO-7 media and write LTO-6 and LTO-7 media. Upward compatibility is not defined, that is an LTO-6 drive cannot read or write LTO-7 media.

Drives for single-user operations are connected with USB-3 or Thunderbolt. They support a single LTO medium which is manually loaded. For large storage requirements and multi-user systems there are many mid-range and enterprise-level tape libraries with multiple drives and robotic arms that can automatically load from arrays of thousands of tapes.

Archival Disc

Archival Disc (AD) is a new optical storage technology developed by Sony and Panasonic. AD is a successor technology to Blu-Ray, but specifically designed for long-term storage of up to 100 years durability. Similar to Blu-Ray physical disc dimensions are 120mm diameter and 405nm lasers, but novel features are a new recording material, a three-layered and two-sided layout and a new logical format. A single AD disc can store up to 300 GB of user data. Available single-user drives can read at up to 140 MB/s, write at speeds of up to 90 MB/s to write once media and at speeds of 40 MB/s to rewritable media.

AD media are packaged and sealed in cartridges of up to 12 discs to protect them from dust and physical contact. A single cartridge appears to the user as one volume that exposes UDF as the only available on-disc file system. Cartridges with write-once or re-writable media are available in capacities of 300 GB, higher capacities of 1.2 TB up to 1.5 TB are planned. Re-writable media can be written about 1000 times. However, when deleting a single file, the capacity on disc is not restored unless the entire cartridge is reformatted.

Several scalable AD Libraries with cartridge auto-load and support for multiple drives and expansion units exist. The Sony Everspan Library System, for example, is a storage jukebox with a base module that can contain 16 to 64 drives. Combined with a robotic arm unit and up to 14 expansion units it can be extended to 181 PB capacity. Each drive can read at 280 MB/s and write 140 MB/s which adds up to 18 GB/s read throughput per base module. Media load time is 60-90 s.