LTO Media Archive with P5 – a recap
Archiware’s P5 Archive solution was created for those many content creators who need a long term archival solution for their valuable media data. The creation of video/audio/image data necessitates a reliable and simple workflow to migrate completed media files away from costly production storage to ‘cold’ storage where the data can reside safely for future decades. Should a media project require later re-purposing, one can access the archive, search/locate the required files and restore them back to production storage.
P5 Archive is a software product focussed on precisely this archive workflow, making the task both simple for users and flexible so that it can be used with a variety of storage technologies.
Storage options for archive
P5 Archive is flexible, supporting disk, cloud and tape as target storage for your archive data. Disk is generally a better option for in-progress production storage, but because spinning disks age and become unreliable over relatively short periods of time, the majority of P5 Archive’s customers prefer LTO tape for their archive storage. Many customers are currently investigating cloud storage too, but LTO remains where most professionals choose to keep their archive data safe.
With the release of version 6.1 of P5 Archive, we now have two different ways to write archive data onto tape. An LTFS volume format is now supported as an alternative way of writing data to tape using P5 Archive. This new format is in addition to the P5 volume format that has been available previously and continues to be available, going forward.
There are various reasons why you may wish to use this format, explored below.
What is LTFS
LTFS, “Linear Tape File System”, was introduced back in 2008 by the LTO Consortium of companies, as a way to make it easier for users to get their data onto LTO tapes. Prior to LTFS, software tools were required to write data in mostly proprietary formats onto tape and keep track of what had been written via an index or some sort. The same software would also be required to browse what had been written to a tape and then recover the data back to disk again (restoring).
LTFS provides a ‘file system’ on the tape itself, so files could simply be ‘dragged and dropped’ onto a tape, just like a regular filesystem on disk, or a USB memory stick. A software tool was still required to ‘mount’ the tape into the operating system, but the software could be very simple in scope and present the tape itself just like a disk. A user able to copy a file to a disk, would be able to do the same to an LTFS tape.
Vendors manufacturing the physical LTO drives included such software with the drives, but over time it wasn’t well maintained, nor easy to get working. Third party software vendors also provided simple tools to mount LTFS tapes into the operating system being used (macOS/Windows/Linux). Because tapes are much slower to access than disks, this would often lead to speed issues. Sometimes operating systems read files on disks for their own admin reasons, e.g. generating small thumbnails for icons. Such tasks would cause LTFS tapes to be winding back and forth for seemingly no reason!
The great beauty of using LTFS for archiving data specifically, is that there is no vendor lock-in for either software or hardware. Any data written to one LTFS tape would be readable on another system, even when using a different vendor’s LTO drive and software. The data on the tape is not written in a proprietary format requiring a specific software tool, and is self-contained: the tape contains everything needed to browse contents and restore.
Why should P5 Archive support LTFS?
P5 Archive is one of the aforementioned software products that tracks what it writes to tapes via an ‘index’ – a database tracking each file archived. The index keeps track of which tape a file has been written to, across many tapes, and our customers appreciate having a single place to search when trying to locate data archived in the past. In this sense, P5 Archive doubles as both the mechanism by which data is physically written to tape, and a highly focussed MAM (media asset management) system for managing and locating everything that’s been written, across many tapes. This article explains this in more detail.
In addition, we control the tape library device, so many tapes can be managed, stored, loaded into drives and read/written in a completely automated fashion.
P5 Archive version 6.1 adds LTFS as a format that tapes can be written in. Users will still use the index as a means by which data is managed across many tapes, but the tapes themselves are being written in the vendor-neutral LTFS format. So our customers now get the benefit that their archive data is not written in a proprietary format, and written tapes may be read outside of P5 by having their on-tape filesystem read by third party software tools.
P5 Volume format or LTFS volume format?
Users of P5 Archive now have a choice when archiving to tape: use the ‘P5 volume format’ or the new ‘LTFS volume format’. The process of archiving remains the same, whichever tape format is chosen. Both single drives and libraries with many slots and multiple drives continue to be supported for both tape volume formats.
In summary, the main reason to switch to the LTFS format, or use it for a new archive, is because you want to:
- avoid vendor lock-in, be able to restore your archived data without P5, albeit one tape at a time
- have a workflow that requires tapes are sent elsewhere, away from the P5 installation, where data can be restored from the tapes. Again, one tape at a time
You’ll notice the common thread across those two points – without the P5 index providing a central database of everything stored across many tapes, the LTFS tapes become like very large USB thumb-drives. Each one must be mounted and browsed, to see what’s contained on the tape.
The table below shows the differences in use between the two:
| Feature | P5 Volume Format | LTFS Volume Format |
| Automatic spanning of large files across two tapes | yes | yes |
| Block-level simultaneous cloning across two tapes at once (requires two LTO drives) | yes | no |
| Writing to multiple drives in parallel (requires two LTO drives) | yes | no |
| Multiple jobs/streams simultaneously writing to one tape | yes | no |
| Importing tape from foreign P5 system | yes | no* |
| Verification of written data by checksum comparison | yes | yes |
| Verification of written data by re-reading of content from tape | yes | no |
| Manufacturer(Archiware)-independent restore of data | no | yes |
* whereby contents of tape imported into index – to be added in future release. Note that LTFS tapes can be read by any LTFS software/system.
In conclusion
So there we have it – P5 Archive can now write LTFS format tapes, with hardly any downsides as compared with our ‘original’ native volume format. Those with requirements to use LTFS, perhaps imposed by insurance requirements or company policy, can now do so while retaining all the benefits of a central index through which the entire archive can be accessed.
