By David Fox
Storing media assets, such as images, video and audio has specific requirements around the lifecycle of the asset.
Media assets are used in professional workflows as components when creating content.
A TV program will contain footage from many different video and audio files. Editing software enables the creative process of editing many assets together into a final piece of work. Such software typically requires very fast access to these assets so that this process is seamless.
In this article, we’ll look at common types of storage devices and assess their usefulness in the context of media assets.
Online and Offline Storage
We use various types of storage devices, including ‘online’ devices such as hard disk drives (HDDs) and solid-state disks (SSDs), through to ‘offline’ storage such as LTO tape, cloud storage, and optical storage.
Online storage is used for fast access to assets during the editing process. Offline storage is designed for the stage in the lifecycle of an asset where it has completed the production process and now needs to be retained for the long term. At this stage, the longevity of the storage is more important than the speed.
Online Storage
Let’s first consider some different types of online storage devices and see how HDD and SSD fit the requirements mentioned above.
HDDs and SSDs
A typical spinning 7200 RPM HDD will deliver around 80-160MB/second of read/write performance. An SSD will deliver 200-550MB/second. In professional storage systems, HDDs are configured into arrays which increase reliability and speed. Pulling data simultaneously from several HDDs can provide multiples of the speed of a single disk. In higher-end arrays, SSDs are also used as a cache to further increase the read/write performance experienced by the user.
With the price of SSDs falling fast, it’s becoming more common for pure SSD arrays to be used when handing high bit-rate video for editing, e.g. 4K and 8K.
Note that the method of connection between disks and the host computer also has a large impact on speed of access. An external disk with a USB2 interface will be limited to around 60MB/sec. Faster interfaces are required to access the benefits of faster disks.
Both HDD and SSD storage has a lifespan, limited by the physical moving parts in the case of HDDs and by the limited life of NAND flash chips used in SSDs. This isn’t a problem during a production phase, where failing disks in an array can be detected and replaced without data loss. It becomes an issue for long term storage however. Data left on an external HDD for a decade, may not be readable when connected back to a computer again.
Cloud Storage
In order to use cloud storage for online workflows, it is necessary to migrate the entire workflow into the cloud. Both the computer upon which the editing is being done and the storage where the media assets reside, can be run within the cloud infrastructure of a single provider. The users run remote-desktop type software to remotely access the screen of the computer in the cloud.
Offline Storage
When media assets have been incorporated into a final piece of work, be it a movie, TV or audio project, they no longer need to be stored online. The assets need to be retained for later use, but can be migrated away from more expensive online storage, to something better suited for archival.
LTO Tape
LTO tape is a popular archive storage format, a single tape can store 12TB (at the time or writing) and has excellent read and write performance. LTO boasts a roadmap with storage capacities doubling every 2-2.5 years.
However, reading and writing to tapes requires positioning/winding which greatly slows down the overall performance, making it unsuitable as online storage. The tape itself has almost no moving parts to wear out over time, versus more complex hard drive mechanisms. Manufacturers quote a 30 year lifespan for LTO tape, if storage requirements are met (not exposing to humidity or physical shock).
Optical Storage Formats
Optical professional storage formats, such as Sony’s Optical Disc Archive (ODA) consist of 11 Blu-ray like disks, combined in a single cartridge. The latest generation can store 5.5TB per cartridge, but doesn’t have the roadmap comparable to LTO for future storage capacities.
Cloud storage
Cloud storage increases in popularity for off-line asset storage. Providing bandwidth exists to push assets into the cloud, and the selected service has acceptable costs, usually charged in GB/month. Cloud vendors offer storage specifically suited to archive use.
Managing Offline Media Assets
Managing the migration of media assets from online to offline storage, requires careful tracking of what has been stored where. If the storage can be accessed as a filesystem, just like HDD or SSD, small numbers of assets can be copied to the storage and a simple spreadsheet or database used to track what is stored.
As things scale up however, tools that can manage both the indexing of what has been written and the physical handling of the media itself, become desirable. Archiware P5 Archive is one such tool which can both drive physical LTO tape hardware and cloud storage, manage many different LTO tapes, and keep a searchable index to allow location of assets within the archive.
Conclusion
Storing media files in a professional setting, choosing types of storage devices requires knowledge of the underlying workflows. While HDDs and tapes have been around for decades, cloud storage is making an increasing impact on the workflows of the media and entertainment industries. However you choose to store your media assets, it’s vital to have a software platform which can accomodate different types of storage into a seamless workflow.
