What is the best way to archive video footage and media files?
By Dr. Marc M. Batschkus
1. What Is a Media Archive System?
A media archive system (MAS) is a collection of software and hardware designed to store and manage large amounts of digital data. They’re often used by companies, media productions, media agencies, marketing departments, libraries, museums, and archives to preserve documents, photos, music, movies, and other forms of media. Additionally, a media archive system can contain any kind of files that need to be stored for long-term like financial data, business transactions, contracts etc.
2. Why Should I Build an Archival System?
If you’re interested in preserving your digital content and files for future reuse, reference and monetisation, then building an archive system is an important step. Many benefits and improvements can result from an archive that keeps all projects, images, photos and videos. Even small and medium businesses should consider setting up one. Often, the existing storage solutions are not optimized or scaled for holding media and video content. For this reason, departmental installs and departmental archives are a good solution for preserving existing media content and making it easily accessible where it is actually needed (as in marketing). Existing content, such as video clips and photos can serve as resources and can be regrouped and edited to create new content, catalogs, marketing campaigns and social media. This can speed up the process of creating content and save resources. Let’s have a closer look at the benefits and details.
An archive system that contains all files and projects that have already been created serves as the one place to look for existing files. This is also called a “single source of truth”, since there is only one place to look for files. Marketing campaigns can be setup quicker when all existing photos and videos are in the archive. Any project can start with a search of existing media and assets. The archive becomes the central resource for media and files that have been produced already. This reduces search times, increases efficiency in production, simplifies referencing files of returning customers and preserves the files for the long term.
Benefits of an archive system:
- Save space on costly production storage
- Save cost for storage when using storage optimised for archiving
- Reduce search time for existing files and media
- Serving returning customers with previous files and projects
- Simplify reuse, referencing and monetisation of existing files
- Preserve media and files for the long-term
3. Definition of Backup vs Archive
An important distinction between Backup and Archive exists and needs to be noticed.
The Backup is a duplication or replication of files used for ongoing daily production.
The Archive is a migration of finalised files and projects to a long-term archive storage.
As a consequence both are needed to protect ongoing production with one and finalised projects with the other.
4. Decide on metadata
Metadata is data about data a description of a file or its contents. This can be either technical metadata, like the resolution of an image or descriptive like what is seen in a video or the location it was taken at. An archive system offers the option of building a metadata schema, a set of relevant metadata that can be browsed and searched to find files that fulfil specific criteria. In theory, having as many metadata fields as possible would be good. In reality, that is limited by the consistency of the available metadata for each file in the archive. Only if metadata fields are filled for every file in an archive, they deliver complete search results. If fields are left blank, a search can not deliver these files. As a result, it makes sense to have only modest metadata: as much as possible but as few as can be filled with every file. Most of the time this leads to 3-6 metadata fields (e.g. 4 technical and one descriptive). Archiware P5 Archive lets you create an individual metadata schema with fields and menus that can contain either technical or descriptive metadata.
The filename is metadata
The filename is extremely important and can already contain part of the metadata. The filename also helps when a file gets misplaced or handed over to a 3rd party. A robust, smart and standardised naming scheme or naming convention avoids mistakes and saves time when working with files.
To learn more, see this article.
5. Head of metadata
Since metadata are such a core piece of the archive, they need to be thoroughly thought through and responsibly adjusted. Deciding anything around metadata is a responsible position since this impacts the usability and usefulness of the entire archive system. For this reason, it makes sense to have a person as “head of metadata” who is a responsible and considerate person. The value of the archive system relies on the decisions of this person and he/she needs to be able to check on the consistent use of the metadata schema by all involved.
To learn more about metadata see this article.
6. Video and media archiving software
Video archiving software keeps track of what files is stored where with what metadata and in how many versions. Archiware P5 Archive additionally offers specific MAM like features to simplify the process of finding and restoring files:
- Browser interface accessible from anywhere
- Supports disk, tape and cloud storage
- Runs on Mac, Win, Linux, QNAP, Synology, NetGear and FreeNAS/TrueNAS
- Customisable metadata fields and menus for individual metadata schema
- Combined searching for filenames, and medatadata
- Thumbnails of still images
- Proxy clips of video files
- Maximum security through tape cloning
7. Make the Media Archive available in the company
When building an archive system, the value of it manifests itself with the number of people using it. This way, efficiency of production increases, the cost of recreating files is eliminated and new projects can easily build on existing ones. Therefore, access to the archive catalog should be given to all relevant persons in the company. The privilege to restore files can be separated from browsing and searching for existing files. P5 Archive offers detailed options for login areas and user privileges.
8. Start as early as possible to build an archive (even if small)
Building a media archive involves many steps and usually multiple persons. Start soon to build experience and gather perspectives of different positions and responsibilities within a company. Starting it as a small project is better and more beneficial than planning for an extended time to build the perfect do-all be-all solution. Even when planning a bigger archive with a tape library of a considerable size, it might be best to start out with a single LTO tape drive. This way, gathering experiences with the various aspects involved starts sooner and leads to improvements that can flow into the expanded system. Finding out how much data has not been accessed in months or years and is blocking space on production storage can be an eye-opening discovery. That is the reason why archiving usually pays off financially relatively soon by saving production storage expansion.
9. Decide on the storage medium (while staying flexible)
Especially when the data set that accumulates over time is really big, cost of storage media becomes an important factor. Learn more about relevant factors and costs for LTO tape and cloud in this article.
Staying flexible and having options to adjust and change your storage strategy at a later point is an important factor. This allows accommodating price changes, policy modifications and security requirements after the archive has been built. Archiware’s P5 Archive offers a unique add-on, the P5 Data Mover, that can move/copy archived files between disk, LTO tape and cloud storage at any time. This makes the archive future-proof and flexible. Learn more about this feature here.
To learn more about ways to archive video files, see this article.
When starting out to build an archive, the growth rate of the data moving into it might not be clear. Experience shows that most of the time data grow faster than expected. This leads to two consequences:
- Anticipate scaling and expansion of the archive in time
- LTO tape becomes even more attractive the more data is being archived due to its low price per TB (starting at about 8 USD/EUR per TB). Additionally, scaling an LTO archive is easy because it just needs additional tapes.
10. Stay flexible concerning the OS
Running on most operating systems is one of the unique features of P5 Archive. This allows you to start out on one platform and easily migrate the archive later to any other OS platform like macOS, Windows, Linux, Synology, QNAP, Netgear, and others. This flexibility can be used for scaling, optimising efficiency or cost.
