By Dr. Marc M. Batschkus
Introduction
Above all, the top reason to start archiving is to free up space on production storage. After all, having an investment in storage expansion can even make the archive pay off financially.
Archive definition: Archiving is a migration that moves finalized files to archive storage. It can also be described as a primary copy of secondary data. For this reason, production storage doesn’t need to grow continuously.
Preserving media and making them findable and reusable in production is another benefit. Having an archive as a Single Source of Truth can save countless hours of searching for files. Independent of archiving video clips, digitized film, photos, sounds, or any kind of files, the Archive is the long-term repository where everything can be found.
Additionally, having a data archiving strategy supports the creation of new production from existing media. The value of existing media can be tremendous, and returning customers or recurring events can build on re-using it.
Today, building a digital archive is easier than ever. With Archive software like Archiware P5 Archive, setting it up takes only minutes.
Building a data archiving strategy and the best and most useful archive for your company takes more time and for good reasons. Many things need to be considered for a data archiving strategy and a project that will be used for many years or decades.
How to Create an Effective Data Archiving Strategy
Step #1: Start immediately
In the beginning, two things are important: start planning immediately (on paper) and start small with the implementation (e.g., with a single LTO drive or just one department). Since planning needs time, the best time to start is now.
Write down what you need to solve and who is involved. In addition, assess what you know about your data landscape. For example, how many files are created each day and need to be archived later. When starting immediately you also start collecting experience that will be invaluable on the way.
Step #2: Decide what to archive
Media assets come to mind first when thinking of preservation. Some other things contribute and make them reusable like project files, metadata, production notes, etc. When recreating a production, this additional information can be crucial. Think of everything that contributes to re-creating a previous production and see if you can include it.
A long-term perspective also needs to take into account that the software, OS, and hardware creating those files might not be available in the future. This makes storing older workstations, including their tools, sometimes essential. Such an outdated workstation is the quickest way to edit an old project with tools, plugins, and add-ons used at the time.
Codecs are a topic in its own right. They need to be looked at from two perspectives, ongoing production and long-term future-proofness. Apple´s announcement recently showed that some codecs would be obsolete soon.
Other vendors will follow with their announcements of deprecated codecs. Codecs that are future-proof (for now) are ProRes, AVCIntra, AVCUltra, etc.
It will not always be possible but will be the best solution to keep the complete workflow in the same codec from editing to archiving.
Step #3: Get people involved
The next step is to get people involved and demonstrate future benefits (e.g., with a software demo) so that everyone can see the picture of the immediate solution. Involvement and input are important for the data archiving strategy for two reasons: experience from different perspectives and acceptance once the project is implemented.
Here are some relevant questions that are important:
- Who will be archiving?
- Who will be searching, and for what purpose?
- What experience does the person have using search terms and criteria?
Writing use cases is a helpful way to address these questions and find suitable solutions. Use cases are like little stories that go through all steps of future use. For example, “In two years a new editor needs to create … and therefore needs clips fulfilling certain criteria like A, B, C. (…)” Be specific and elaborate. This can save countless hours of later adjustment. Include your company principles in the planning process.
Step #4: Build a metadata schema
Keep metadata modest and reasonable. Building a metadata schema that is unique to each company and supports later search is a core task.
What technical and descriptive metadata is available, and what is needed? Who puts in the metadata, and who controls accuracy and consistency?
A metadata schema is only as good as how consistently the existing fields are filled out. The best schema is almost useless if only a fraction of the fields are filled out. The metadata schema becomes the key to the archive. When the one searching for files has no knowledge or memory of the specific production, metadata is the only way to find files.
P5 Archive offers customisable metadata fields and menus to build a metadata schema in minutes. Additionally, combined searching and media previews give P5 Archive a mini-MAM feature set.
Step #5: Keep digitisation as a separate project
The digitisation of analog or historic media like film, photos and audio tape is a project in its own right. Film scanners are needed to digitise existing film.
With other historical media, second-hand devices are sometimes the only option to read the media. Depending on the amount of historic media, digitisation alone can take years to complete. For this reason, it makes sense to keep the project separate from the archive as such because it could delay everything.
A good approach is to start archiving the most recent finalised files. Optimising this workflow is important before a digitisation workflow can be faded in.
Step #6: Decide on data security and archive media
What storage is best used for the archive, disk, tape or cloud? Each has its features and benefits.
For the long-term perspective, tape and Cloud storage stand out as more efficient and durable. Next, you have to decide on the level of data security and redundancy for the archive.
A professional archive should always have two identical sets of archive media (like LTO tapes). One set should be stored offsite for maximum security.
Do you have an existing location, or do you need to find one? Who will put the media offsite, and when?
P5 Archive has a built-in feature to clone tapes (or disk storage) at the time of archiving. Another option to solve this is to use Cloud storage as a disaster recovery copy.
Access and restore rights and privileges are also part of this topic. Who needs to see what part of the archive, who is allowed to restore to what storage, etc.?
Step #7: Document, document, document
Documenting the planning process and all decisions is crucial. Later, this becomes the roadmap and reference for the project. Writing down what decisions were made and why saves a lot of time during further discussions and implementation.
Documentation and checklists also play an important role in operation if tasks need to be taken over by new employees.
Step #8: Decide on the specific workflow
List each step of the workflow and answer all relevant questions:
- Who archives
- From which workstation
- Which files
- To which archive storage
- Using which metadata?
Which hardware and software are involved, and what connections are used?
Additionally, planning for maintenance, updates, expansion and operations is necessary for long-term projects.
Conclusion
Building a data archiving strategyand implementing it is easier than ever. The archive has value as a file memory of the company, foundation for future production, returning customers and monetisation of media. Saving investment in production storage expansion alone can make the archive pay off financially. Check out Archiware P5 Archive and start your archive now.
