How to organise backup for a large number of VMs
Just wanted to share an interesting question we got through our support - "Hi. We are testing Pure and were wondering how best to organise backups (we have around 300 VMs). Do we create a single job for all of them or is it better to split them into separate jobs?"
Well, the short (and cheeky) answer is - none of the above.
Archiware Pure does not organise backups using traditional job configuration, opting instead to use Backup windows. Backup windows are time slots, typically outside your workload peak times, during which you allow Pure to perform backups (e.g. during night or lunch break). While a Backup window lasts, Pure will try to backup all the VMs that have been assigned to that particular BW and will prioritise VMs that have not been backed up for the longest. When a Backup window is closed, Pure will not start any further backups (it will finish those backups that have started already). If any VMs could not be backed up, they will have the highest priority in their next Backup window.
The precise processing order is this:
1. normal VM backups, starting with the ones that have not been backed up for the longest
2. backup of a vCenter VM (if present and configured for backup)
3. normal VM verification (if configured)
4. vCenter VM verification (if present and configured)
Steps 1. and 3. can process multiple VMs in parallel and across different Backup volumes. Processing of a vCenter VM has to be isolated from other tasks. If you find that waiting for a vCenter backup introduces too much delay in your schedule (because instead of waiting for a backup of that single VM Pure could be verifying multiple normal VMs in parallel), you can simply create a separate Backup window for vCenter VM only.
Because backups do not interrupt VM usage apart from a decrease in performance due to the increased I/O load on their hosts, Backup windows can be scheduled as ofter and be configured to last as long as you need. This way you do not have to know in advance how long each VM backup will last (in fact, backup times will vary depending on the amount of data changed) and how many VMs you can assign to a job for it to still finish with success.
That being said, when you have 300 VMs, you can expect the initial full backups to last significantly longer than the subsequent incremental ones. For this, you may wish to initially configure larger Backup windows or even start some backups manually. You can further speed things up by increasing the number of parallel backups or skipping backup verification initially, postponing it for later.
I hope this description proves useful. And as always, feel free to comment or ask questions.