I have a small setup with VMware 7.0U3 and Pure 3.0.1
Everything works but I'm struggling to fit my backups into a nightly window without them spilling over into the morning. I have 7 VMs (total about ~1.2TB) that i am backing up to a Pure repo that is backed by a locally-attached 4TB SATA SSD. When I run speed tests on this disk I get read/write speeds of approx 150-200MB / sec. But in Pure, the verification seems to slog along at around 10MB / sec. Thus, my backups take hours and hours to complete even for very small deltas (just a few MB)
Is there anything I can do to make this faster? Or do I have to just give up and turn off verification and keep fingers crossed tightly? 🤞
Thank you
Hi.
Are the VMs using over-provisioned thin disks (e.g. 1TB disk where only a portion is actually allocated)?
If so, you can save a lot of time by using the 'Quick Verify' option, allowing Pure to skip verifying the parts of the disk that vSphere reports as not allocated. Otherwise, Pure will be thorough and read the disk till the end.
Otherwise, how much throughput do you get during backup? Is it as slow as verify or does it run significantly faster?
Thank you @marijan
The forum has been down so I was not able to reply.
The quick verify option was already enabled in my case. Some of the VMs are thin-provisioned, and some are thick. But, it made little difference. The backup throughput seems to be quite good, the incremental backups complete within 3-5 minutes. But, the verification takes hours. Not sure why.
I had to disable the verification for now, otherwise the backups take over 20 hours to complete. I don't know how to really debug why the thruput is so slow during the verification. When I benchmark the underlying SSD, I get 100MB/sec+ read performance. I wish there was a way I could schedule a separate "verification only" job e.g. and make this happen on Friday night so it has the whole weekend to complete.
I didn't ask you about your Pure setup. Are you using Pure as a VA and accessing VM data using hotadd or are you using network transfer, which is always slower?
Also, backup and verification is performed in two separate phases. First all the VMs get backed up and afterwards the verifications start. In other words, even if verification is slow and does not complete in the allocated window, at least backups should finish every time. And yes, you are free to schedule more than one backup/verification window so you could schedule daily backups without verification for Mon-Fri and have a separate schedule for backup+verify to start e.g. on Saturday and last longer to make sure everything is checked.
Btw., do you see any warnings or errors in your logs or is everything ok? Logs are found in <pure folder>/log/vmsn.log[.xxx]
Thanks @marijan
Yes I am using Pure as a VA running on the same host as my VMs, so hotadd is definitely being used. I can confirm this happening from looking at the logs at the ESXi console.
I will experiment with a separate backup+verify job to begin Sat morning with a 48 hour window.
Also, since I turned off verification, I don't see anything weird in my logs right now. I checked with
grep -Ei '(warning|error)' /usr/local/pure/vmsn/log/vmsn.log
So, I will enable a new job for Sat that has the verification enabled and then report back, thank you again.
