We have a mixture of licenses with hyper-v hosts and bare metal servers. We're running the old backup format (non-gfs) on all of our servers.
Currently, our test restore process is very hands-on. Every 6 months we renew all of our trial licenses with our MSP360 rep, log in to our test server with each client's credentials one at a time, run a full restore, take screenshots of the desktop for evidence, and finally delete the files from the test server.
We would like to automate our test restorations. I've played around with the restore verification a little bit. There's a driver bug that prevents you from logging in as multiple users on the same computer so that rules out test verification on our test server. Plus, renewing trial licenses seems like we're using a clumsy process. I've also seen that we can run the test restore on our clients' servers, but these are domain controllers so I'd rather not use any resources on their production machines.
Is it possible to automate test restores on a recurring basis? If so, how do you do this and what are the best practices?
Hi Justin, have you reported the driver bug that you see with the restore verification feature to our tech support? Product-wise we're likely to go with fixing and improving our new built-in restore verification feature.
Justin, would be curious how your testing goes. Is it possible that you may do this with the New Format backups and with Restore Verification? We are running this on one machine but have not gone further than to view the Restore Verification daily backup email screen shot and once in a while manually doing a test restore to virtual machine. Although we have only done limited testing so far things seem to be working.
Alexander, we haven't tested it yet but I'm glad it may be working now.
My root question here is about test restoration best practices. I don't feel like my clumsy process of using a test server, renewing trial licenses, and manually logging in with each client is the best way to do test restores.
I'd like advice on automating test restorations. Are there any guides or articles for automating test restores with MSP360, or am I doing it the correct way already?
Are you sticking with the legacy format or do you plan to move to the new backup format? If new backup format, you can use Restore Verification if what you're after is image restore verification. If you want to test a file-level restore from a VM or Image, then you can set up a scheduled restore that restores a single common Windows file.
Or, as an alternative, or for file-level backups, you can create a dummy file in the same location on every computer and make sure it's backed up . That way you can test restores if you need by restoring this dummy file.
But I have not run into any customers doing this. I've probably asked before, but what is your major concern? Is it image/VM backup with a broken incremental chain, a corrupted backup file, something else? Is it only for image/VM or are you concerned about file backups too?
Also, the new backup format performs backup consistency checks every backup to ensure the backup sets are in backup storage and they are the correct ones. That's an additional check that is not performed by the legacy format. In cases, where you have a full backup and chained incremental backups, the new backup format avoids backup set integrity issues (like a missing incremental) by performing this check and escalating to a new full backup if it detects an issue.
I would also like to know how to do test restores. I am also logging into each customers account to do a restore. I would like to have a random file restore. Like one .doc one .jpg, etc.
I will be sticking with the old backup format for now. I tried the new format and it made my customers Wasabi bill skyrocket even with the bare minimum retention. I don't mind setting scheduled restores but I don't want to restore the same file every time. Right now we do a handful of random files with different extensions. But that is VERY time consuming. I am looking for a way to cut down on time.
If file, what's driving you to test a restore of a single file? Since the legacy format uses a different cloud object for each file, a test restore of one file, even a random one, only ensures that file can be restored. I'm not suggesting that there is going to be a problem restoring other files, but I am trying to understand the reason to run these test restores. With the new backup format, there is backup verification built-in. But with file backups, you may have an increase in storage, as you noted.
If you're running image backups, then the new backup format is far superior.
There is an API for the product, so you could try to automate in some way, programmatically.
I'm sticking with the legacy format for now for several reasons: 1) It doesn't support hybrid backups, and we have external drives deployed at each site for faster restores if needed. 2) The new format isn't available with VM servers, and we want all our clients to have the same backup configurations for easier management. 3) We only want 3 months' worth of backups, so GFS feels like overkill.
All of our backups are image-based, so I don't know if the dummy file idea would work.
My major concern driving the test restores is restoration failures. I worry that an msp360 update, bucket change, API issue, settings change, Windows update, etc could break a backup configuration. It helps me sleep at night knowing that the backups are bootable if needed. On top of that, industry best practices say to regularly test your backups.
A few points.
The new backup format supports VMs.
GFS is not required. If you want 90 days, set retention to 90 days and it will operate much like the legacy format.
The new format supports restore verification for image backups.
But you're correct that hybrid support is not available yet, so you'd need to run two backups for local and cloud.
Restore verification for image backup is great but I can't restore a singe file from an image backup for a customer so it isn't the best backup format for a lot of cases. We need restore verification of some kind for all backup formats.
You can restore perform file-level restores from Image Backups. That has always been supported. Having said that, it may not be the best format if you do not really need an image backup. If you're just backing up files, then the file backup format is best and it includes automatic backup consistency checks when using the new backup format. If a problem is found, a new full backup is run and you are alerted.
It's for Managed Backup, RMM, and Connect customers who want to know what's been released in the last few months. Also, we are giving away an Oculus / Meta Quest 2 VR headset to a lucky winner who attends the webinar.