• Florian
    0
    Hi,
    I am not able to use Managed Backup with Amazon Glacier. I have added an Amazon Glacier account to managed backup, but I am not able to create "bucket" or add created vaults to the managed backup. If I try to create a "bucket" I get the following error:
    Unable to create bucket 'test2018'.
    The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
    
    The name of the bucket would be test2018.

    What am I doing wrong?
    Florian
  • Matt
    91
    For Managed Backup Software Glacier backups are performed via lifecycle rules, so you don't need to use Vaults, just S3 buckets..
    You can find more info here: https://mspbackups.com/Admin/Help.aspx?c=Contents/help_move_to_Glacier.html
  • David Gugick
    118
    As Matt said, backing up directly to Glacier is not supported in Managed Backup for many reasons - the most important ones being that Glacier is designed for long term storage, is slow to access and see an inventory of files, but can be leveraged through S3 Lifecycle Policies. What you'll be doing to leverage Glacier is to back up to S3 (you can use S3-Standard, S3-Infrequent Access, or S3-One Zone IA) and then create a lifecycle policy (in MBS or directly in AWS) that moves data from S3 to Glacier after X days (can be 1 day if that's what you truly need). You'll get fast inventories of files (since metadata exists in S3 for all files tiered to Glacier) and you'll see reduced storage costs - assuming you plan to leave the data in Glacier for a minimum of 90 days. But, you need to be aware that Glacier restores have an additional cost and requires 3-5 hours by default for AWS to get the data ready for restore. So, it's not a good solution for most customers if you know you'll need to restore files frequently or need them restored quickly.
  • Florian
    0
    Thank you for your information,
    that's frustrating, because it means that we need to rewrite our backend management application, because it bases on glacier. :sad:
    We decided to use glacier because we only provide hybrid backup solution to our customers. Retention via Cloud backup is only used in case of fire, water or property damage. Using S3 also means, that we need to adapt our fees.

    Thanks,
    Florian
  • David Gugick
    118
    I believe Amazon allows you to transition from S3 to Glacier after 0 days. I do not know what the actual delay is in transition (if it's immediate or has some delay like up to one day).

    In addition there is some metadata that is stored in S3 and Glacier. This may be important to you if you are storing a large number of very small files. You are correct that you probably need to examine the fees to see how they change for your particular data sets.

    From: https://docs.aws.amazon.com/AmazonS3/latest/dev/lifecycle-transition-general-considerations.html

    Storage overhead charges – When you transition objects to the GLACIER storage class, a fixed amount of storage is added to each object to accommodate metadata for managing the object.

    For each object archived to Amazon Glacier, Amazon S3 uses 8 KB of storage for the name of the object and other metadata. Amazon S3 stores this metadata so that you can get a real-time list of your archived objects by using the Amazon S3 API. For more information, see Get Bucket (List Objects). You are charged standard Amazon S3 rates for this additional storage.

    For each archived object, Amazon Glacier adds 32 KB of storage for index and related metadata. This extra data is necessary to identify and restore your object. You are charged Amazon Glacier rates for this additional storage.
  • Florian
    0
    Thank you for your explanation. We need to evaluate, but we are not sure if it make sense to us, that we first backup to s3 and than move the backup to glacier because it complicates our workflow.
    But first we need to check the costs if they are practicable with our flat rates.
    Personally I do not understand if in standalone mode of Cloudberry, backup to Glacier is possible but not in managed backup.

    Thanks,
    Florian
  • David Gugick
    118
    To be honest, backing up to Glacier is not requested nor used by most customers in Managed Backup. Transitioning the data is easy and automated and has many benefits over moving data directly to Glacier:
    1. Glacier uses a different API from Amazon from the S3 API
    2. Getting a real-time file inventory from a Glacier vault is not possible. This makes the user experience suffer since many customers expect immediate access to the vaults and the data they contain. Inventory in Vaults is only updated by AWS once a day and it may take up to a day before the data is ready for retrieval
    3. Restores take an average of 3-5 hours to prepare unless you spend more for expedited retrieval
    4. Requests and downloads are expensive from Glacier, so most customers are happy to leave the data in S3 for a short time during which restores are more likely to occur. For those customers who do not expect restores (as in your case), you can move the data to the S3 Glacier storage class the same day (automatically), while having real-time file inventories available for restores.
    5. Glacier has a 90 day retention requirement. That means if you're deleting backups of files from before the 90 days, you are still paying for the full 90. S3 - Standard has no retention requirement and objects can be removed immediately with average monthly use as the billing metric.

    I understand that Glacier is the second backup for your business and the likelihood that you need to restore from Glacier is low when compared to a local restore. But I think you'll find that using a lifecycle policy to move the S3 data immediately to the Glacier storage class provides benefits without much of a change in costs. Depending on your region, there are lower-cost storage alternatives you may want to consider (like Backblaze B2 and Wasabi). But I think the S3 --> Glacier Storage Class will work for you.

    Let us know how it goes.
bold
italic
underline
strike
code
quote
ulist
image
url
mention
reveal
youtube
tweet
Add a Comment