Purging files halts all other jobs
  • Gustavo Moraes
    0
    Hello,
    I have multiple jobs running on a server (using S3 as repository) and every time one of the jobs goes into "purging files", it halts all other jobs until the purging is done (usually 25-30min).
    is this normal procedure?

    Here is what the log shows
    2022-09-28 10:08:53,901 [PL] [1] NOTICE - Creating purge queue started. Purge conditions: version count: 10, delete topurge elements
    2022-09-28 10:34:14,163 [PL] [1] NOTICE - SQL query 'SELECT id, local_path FROM (select local_path, cloud_files.id, COUNT(*) as cnt, MAX(cloud_file_versions.date_modified_utc) as last_modified_date from cloud_files inner join cloud_file_versions on cloud_files.id = cloud_file_versions.file_id where destination_id = ? and not exists (select 1 from ransomware_file_versions rfv where rfv.file_id = cloud_files.id) group by cloud_files.id) WHERE cnt>1 '. Parameters: '[param_1]='1'' takes in total: 00:25:20
    2022-09-28 10:34:47,782 [PL] [1] INFO - Files to purge with versions found: 2761281
    2022-09-28 10:35:16,906 [PL] [1] INFO - Folder markers to purge found: 0
    2022-09-28 10:35:16,906 [PL] [1] NOTICE - Purge version search time: 00:26:23. Files checked for this plan: 16883. Files for purge found: 0
    Gustavo Moraes Options
  • David Gugick
    118
    ↪Gustavo Moraes
    What do mean, “halts other jobs”? Do you mean other scheduled backup plans do run run on schedule until the purge is complete, or do you mean other plans that are executing in parallel stop processing while the purge is working?

    Also, please post your version.

    Let me add, that based on the information I see in the log, that you’re purging / checking 2.7 million files. That is going to take a lot of time, it’s possible that the agent is tied up with all the updating. You may want to consider moving to the new backup format which can do these purges very quickly if this is a common situation where you’re removing / checking that many files in the cloud. But give me an answer to my questions above first and we’ll take it from there. Thanks.
    David Gugick Options
  • Gustavo Moraes
    0
    Plans that are executing in parallel halts until purge is working.
    This is what I see in their logs while it waits for the other plan's purge to complete

    [CL] [10] WARN - Generating chunks is idle for 15 minutes.....
    Gustavo Moraes Options
  • David Gugick
    118
    ↪Gustavo Moraes
    you’re going to have to open a support case likely to get firm resolution, but it sounds like the PC where you’re running backup is tied up with all the processing. If the plans are pausing, it may be because of all the updating and checking that’s going on in the local repository database for the 2.7 million files. The new backup format would address that particular situation. Having said that it what’s the issue with the temporary pause on running plans while the long purge operation runs? Is it pushing you outside your normal operating times? Are the plans ultimately failing because of the pause? Something else?

    See the additional questions I put in my original reply which I was probably replying to while you were replying to me.
    David Gugick Options
  • Gustavo Moraes
    0
    pushing outside of normal operating times. I'll open a ticket.
    My cbbackup.db is over 70GB which it has been for some time but this issue just started a couple of days ago.
    Gustavo Moraes Options
  • David Gugick
    118
    ↪Gustavo Moraes
    The issue of the purge operation taking that long? Or the purge operation temporarily pausing other running plans? I am just wondering if the purge is now taking much longer than previous and that's why you are noticing the issue - whereas if it ran in 5 minutes maybe you would not have noticed.

    • What version of the product are you running?
    • Were there any changes in plan scheduling prior to noticing this issue?
    • Were there any major changes in the data being backed up - a lot of files being added or moved between folders?
    • Did you recently upgrade versions? If so, from what to what?
    • Any new processes, applications, or services running on that server that may be using CPU / Disk and causing contention with the purge operations?
    • Anything else you can think of?
    David Gugick Options
bold
italic
underline
strike
code
quote
ulist
image
url
mention
reveal
youtube
tweet
Add a Comment