User Tools

Site Tools


Why is the transfer rate of my backup job so inconsistent

The transfer rate of my backup job is very inconsistent. During a backup, the transfer rate can goes up to 50+ Mbit/s, then it will drop down to 0 bit/s for minutes before going up again. Why is that? Why would the transfer rate be at 0 bit/s during a backup?

When the transfer rate is at 0 bit/s, the backup process is:

  1. awaiting for the Volume Shadow Copy to be created, or
  2. at the initial stage, when the client agent is reading the backup source, or
  3. 'combining' or 'splitting' the data files into 'block' files to be transferred to the backup destination.

Transfer rate, specifically refers to the transfer speed of the individual block files when they are being 'transferred'.

More details on the point 3:
Individual backup files are 'combined' into 'block' file before they are transferred to the backup destination (e.g. local destination, FTP/SFTP, cloud storage destination), reducing the overhead during the upload, while also reduces the number of files in the backup destination.

Files that are 16 MB or smaller are 'combined' into a single 16 MB 'block' file before they are transferred to the backup destination

Files that are larger than 16 MB will be 'split up' into multiple 32 MB 'block' files when transferred to the backup destination.

For example, a file with size of 100 MB would be split up into 4 'block' files:

000000.bak - 32 MB 000000_000001.bak - 32 MB 000000_000002.bak - 32 MB 000000_000003.bak - 4 MB

Data uploaded by the backup client to the backup destination are stored in the following format:

%backup_destination%\%backupset_id%\blocks\YYYY-MM-DD-hh-mm-ss\0\000000.bak %backup_destination%\%backupset_id%\blocks\YYYY-MM-DD-hh-mm-ss\0\000000_000001.bak %backup_destination%\%backupset_id%\blocks\YYYY-MM-DD-hh-mm-ss\0\000000_000002.bak %backup_destination%\%backupset_id%\blocks\YYYY-MM-DD-hh-mm-ss\0\000000_000003.bak

Therefore, when you see the transfer rate of the backup job drops to 0 bit/s, it does not necessarily mean that the backup job had halt / paused. It maybe the case where the client agent is in the process of combining the backup data into block files.

Note: Since the combine / splitting process requires CPU processing power and memory of the client computer. Further to the bandwidth available, the backup speed / transfer rate also depends on the hardware specification of the client computer.

public/faq/why_transfer_rate_backup_job_is_inconsistent.txt · Last modified: 2019/09/26 13:43 by ronnie.chan