Don't write new incremental set if nothing changed

Bug #768481 reported by Adam Porter
58
This bug affects 11 people
Affects Status Importance Assigned to Milestone
Duplicity
New
Undecided
Unassigned

Bug Description

I have a script that calls Duplicity separately on smaller sets of data so that important ones get backed up first. Some of those early sets rarely change, but even if nothing changes, Duplicity uploads a new incremental set of files. This not only wastes time when backing up, but it causes a LOT of time to be wasted when doing a restore.

For example, a few days ago I needed to restore three small files from a small backup set that hadn't changed in years. But to do so, Duplicity had to download probably over a hundred incremental sets in which nothing changed. This took a LONG time! It's especially slow since, judging from Duplicity's verbose output, it downloads one file at a time--the latency between commands adds up quickly (I'm ignorant about SFTP: is there no way to request more than one file at a time?). Restoring less than 20 KB of data took over 10 minutes because of all the empty incremental sets that had to be downloaded.

I realize that it's important to indicate when incremental backups were made, even if nothing changed, so I think Duplicity should store empty incremental sets in a different way. For example, it could store a single file in the remote directory that lists all the times incremental backups were made in which nothing changed. When making a new, unchanged incremental backup, Duplicity could simply append the date and time to the file and replace the copy on the server. This would save so much time it's not even funny! :)

Revision history for this message
den (space-trip) wrote :

In my case, it would be ok to not know about a run where there is no change. So an option which let duplicity simply not write any thing if there is no change would be great.

Revision history for this message
l0co (l0co) wrote :

A workaround with --dry-run and checking DeltaEntries line:

CHANGES=`duplicity ---dry-run [...] | grep DeltaEntries | sed "s/DeltaEntries //"`
if [ "$CHANGES" == "0" ]; then
  echo "No changes detected, skipping"
else
  duplicity --progress [...]
fi

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.