Don't write new incremental set if nothing changed
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Duplicity |
New
|
Undecided
|
Unassigned |
Bug Description
I have a script that calls Duplicity separately on smaller sets of data so that important ones get backed up first. Some of those early sets rarely change, but even if nothing changes, Duplicity uploads a new incremental set of files. This not only wastes time when backing up, but it causes a LOT of time to be wasted when doing a restore.
For example, a few days ago I needed to restore three small files from a small backup set that hadn't changed in years. But to do so, Duplicity had to download probably over a hundred incremental sets in which nothing changed. This took a LONG time! It's especially slow since, judging from Duplicity's verbose output, it downloads one file at a time--the latency between commands adds up quickly (I'm ignorant about SFTP: is there no way to request more than one file at a time?). Restoring less than 20 KB of data took over 10 minutes because of all the empty incremental sets that had to be downloaded.
I realize that it's important to indicate when incremental backups were made, even if nothing changed, so I think Duplicity should store empty incremental sets in a different way. For example, it could store a single file in the remote directory that lists all the times incremental backups were made in which nothing changed. When making a new, unchanged incremental backup, Duplicity could simply append the date and time to the file and replace the copy on the server. This would save so much time it's not even funny! :)
In my case, it would be ok to not know about a run where there is no change. So an option which let duplicity simply not write any thing if there is no change would be great.