please add a --changed option to 'swift download'
'swift upload' has a --changed and a --skip-identical option. Both do not upload data if it already exists at the destination. --skip-identical compares the md5sum of the source with the etag of the destination. --changed just compares the mtime of the 2 files. --skip-identical is more accurate but not practical in cases where multiple terabytes of data have to be kept in sync. In that case all data in the source would have to be read each time a synchronization is triggered to produce the md5sum .
--skip-identical also has problems (https:/
'swift download' does not have a --changed option and the --skip-identical option has the same problems and limitations as 'swift upload' . please add a --changed option to 'swift download' that compares X-Object-Meta-Mtime with the mtime of the files in my local file system.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Dirk Petersen
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- New
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
I too have a need for this. Dirk is 100% correct that skip-identical is not practical for large containers. Something more efficient is definitely needed. While timestamp is great, just wanted to mention that in my case just file names would be sufficient as my application does not change files once they are written to object storage.
