Support for append-only Swift objects

Registered by Constantine Peresypkin on 2012-08-24

Support for append-only objects.
This is just an optimization for PUT->GET->PUT or PUT->GET->DELETE->PUT for existing objects.
Instead of sending a new object each time to overwrite the old one, the end-user can just send the difference in HTTP request.

Current flow for PUT is as follows:
- create temp file
- write user data into temp file
- check temp file for consistency (if possible)
- rename temp file to a timestamp
- delete old file (with old timestamp)
- update file metadata (in xattrs)

Proposed flow for append:
- lock the original file for append
- write user data into file
- check file for consistency (if possible)
- unlock file
- rename old file to new timestamp
- update revision data (in xattrs)
- update file metadata (in xattrs)

Concurrent append will be impossible (but it's not needed in most cases)
All replication/async update logic will stay the same.
The end result will be no different from PUT on existing file.

But, there will be some logic changes in object server behavior:
- file size have to be stored and retrieved from metadata only
- if file is actually bigger than its size (from metadata) it should be left alone and not quarantined
- new xattr entity have to be added to store the file revisions (should be stored in extents, not in inode)
- revision data should store: revision number, size and md5 state (maybe also the timestamp)
- md5 state should be clawed out from hashlib (not that hard)
- append should be embedded in POST and not in PUT request

Blueprint information

Status:
Started
Approver:
None
Priority:
Undefined
Drafter:
Constantine Peresypkin
Direction:
Needs approval
Assignee:
Constantine Peresypkin
Definition:
New
Series goal:
None
Implementation:
Started
Milestone target:
None
Started by
Constantine Peresypkin on 2012-08-25

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/object-append,n,z

Addressed by: https://review.openstack.org/11969
    initial implementation of blueprint object-append

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.