Support for object system metadata

Registered by Alistair Coles on 2014-03-12

The original system metadata patch ( supported only account and container system metadata.

There are now patches in review that store middleware-generated metadata against objects, e.g.:
• on demand migration
• server side encryption

Object system metadata should not be stored in the x-object-meta- user metadata namespace because (a) there is a potential name conflict with arbitrarily user metadata and (b) system metadata in the x-object-meta- namespace will be lost if a user sends a POST request to the object.

The goal of this work is to enable system metadata to be persisted with objects in a similar way to user metadata, but with the ability to update individual items of system metadata independently when making a POST request, unlike user metadata which is replaced as a whole by a POST request.

Proposed approach:

Initially enable object system metadata only on PUT requests to support existing use cases:
• System metadata stored in .data file xattrs and copied unchanged to .meta files.
• Use the x-object-sysmeta- namespace already supported by proxy and reserved by gatekeeper middleware.
• Store x-object-sysmeta- headers in object file xattrs alongside other object metadata.
• Store object system metadata attributes as key: (value, timestamp) as per account and container metadata, enabling most-recent-wins rule to be applied to individual attributes in future.

 Follow up with enabling system metadata updates on POST:
• Allow for concurrent POST requests resulting in multiple .meta files with divergent sets of system metadata.
• When handling POST the object server (DiskFile) reads potentially multiple existing .meta files and merges system metadata to a unified set written to a new .meta file – the system metadata key: (value, timestamp) format allows most recent version of each system metadata key to be selected from the set of .meta files.
• When handling GET/HEAD similarly read and merge system metadata from potentially multiple .meta files.
• Only delete .meta files when their system metadata has been merged into a newer .meta file.

Detailed discussion on wiki page:

Blueprint information

Not started
Alistair Coles
Needs approval
Alistair Coles
Series goal:
Milestone target:

Related branches



Gerrit topic:,topic:bp/object-system-metadata,n,z

Addressed by:
    Enable object system metadata on PUTs

Addressed by:
    Enable object system metadata on PUTs

Gerrit topic:,topic:bug/1351323,n,z

Addressed by:
    Merging master into feature/ec branch


Work Items

This blueprint contains Public information 
Everyone can see this information.