Storage Policies

Registered by John Dickinson on 2013-11-20

1) Choose the subset of hardware to use
2) Choose the way data is stored across that subset
3) Optimize the way Swift talks to individual storage volumes

Blueprint information

Status:
Complete
Approver:
None
Priority:
High
Drafter:
None
Direction:
Approved
Assignee:
None
Definition:
Approved
Series goal:
Accepted for juno
Implementation:
Implemented
Milestone target:
milestone icon 2.0.0
Started by
John Dickinson on 2013-11-20
Completed by
John Dickinson on 2014-06-30

Related branches

Sprints

Whiteboard

Historical notes in https://blueprints.launchpad.net/swift/+spec/object-replication-strategies

Gerrit topic: https://review.openstack.org/#q,topic:bp/storage-policies,n,z

Addressed by: https://review.openstack.org/61585
    Storage Policy / DiskFile Interface Support

Addressed by: https://review.openstack.org/64461
    Add Storage Policy Support to the Auditor

Addressed by: https://review.openstack.org/52194
    Add storage policy support for the Replicator

Addressed by: https://review.openstack.org/64737
    Add Storage Policy Support to the Updater

Addressed by: https://review.openstack.org/65329
    Add Storage Policy Support to ssync

Addressed by: https://review.openstack.org/65347
    Add Storage Policy Support to ssync

Addressed by: https://review.openstack.org/65517
    Add Storage Policy Support to list_endpoints

Gerrit topic: https://review.openstack.org/#q,topic:bp/ec-proxy-work,n,z

Addressed by: https://review.openstack.org/69700
    Add Storage Policy Support for /Info

Addressed by: https://review.openstack.org/70265
    Replace Policy Index with Policy Name in Response Headers

(we need python-swiftclient updated with CLI support as well, right? swift -p fish stat? or is it on a separate blueprint?)

Addressed by: https://review.openstack.org/73747
    Add Storage Policy Support to Account HEAD

Addressed by: https://review.openstack.org/72535
    Store per-object policy index in container

Addressed by: https://review.openstack.org/71703
    Send policy index to container on sync update

Addressed by: https://review.openstack.org/71702
    Send policy index on async update

Addressed by: https://review.openstack.org/71704
    Store policy index in container_stat table

Addressed by: https://review.openstack.org/72536
    Fix changing of storage policy index

Addressed by: https://review.openstack.org/79731
    Prevent object with storage policy mismatch from being updated in a container

Gerrit topic: https://review.openstack.org/#q,topic:container-put-object-409,n,z

Addressed by: https://review.openstack.org/82590
    Fix PUT to existing container w/o specific storage policy

Addressed by: https://review.openstack.org/83252
    Move Storage Policy validation from parser to class

Addressed by: https://review.openstack.org/83840
    Fix issue with account broker and pre-SPI databases

Addressed by: https://review.openstack.org/85824
    Add Storage Policy Documentation

Addressed by: https://review.openstack.org/86469
    Add Storage Policy Support to Container Sync

Addressed by: https://review.openstack.org/87001
    Minor refactor of storage policy helper functions

Addressed by: https://review.openstack.org/87045
    Slight modification to the object updater sweep code

Addressed by: https://review.openstack.org/87387
    Add Storage Policy Support to Recon Middleware

Gerrit topic: https://review.openstack.org/#q,topic:container-obj-put-409,n,z

Addressed by: https://review.openstack.org/87424
    Super small code cleanup in Updater

Addressed by: https://review.openstack.org/88079
    Quick fix to Storage Policy Support for List Endpoints

Addressed by: https://review.openstack.org/88098
    Add Storage Policy support to the Account Reaper

Gerrit topic: https://review.openstack.org/#q,topic:container-enqueue,n,z

Addressed by: https://review.openstack.org/89747
    Minor policy test cleanup

Addressed by: https://review.openstack.org/91685
    Add probetest for container sync

Addressed by: https://review.openstack.org/91732
    Add probe tests for Account Reaper

Addressed by: https://review.openstack.org/93186
    Update storage_policy_index in expiring_objects container

Addressed by: https://review.openstack.org/96026
    Add Storage Policy Documentation

Addressed by: https://review.openstack.org/96027
    Add Storage Policy Support

Gerrit topic: https://review.openstack.org/#q,topic:shadow-listings,n,z

Addressed by: https://review.openstack.org/96028
    Update FakeLogger with better log level handling

Addressed by: https://review.openstack.org/96029
    Expose Backend container info on deleted containers.

Addressed by: https://review.openstack.org/96030
    Add Storage Policy support to Object Server

Addressed by: https://review.openstack.org/96031
    Add Storage Policy support to Object Updates

Addressed by: https://review.openstack.org/96032
    Add Storage Policy Support to the Auditor

Addressed by: https://review.openstack.org/96033
    Add storage policy support for the Replicator

Addressed by: https://review.openstack.org/96034
    Add Storage Policy Support to ssync

Addressed by: https://review.openstack.org/96035
    Put X-Timestamp in object 404 responses

Addressed by: https://review.openstack.org/96036
    Extend interface on InternalClient

Addressed by: https://review.openstack.org/96037
    Extend direct_client

Addressed by: https://review.openstack.org/96038
    Add container-reconciler daemon

Addressed by: https://review.openstack.org/96039
    Add reconciler probetest outline

Addressed by: https://review.openstack.org/96040
    Merge container storage_policy_index values.

Addressed by: https://review.openstack.org/96041
    Add Storage Policy Support to Account HEAD

Gerrit topic: https://review.openstack.org/#q,topic:policy_func_tests,n,z

Addressed by: https://review.openstack.org/96042
    Adding functional tests for Storage Policy

Gerrit topic: https://review.openstack.org/#q,topic:container_info_sp,n,z

Addressed by: https://review.openstack.org/96043
    Update swift-container-info to be storage policy aware

Addressed by: https://review.openstack.org/96044
    Enqueue misplaced objects during container replication

Addressed by: https://review.openstack.org/96045
    Add LRUCache to common.utils

Addressed by: https://review.openstack.org/96046
    Quick fix to Storage Policy Support for List Endpoints

Addressed by: https://review.openstack.org/96047
    Add Storage Policy Support to Container Sync

Addressed by: https://review.openstack.org/96048
    Add Storage Policy Support to Recon Middleware

Addressed by: https://review.openstack.org/96049
    Add Storage Policy support to the Account Reaper

Addressed by: https://review.openstack.org/96313
    Make quarantine and tmp dirs in DiskFile Storage Policy Aware

Gerrit topic: https://review.openstack.org/#q,topic:fix_probe_nondefault,n,z

Addressed by: https://review.openstack.org/96353
    Fixes probe tests with non-zero default storage policy

Addressed by: https://review.openstack.org/98511
    Fix object-expirer for missing objects

Addressed by: https://review.openstack.org/99315
    Add two vector timestamps

Addressed by: https://review.openstack.org/99979
    Add Storage Policy support to Object Updates

Addressed by: https://review.openstack.org/100807
    Refactoring storage policies merge_timestamps

Addressed by: https://review.openstack.org/101408
    Merge remote-tracking branch 'feature/sp-review'

Addressed by: https://review.openstack.org/101409
    Merge storage poicies feature commit chain into master

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.