Storage Policies

Registered by John Dickinson

1) Choose the subset of hardware to use
2) Choose the way data is stored across that subset
3) Optimize the way Swift talks to individual storage volumes

Blueprint information

Status:
Complete
Approver:
None
Priority:
High
Drafter:
None
Direction:
Approved
Assignee:
None
Definition:
Approved
Series goal:
Accepted for juno
Implementation:
Implemented
Milestone target:
milestone icon 2.0.0
Started by
John Dickinson
Completed by
John Dickinson

Related branches

Sprints

Whiteboard

Historical notes in https://blueprints.launchpad.net/swift/+spec/object-replication-strategies

Gerrit topic: https://review.openstack.org/#q,topic:bp/storage-policies,n,z

Addressed by: https://review.openstack.org/61585
    Storage Policy / DiskFile Interface Support

Addressed by: https://review.openstack.org/64461
    Add Storage Policy Support to the Auditor

Addressed by: https://review.openstack.org/52194
    Add storage policy support for the Replicator

Addressed by: https://review.openstack.org/64737
    Add Storage Policy Support to the Updater

Addressed by: https://review.openstack.org/65329
    Add Storage Policy Support to ssync

Addressed by: https://review.openstack.org/65347
    Add Storage Policy Support to ssync

Addressed by: https://review.openstack.org/65517
    Add Storage Policy Support to list_endpoints

Gerrit topic: https://review.openstack.org/#q,topic:bp/ec-proxy-work,n,z

Addressed by: https://review.openstack.org/69700
    Add Storage Policy Support for /Info

Addressed by: https://review.openstack.org/70265
    Replace Policy Index with Policy Name in Response Headers

(we need python-swiftclient updated with CLI support as well, right? swift -p fish stat? or is it on a separate blueprint?)

Addressed by: https://review.openstack.org/73747
    Add Storage Policy Support to Account HEAD

Addressed by: https://review.openstack.org/72535
    Store per-object policy index in container

Addressed by: https://review.openstack.org/71703
    Send policy index to container on sync update

Addressed by: https://review.openstack.org/71702
    Send policy index on async update

Addressed by: https://review.openstack.org/71704
    Store policy index in container_stat table

Addressed by: https://review.openstack.org/72536
    Fix changing of storage policy index

Addressed by: https://review.openstack.org/79731
    Prevent object with storage policy mismatch from being updated in a container

Gerrit topic: https://review.openstack.org/#q,topic:container-put-object-409,n,z

Addressed by: https://review.openstack.org/82590
    Fix PUT to existing container w/o specific storage policy

Addressed by: https://review.openstack.org/83252
    Move Storage Policy validation from parser to class

Addressed by: https://review.openstack.org/83840
    Fix issue with account broker and pre-SPI databases

Addressed by: https://review.openstack.org/85824
    Add Storage Policy Documentation

Addressed by: https://review.openstack.org/86469
    Add Storage Policy Support to Container Sync

Addressed by: https://review.openstack.org/87001
    Minor refactor of storage policy helper functions

Addressed by: https://review.openstack.org/87045
    Slight modification to the object updater sweep code

Addressed by: https://review.openstack.org/87387
    Add Storage Policy Support to Recon Middleware

Gerrit topic: https://review.openstack.org/#q,topic:container-obj-put-409,n,z

Addressed by: https://review.openstack.org/87424
    Super small code cleanup in Updater

Addressed by: https://review.openstack.org/88079
    Quick fix to Storage Policy Support for List Endpoints

Addressed by: https://review.openstack.org/88098
    Add Storage Policy support to the Account Reaper

Gerrit topic: https://review.openstack.org/#q,topic:container-enqueue,n,z

Addressed by: https://review.openstack.org/89747
    Minor policy test cleanup

Addressed by: https://review.openstack.org/91685
    Add probetest for container sync

Addressed by: https://review.openstack.org/91732
    Add probe tests for Account Reaper

Addressed by: https://review.openstack.org/93186
    Update storage_policy_index in expiring_objects container

Addressed by: https://review.openstack.org/96026
    Add Storage Policy Documentation

Addressed by: https://review.openstack.org/96027
    Add Storage Policy Support

Gerrit topic: https://review.openstack.org/#q,topic:shadow-listings,n,z

Addressed by: https://review.openstack.org/96028
    Update FakeLogger with better log level handling

Addressed by: https://review.openstack.org/96029
    Expose Backend container info on deleted containers.

Addressed by: https://review.openstack.org/96030
    Add Storage Policy support to Object Server

Addressed by: https://review.openstack.org/96031
    Add Storage Policy support to Object Updates

Addressed by: https://review.openstack.org/96032
    Add Storage Policy Support to the Auditor

Addressed by: https://review.openstack.org/96033
    Add storage policy support for the Replicator

Addressed by: https://review.openstack.org/96034
    Add Storage Policy Support to ssync

Addressed by: https://review.openstack.org/96035
    Put X-Timestamp in object 404 responses

Addressed by: https://review.openstack.org/96036
    Extend interface on InternalClient

Addressed by: https://review.openstack.org/96037
    Extend direct_client

Addressed by: https://review.openstack.org/96038
    Add container-reconciler daemon

Addressed by: https://review.openstack.org/96039
    Add reconciler probetest outline

Addressed by: https://review.openstack.org/96040
    Merge container storage_policy_index values.

Addressed by: https://review.openstack.org/96041
    Add Storage Policy Support to Account HEAD

Gerrit topic: https://review.openstack.org/#q,topic:policy_func_tests,n,z

Addressed by: https://review.openstack.org/96042
    Adding functional tests for Storage Policy

Gerrit topic: https://review.openstack.org/#q,topic:container_info_sp,n,z

Addressed by: https://review.openstack.org/96043
    Update swift-container-info to be storage policy aware

Addressed by: https://review.openstack.org/96044
    Enqueue misplaced objects during container replication

Addressed by: https://review.openstack.org/96045
    Add LRUCache to common.utils

Addressed by: https://review.openstack.org/96046
    Quick fix to Storage Policy Support for List Endpoints

Addressed by: https://review.openstack.org/96047
    Add Storage Policy Support to Container Sync

Addressed by: https://review.openstack.org/96048
    Add Storage Policy Support to Recon Middleware

Addressed by: https://review.openstack.org/96049
    Add Storage Policy support to the Account Reaper

Addressed by: https://review.openstack.org/96313
    Make quarantine and tmp dirs in DiskFile Storage Policy Aware

Gerrit topic: https://review.openstack.org/#q,topic:fix_probe_nondefault,n,z

Addressed by: https://review.openstack.org/96353
    Fixes probe tests with non-zero default storage policy

Addressed by: https://review.openstack.org/98511
    Fix object-expirer for missing objects

Addressed by: https://review.openstack.org/99315
    Add two vector timestamps

Addressed by: https://review.openstack.org/99979
    Add Storage Policy support to Object Updates

Addressed by: https://review.openstack.org/100807
    Refactoring storage policies merge_timestamps

Addressed by: https://review.openstack.org/101408
    Merge remote-tracking branch 'feature/sp-review'

Addressed by: https://review.openstack.org/101409
    Merge storage poicies feature commit chain into master

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.