Storage Base

Registered by Flavio Percoco

Base storage structure for each controller in order to support multiple implementations based on different technologies.

This implementation needs to specify:

* QueueController
  * Can set arbitrary metadata, up to 4K when serialized as JSON
  * Max queues that can be listed at a time is 100 (default 10)
  * Queue names are limited to 64 US-ACII letters, digits, underscores, and hyphens
* MessageController
  * Message bodies are limited to 4K when serialized as JSON
  * Max messages that can be returned or posted at a time is 100 (default 10)
  * TTL is 60 seconds to 1209600 seconds (14 days)
* ClaimsController
  * Max messages that can be claimed at a time is 100 (default 10)
  * TTL is 60 seconds to 1209600 seconds (14 days)
  * Lifetime of claimed messages is extended to be at least as long as claim lifetime (plus a grace period, covered in a separate blueprint)

Considerations for future implementations:
* Every specific implementation, should, and will be, based on this one.
* All implementations should be capable of handlling 100s of thousands of queues per tenant.
* Implementation's should guarantee and preserve the order for incoming and outgoing messages either they're being posted / queried using normal message / claim controller.
* Tenant's default value is None and each implementation will have to find the smartest way to handle a public / common tenant.
* Queue upserts operation must be atomic, isolated and return True when a queue was created and False when it was updated.

Blueprint information

Status:
Complete
Approver:
None
Priority:
Essential
Drafter:
Flavio Percoco
Direction:
Approved
Assignee:
Flavio Percoco
Definition:
Approved
Series goal:
Accepted for havana
Implementation:
Implemented
Milestone target:
milestone icon havana-1
Started by
Flavio Percoco
Completed by
Kurt Griffiths

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/storage-base,n,z

Addressed by: https://review.openstack.org/23674
    Implements base classes for storage controllers

Addressed by: https://review.openstack.org/24423
    Add gettext support

Addressed by: https://review.openstack.org/24430
    Implement some logic in abstract methods

Addressed by: https://review.openstack.org/24450
    Added tenant to the *ControllerBaseTest

Gerrit topic: https://review.openstack.org/#q,topic:bug/1155657,n,z

Addressed by: https://review.openstack.org/24529
    Replace create / update with upsert

Addressed by: https://review.openstack.org/25009
    split message's get method in 2 separate methods

Addressed by: https://review.openstack.org/25318
    MessageController implementation for mongodb storage

Addressed by: https://review.openstack.org/25410
    Use metadata instead of **metadata for claim's updates

Addressed by: https://review.openstack.org/26014
    ClaimMessage Base Tests

Addressed by: https://review.openstack.org/26097
    Remove unused stats() on Claims.

Addressed by: https://review.openstack.org/26036
    Queue Stats in WSGI.

Addressed by: https://review.openstack.org/25931
    Check for malformed JSON when creating a queue.

Addressed by: https://review.openstack.org/25501
    An API limits checking module.

Addressed by: https://review.openstack.org/25948
    List queues in WSGI (w/o paging, but compatible).

Addressed by: https://review.openstack.org/26085
    Make use of the features in falcon 0.1.3

Addressed by: https://review.openstack.org/25938
    Delete queue in WSGI.

Addressed by: https://review.openstack.org/25927
    Claim messages in WSGI.

Addressed by: https://review.openstack.org/26236
    Queue stats returns claimed, expired, and total.

Addressed by: https://review.openstack.org/26252
    client_uuid is not optional to post messages.

Gerrit topic: https://review.openstack.org/#q,topic:sqlite-use-storage-tests,n,z

Addressed by: https://review.openstack.org/27133
    Specify the queue pagination in storage base.

Gerrit topic: https://review.openstack.org/#q,topic:new-paging,n,z

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.