Storage Base

Registered by Flavio Percoco

Base storage structure for each controller in order to support multiple implementations based on different technologies.

This implementation needs to specify:

* QueueController
  * Can set arbitrary metadata, up to 4K when serialized as JSON
  * Max queues that can be listed at a time is 100 (default 10)
  * Queue names are limited to 64 US-ACII letters, digits, underscores, and hyphens
* MessageController
  * Message bodies are limited to 4K when serialized as JSON
  * Max messages that can be returned or posted at a time is 100 (default 10)
  * TTL is 60 seconds to 1209600 seconds (14 days)
* ClaimsController
  * Max messages that can be claimed at a time is 100 (default 10)
  * TTL is 60 seconds to 1209600 seconds (14 days)
  * Lifetime of claimed messages is extended to be at least as long as claim lifetime (plus a grace period, covered in a separate blueprint)

Considerations for future implementations:
* Every specific implementation, should, and will be, based on this one.
* All implementations should be capable of handlling 100s of thousands of queues per tenant.
* Implementation's should guarantee and preserve the order for incoming and outgoing messages either they're being posted / queried using normal message / claim controller.
* Tenant's default value is None and each implementation will have to find the smartest way to handle a public / common tenant.
* Queue upserts operation must be atomic, isolated and return True when a queue was created and False when it was updated.

Blueprint information

Flavio Percoco
Flavio Percoco
Series goal:
Accepted for havana
Milestone target:
milestone icon havana-1
Started by
Flavio Percoco
Completed by
Kurt Griffiths

Related branches



Gerrit topic:,topic:bp/storage-base,n,z

Addressed by:
    Implements base classes for storage controllers

Addressed by:
    Add gettext support

Addressed by:
    Implement some logic in abstract methods

Addressed by:
    Added tenant to the *ControllerBaseTest

Gerrit topic:,topic:bug/1155657,n,z

Addressed by:
    Replace create / update with upsert

Addressed by:
    split message's get method in 2 separate methods

Addressed by:
    MessageController implementation for mongodb storage

Addressed by:
    Use metadata instead of **metadata for claim's updates

Addressed by:
    ClaimMessage Base Tests

Addressed by:
    Remove unused stats() on Claims.

Addressed by:
    Queue Stats in WSGI.

Addressed by:
    Check for malformed JSON when creating a queue.

Addressed by:
    An API limits checking module.

Addressed by:
    List queues in WSGI (w/o paging, but compatible).

Addressed by:
    Make use of the features in falcon 0.1.3

Addressed by:
    Delete queue in WSGI.

Addressed by:
    Claim messages in WSGI.

Addressed by:
    Queue stats returns claimed, expired, and total.

Addressed by:
    client_uuid is not optional to post messages.

Gerrit topic:,topic:sqlite-use-storage-tests,n,z

Addressed by:
    Specify the queue pagination in storage base.

Gerrit topic:,topic:new-paging,n,z


Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.


No subscribers.