Smarter Partition Selection

Registered by Allele Dev

This blueprint has been superseded. See the newer blueprint "Storage Sharding" for updated plans.

Currently, queue assignment to a partition at creation time for the proxy is determined by a static quantity: weight.

This blueprint proposes to improve this approach by adding a heuristic to determine where the queue is created. This way, partition load is taken into account when making the selection.

Some information to consider when designing this heuristic:
1. Track # of queues created at each partition
2. Track # of requests sent to each partition

Basically, gather any information that could be used to track partition load without actually checking that partition and without requiring more than a constant amount of space to store that information.

The heuristic should be pluggable to the spectrum-mapping algorithm currently used to select partitions, meaning: given an integer, it should output a modified integer.

heuristic :: Int -> IO () -> Int

or better yet:

def heuristic(int) -> int
def partition.load() :: IO () -> dict

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Allele Dev
Direction:
Needs approval
Assignee:
None
Definition:
Superseded
Series goal:
None
Implementation:
Unknown
Milestone target:
None
Completed by
Allele Dev

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.