Multiple Database Support in MongoDB (Prototype)

Registered by Ozgur Akan

A write operation in MongoDB locks whole database. At the moment marconi has only one database with two collections (queues and messages). In order to be able to fully utilize system resources, we need to support multiple databases which will increase write/read performance by distributing write locks. A proof of concept would be helpful to see how much this is going to help before fully implementing the solution. During the implementation we can hash queues to different databases or use placement service to make this decision as well.

Blueprint information

Status:
Complete
Approver:
None
Priority:
High
Drafter:
None
Direction:
Approved
Assignee:
Ozgur Akan
Definition:
Approved
Series goal:
Accepted for havana
Implementation:
Implemented
Milestone target:
milestone icon havana-3
Started by
Kurt Griffiths
Completed by
Kurt Griffiths

Related branches

Sprints

Whiteboard

implementation here;

https://github.com/rackerlabs/marconi/tree/multidb

Do we really have confirmed issues related to database lock?
DB-lock handling is supposed to be improved in next versions of Mongo
Why not use sharding to scale writes horizontally?
I think that such changes will only add extra complexity to our code
~alexei-kornienko

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.