Implement parallel replicaton

Registered by Denis M. on 2014-02-10

Parallel replication. Parallel apply is an important technique for achieving high speed replication and curing slave lag. It works by spreading updates to slaves over multiple threads that split transactions on each schema into separate processing streams. This in turn spreads I/O activity across many threads, which results in faster overall updates on the slave. In ideal cases throughput on slaves may improve by up to 5 times over single-threaded MySQL native replication.

Parallel replication works best on workloads that meet the following criteria:
• Data are stored in independent schemas. If you have 100 customers per server with a separate schema for each customer, your application is a good candidate.
• Transactions do not span schemas. Service serializes such transactions, which is to say it stops parallel apply and runs them by themselves. If more than 2-3% of transactions are serialized in this way, most of the benefits of parallelization are lost.
• Workload is well-balanced across schemas.
• The slave host(s) are capable and have free memory in the OS page cache.
• The host on which the slave runs has a sufficient number of cores to operate a large number of Java threads.
• Not all workloads meet these requirements. If your transactions are within a single schema only, you may need to consider different approaches, such as slave prefetch.

Parallel replication does not work well on underpowered hosts, such as m1.small instances. In fact, any host that is already I/O bound under single-threaded replication will typical will not show much improvement with parallel apply.

Required components:
1. The ReqRouter service. ReqRouter (known as routers within the dataservice) provide a routing mechanism between client applications and the dataservice. ReqRouter component automatically routes database operations to the master or slave, and takes account of the current cluster status as communicated to it.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
Denis M.
Direction:
Needs approval
Assignee:
None
Definition:
Drafting
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

[SlickNik] Please follow BP template if approval is needed. Thanks!

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.