MT LRU flusher

Registered by Laurynas Biveinis

https://github.com/percona/percona-server/pull/349

Note to docs: https://www.percona.com/doc/percona-server/5.6/performance/xtradb_performance_improvements_for_io-bound_highly-concurrent_workloads.html#lru-manager-thread should be carried forward with the changes (s/single-threaded LRU/multi-threaded LRU/ etc), and selected items from the below. This also fixes upstream bug http://bugs.mysql.com/bug.php?id=74637.

Implement a true multithreaded asynchronous LRU flusher.

The existing 5.7 MT flusher is not fully exploiting parallelism: the cleaner threads still serialize LRU and flush list flushing, and the coordinator thread still synchronizes across all the workers for every loop iteration. Both of these are redundant, complicate code, and make flush not as effective as it can be.

Instead, create as many LRU flusher threads as there are buffer pool instances. Each thread is assigned an instance, flushing that instance in a loop and is monitoring its free list length. According to the length, the sleep time between iterations is adapted, ranging from 0ms sleeps for empty and nearly empty free lists to 1 second for free lists having innodb_lru_scan_depths number of pages. The threads are fully independent between themselves and from page cleaner coordinator/worker threads. These have any LRU flushing tasks removed from them.

This also allows to change the innodb_empty_free_list_algorithm server variable default to "backoff."

The following InnoDB metrics are no longer accounted, as their semantics do not make sense under the current LRU flushing design. If there is need, new metric counters with different names will be created.
buffer_LRU_batch_flush_avg_time_slot
buffer_LRU_batch_flush_avg_pass
buffer_LRU_batch_flush_avg_time_thread
buffer_LRU_batch_flush_avg_time_est

The need for InnoDB recovery thread writer threads is also removed, consequently all associated code is deleted.

This work is evolution of a similar feature in 5.6 (https://blueprints.launchpad.net/percona-server/+spec/lru-manager-thread), which only had one LRU flusher thread.

Blueprint information

Status:
Complete
Approver:
Laurynas Biveinis
Priority:
High
Drafter:
Laurynas Biveinis
Direction:
Approved
Assignee:
Laurynas Biveinis
Definition:
Approved
Series goal:
Accepted for 5.7
Implementation:
Implemented
Milestone target:
milestone icon 5.7.10-3
Started by
Laurynas Biveinis
Completed by
Laurynas Biveinis

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.