Priority refill for the buffer pool free list
Under highly-concurrent I/O-bound workloads the following situation happens:
1) Buffer pool free lists are used faster than they are refilled by the LRU cleaner thread.
2) Buffer pool free lists become empty and more and more query and utility (i.e. purge) threads start looping in buf_LRU_
3) The number of free list mutex waiters increases, we have seen 200-300 query threads waiting for a free list mutex on 512-thread RW Sysbench runs. The buf_LRU_
4) When the page cleaner thread (or a single page LRU flush by a query thread) finally produces a free page, it is starved from putting it on the free list as it must acquire the free list mutex too. However, being one thread in hundreds, the chances of a prompt acquisition are low.
The priority mutex framework is unable to fully deal with this situation: even though the cleaner thread would be high priority, so are the purge threads, which also consume free pages. Likewise the query threads are low priority, even though they would produce free pages while performing single-page flushes.
Thus:
- Adjust the free list producer, buf_LRU_
- Adjust the free list consumer, buf_LRU_
Ideally the setup would be three-tier, i.e. producers > utility consumers (purge) > query consumers, however such implementation would be complicated and should be done only when clearly shown that it's needed.
Blueprint information
- Status:
- Complete
- Approver:
- Alexey Kopytov
- Priority:
- High
- Drafter:
- Laurynas Biveinis
- Direction:
- Approved
- Assignee:
- Laurynas Biveinis
- Definition:
- Approved
- Series goal:
- Accepted for 5.6
- Implementation:
-
Implemented
- Milestone target:
-
5.6.13-61.0
- Started by
- Laurynas Biveinis
- Completed by
- Laurynas Biveinis
Related branches
Related bugs
Sprints
Whiteboard
Work Items
Dependency tree

* Blueprints in grey have been implemented.