ThreadPool: low priority queue throttling in 5.5

Registered by Alexey Kopytov

One case that can limit thread pool performance and even lead to
deadlocks under high concurrency is a situation when thread groups are
oversubscribed due to active threads reaching the oversubscribe limit,
but all/most worker threads are actually waiting on locks currently held
by a transaction from another connection that is not currently in the
thread pool.

What happens in this case is that those threads in the pool that have
marked themselves inactive via thd_wait_begin() are not accounted to the
oversubscribe limit. As a result, the number of threads (both active and
waiting) in the pool grows until it hits threadpool_max_threads. If the
connection executing the transaction which is holding the lock has
managed to enter the thread pool by then, we get a large (depending on
the threadpool_max_threads value) number of concurrently running threads
and thus, suboptimal performance as a result. Otherwise, we get a
deadlock as no more threads can be created to process those
transaction(s) and release the lock(s).

The goal of this blueprint is to prevent such situations by throttling
the low priority queue when the total number of worker threads (both
active and waiting ones) reaches the oversubscribe limit. That is, if
there are too many worker threads, do not start new transactions and
create new threads until queued events from the already started transactions
are processed.

This rather simple change has shown a great impact on performance in our
benchmarks.

Blueprint information

Status:
Complete
Approver:
Alexey Kopytov
Priority:
High
Drafter:
Alexey Kopytov
Direction:
Approved
Assignee:
Alexey Kopytov
Definition:
Approved
Series goal:
Accepted for 5.5
Implementation:
Implemented
Milestone target:
milestone icon 5.5.35-33.0
Started by
Alexey Kopytov
Completed by
Laurynas Biveinis

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.