statistics collection using scoreboard
work towards redoing the current statistics approach used in the server to use a scoreboard approach of collecting statistics. This will be a many step process as there are lots of statistics that could use this approach.
Blueprint information
Whiteboard
Jay Pipes pointed out a interested approach that should be expored. A description from him is below.
As an aside, you might want to make a blueprint for enhancing the scoreboard algorithm to use a modulo-vector based approach. This would significantly improve performance for installations with a very high number of concurrent sessions.
The idea behind a modulo-vector scoreboard approach is to split the scoreboard into X number of modulo divisions, each containing a fixed-size vector of slots. To find a free slot, you would modulo the session ID by X and then only search that vector of slots for a free scoreboard slot. In this way, you cut the coefficient of the current algorithm performance of O(logN) by X to O(logN/X).
Assume that you have 1000 scoreboard slots in your original approach. Since you are linearly reading the 1000 slots from front to back every time you search for a free slot, the average time to find a free slot for normal operations is O(log1000). If you use a simple modulo-vector algorithm and instead of linearly searching through 1000 slots, you split the scoreboard into 10 vectors of 100 slots each. Instead of having a single rw_lock for the scoreboard, you have one lock per division of the scoreboard (so, 10 rw_locks). When searching for a free slot, you first modulo the session ID against 10, giving you the division to search in. You then take a lock on that dividion and then search linearly through those 100 slots. In this way, you cut down the average search time to O(log100). Since you are holding a write lock during the search, this would, IMHO, significantly improve performance and scalability.
