Profile and speed up restart from snapshot
Most likely we can double restart speed by tuning read parameters/read buffers, moving out crc validation and parsing into a separate thread.
Other ways to improve restart speed:
- we can fire off a separate thread to build each TREE index
Since slab is single-threaded, we can't populate spaces concurrently, but we can do it in parallel to building the indexes.
The overall scheme looks like this:
The main thread, AKA snap_reader thread:
reads the snapshot, validates crc, allocates tuples on the slab_alloc_arena,
adds to the queue for insert into the space.
Subordinate to snap_reader, insert_thread, inserts data into the primary key index, and secondary HASH indexes. Once done with a given space, puts a task to the sort thread queue, for sorting.
Sort threads: create secondary keys by sorting, yield back sorted arrays.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- None
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- New
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by