Profile and speed up restart from snapshot

Registered by Kostja Osipov

Most likely we can double restart speed by tuning read parameters/read buffers, moving out crc validation and parsing into a separate thread.

Other ways to improve restart speed:
- we can fire off a separate thread to build each TREE index

Since slab is single-threaded, we can't populate spaces concurrently, but we can do it in parallel to building the indexes.

The overall scheme looks like this:

The main thread, AKA snap_reader thread:
reads the snapshot, validates crc, allocates tuples on the slab_alloc_arena,
adds to the queue for insert into the space.

Subordinate to snap_reader, insert_thread, inserts data into the primary key index, and secondary HASH indexes. Once done with a given space, puts a task to the sort thread queue, for sorting.

Sort threads: create secondary keys by sorting, yield back sorted arrays.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
None
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.