SAVE SNAPSHOT takes too much CPU

Registered by Kostja Osipov

SAVE SNAPSHOT can be sped up quite a bit by avoiding costly iteration and doing I/O buffering more efficiently.
Currently it takes up to 80% CPU.

What needs to be done:
(done) - iter_inner and iter_outer removed
- kill conver_snap_row_to_wal
(done) - find the optimal value for setvbuf/IO cache
- experiment with allocating index memory with MAP_SHARED flag or quickly unmmapping it after fork().
- log_io error messages are often quite obscure. At least on Linux, we can extend them
with file names (using /proc filesystem):
static const char *
nfilename(int fd)
{
#ifdef TARGET_OS_LINUX
    char proc_path[32];
    static __thread char filename_path[PATH_MAX];

    sprintf(proc_path, "/proc/self/fd/%d", fd);

    ssize_t sz = readlink(proc_path, filename_path,
                  sizeof(filename_path));

    if (sz >= 0) {
        filename_path[sz] = '\0';
        return filename_path;
    }
#endif /* TARGET_OS_LINUX */
    return ""; /* Not implemented. */
}

- look into pipelining the write, when one thread is busy doing crc32 and formatting the data and another is writing to disk.

Each step ideally should be done separately.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
None
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.