Comment 2 for bug 803556

Revision history for this message
Ben Hencke (brainstar) wrote :

I'm attaching a patch made from xtrabackup-1.6.3 (innobackupex reports v1.5.1-xtrabackup). This adds a --rsync option, which reduces the lock time significantly, but still keeps

The script does an initial pre-lock rsync of the various db files, then during lock rsync any changes that were flushed. The 2nd rsync should run much faster since it would only need to copy changed files, and rsync is very efficient when comparing thousands of files.

It still uses all of the same logic it would have to run the copy, but instead writes the list of files to a temp file. Then calls rsync once and so should also avoid spawning thousands of processes, and still be compatible with all of the various single table/db options.

LIMITATIONS: I haven't made it work with --remote-host or --stream options. It will not remove any files that existed in the first copy, and were then deleted by the time the tables were locked ie DROP or RENAME. It would be possible to diff the first pass file list with the second, and remove the appropriate files, but I haven't implemented that.

To test, a created 10,000 dummy tables, then ran a backup with different settings, dropping the filesystem cache before each.

Using the unmodified version, the copy took 42 seconds. Total lock time was 54 seconds (from acquire to release).

Using the --rsync version, the first rsync took 21 seconds (no lock), and the second (lock) took 2 seconds. Total lock time was 15 seconds.