Make innobackupex use rsync to copy non-InnoDB files

Registered by Alexey Kopytov on 2011-11-10

Currently innobackupex copies non-InnoDB files by spawning a separate cp/tar process for every file which is inefficient for servers with huge numbers of databases/tables. This can be optimized by using rsync instead of cp, i.e. for local non-streaming backups.

Blueprint information

Status:
Complete
Approver:
Stewart Smith
Priority:
Undefined
Drafter:
Alexey Kopytov
Direction:
Approved
Assignee:
Alexey Kopytov
Definition:
Approved
Series goal:
Accepted for 1.6
Implementation:
Implemented
Milestone target:
milestone icon 1.6.4
Started by
Alexey Kopytov on 2011-11-15
Completed by
Alexey Kopytov on 2011-11-28

Whiteboard

In addition to handling large number of files more efficiently than cp, rsync can also help to decrease FTWRL time even further if used twice: by first copying non-InnoDB files before locking tables, and then syncing any possible changes inside FTWRL.

The goal of this blueprint is to introduce the --rsync option to innobackupex which will be mutually exclusive with --remote-host and --stream. When specified, backup_files() will be called twice: first before calling mysql_lockall() to create a preliminary backup of non-InnoDB files using rsync, and then after mysql_lockall() to sync changes that might occur before the lock.

We have a contributed patch for this blueprint in bug #803556.

There's also a dependency on a proper fix for bug #408803. Since rsync can take longer than wait_timeout, we need a background process pinging the server during its execution. So bug #408803 must be fixed before implementing this blueprint to avoid implementing a separate keepalive logic here.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.