Support for decompression in xbstream

Registered by Sergei Glushchenko on 2012-07-19

Support for qiucklz decompression in xbstream.

The purpose of this functionality is to allow extract backups produced by
xtrabackup using single xbstream command.

xtrabackup can produce stream which contains compressed files. Currently
only quicklz compression algorithm is supported. Compressed files have name
which ends with ".qp". Important note is that each file compressed separately, not
the whole stream like we have in case of tar + gz/bz2 streams.

New option to be added --decompress-program for xbstream which will work for
extracting only. When this option specified, xbstream will automatically
decompress compressed files.

This will be done by implementing datasink_popen, which will call external command
like 'qpress' for decompressing each file. Command will be executed with popen system
call and the entire archive will be passed to standard input of this command.

Example usage:
xbstream -C tmp -xv --decompress-program="qpress -diT4 ." < stream.xbs

Blueprint information

Status:
Complete
Approver:
Alexey Kopytov
Priority:
Undefined
Drafter:
Sergei Glushchenko
Direction:
Needs approval
Assignee:
Sergei Glushchenko
Definition:
Obsolete
Series goal:
None
Implementation:
Blocked
Milestone target:
None
Started by
Alexey Kopytov on 2012-08-28
Completed by
Alexey Kopytov on 2012-08-29

Whiteboard

After further analysis and discussions it was decided that this feature does not really have real applications, while complicating the xbstream architecture significantly.

Users willing to utilize compression only to speed up transferring the entire archive to another host do not really need built-in compression in XtraBackup. That feature was implemented for different use cases. Instead, users can use normal uncompressed stream in XtraBackup (either tar, or xbstream in case of parallel or incremental backups) and then use external compression utilities (either single- or multi-threaded) to compress the stream and decompress it on remote host.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.