Initial port to 5.7.9
https:/
Merge Percona Server 5.6 to MySQL 5.7.9, producing Percona Server 5.7.9
This implements blueprint
https:/
with user-visible changes as described at
https:/
Non-exhaustive list of non-trivial code changes in the port, core server:
- Percona Server version now consists of only single number.
- Annotate core server, MEMORY engine, and XtraDB heap memory
allocations for Performance Schema
(http://
http://
performance_
instrumentations fit into default sizing. Bump
performance_
instrumentations still leave room for at least one of the bundled
plugins, when loaded dynamically.
- Snapshot cloning, kill idle transaction features, thread pool, and
INFORMATION_
to use Global_THD_manager
(http://
- Even though bug 57430 is fixed in upstream 5.7, keep our patch for
the same, as it, combined with other TokuDB patches to prefer
covering index accesses, still results in improved query plans.
- Removed debug-only Q_QUERY_EXEC_TIME binlog event status flag.
- Update thread pool for the connection handling refactoring of 5.7
(https:/
- Added support for global backup and binlog locks for
PERFORMANCE_
- Per-statement variable feature parser part rewritten for the
bottom-up parser (http://
http://
https:/
- Utility user feature updated for auth code refactoring
(https:/
- Query optimizer patches updated for
http://
https:/
http://
- Slow query log and binlog file management features have been updated
for binlog/slow log split in the code
(https:/
- Query cache patches now have search keys decoupled from its memory
layout, QueryStripComme
and made local to sql_cache.cc, and
Query_
in place for the query cache key computation.
- Updated code for diagnostics API changes because of
http://
- Updated code for THD field locking changes because of
http://
- Updated code for LOCK_thread_count split
(http://
- Updated code for MY_ATOMIC_
(http://
- Updated code and testcase for removed INSERT DELAYED
(http://
- Cleaned up incomplete revert of our fix to bug 915814 / 64624.
XtraDB:
- Buffer pool mutex split updated for online buffer pool resizing by
protecting buf_pool_
(http://
- InnoDB redo log archiving feature removed due to W#8845 changing the
log format (including clobbering the "log archived up to this LSN"
header fields) and general lack of users of the feature.
- XtraDB fake changes prefetching feature removed.
- LRU manager thread not ported forward due to upstream multi threaded
flusher requiring a different design for independent LRU flusher
(http://
- innodb_
(the 5.6 default) may currently cause deadlocks due to lack of
LRU flusher which would be separate from flush list flusher.
- Priority mutex framework not ported forward due to it needing
updating for the upstream policy mutex
(https:/
achieve its scalability gain differently.
- Priority rwlock framework not ported forward due to it needing
updating for the upstream SX-lock
(http://
alleviated by converting the index latch to this SX-lock
(http://
- Snapshot cloning updated for 5.7 transaction management changes:
remove the assumption on RO to RW transaction upgrade that a
transaction always sees its own changes, which is a reasonable
assumption in general, but is not the case for snapshot
cloning. Also, preassign transaction ids for RO transaction views
being cloned to use in the case of later RO to RW upgrade. Do not
port forward the transaction descriptor
feature. (https:/
https:/
function has been resurrected as ReadView::print(). Expand
clone_
repeated cloning of the same transaction, and cascaded cloning with
different RO/RW transaction combinations.
- Update changed page tracking to handle new MTR types
(http://
http://
format/checksum algorithm
(http://
- Updated XtraDB corrupt table action feature to handle InnoDB native
partitioning (where ha_innobase:
before).
- Make XtraDB track redo log synchronously in case of redo log
resizing.
- XtraDB diagnostics were converted from fprintf to
ib::info(
- Updated code for innodb_
(https:/
- INFORMATION_
with three columns PHYSICAL_PAGE_SIZE, LOGICAL_PAGE_SIZE, and
IS_COMPRESSED.
- INFORMATION_
been removed.
Plugins:
- Audit_log, query response time, and scalability_metrics plugins
updated for 5.7 audit plugin API changes.
- Audit_log was converted to use MySQL plugin services for heap memory
allocation.
- Plugin MTR tests were moved to plugin/tests/mtr directory from the
main suite, effectively creating one testsuite per plugin.
- Removed HandlerSocket plugin.
Tests:
- Testsuite updates to account for: MBR temp table binlogging
differences caused by our fix for bug 1313901 / 72475; for memory to
disk table conversion happennning at a different point due to
combination of dynamic VARCHAR patch for MEMORY engine and bug
55559; for INFORMATION_
INFORMATION_
(http://
http://
MTR engine (http://
- Updated partitioned InnoDB table testcases that were testing the
partition handler to use MyISAM instead in order to avoid InnoDB
native
partitioning; remove --source include/
non-InnoDB-
testsuite to use include files for server restart/
more; added various new testcases.
- Since 5.7 removed MDL_map_
1405076 disappears, and its testcase is removed, as debug sync
points there do not exist anymore nor it's possible to re-introduce
them.
- percona_
the VARCHAR size as the latter operation became online in 5.7.1.
- Removed tests that do not make sense after upstream remove
--skip-innodb option, and some supporting code
(RETURN_
- Removed MTR Subunit patch. Jenkins reporting will be handled externally
to the source tree.
Other:
- Mechanical code changes: removal of HAVE_ATOMIC_
DYNAMIC_ARRAY in C++ code, ib_logf, HAVE_purify, UNIV_INTERN,
EXTENDED_FOR_*, QUERY_CACHE_
replacement with ULLONGMAX, ibool/FALSE/TRUE with bool/false/true,
string constification by using LEX_CSTRING in place of LEX_STRING
(http://
http://
This also takes upstream fixes for the following bugs fixed in upstream
5.7:
- https:/
http://
variable names);
- https:/
http://
handler:
- https:/
http://
InnoDB table);
- https:/
http://
that fail on master);
- https:/
http://
- https:/
http://
void Item_row:
At the same time fix
- https:/
deadlock counter in PS5.6.11 is not needed) by dropping the
innodb_deadlocks status variable;
- https:/
http://
INFORMATION_
metric collection site adjustment.
Revert our features and bugfixes implemented in upstream 5.7:
- --innodb-
mysqld_safe.sh option due upstream --innodb-
- super_read_only (http://
- innodb_
- bug 1334330 / 70860;
- bug 1157037 / 68713;
- bug 1351148 / 73979;
- bug 1169494 / 68970;
- bug 1132350 / 68476;
- innodb_
(http://
- --secure-
- --innodb-
- all the XtraDB status counters that have enabled-by-default
INNODB_METRICS counterparts;
- bug 1268729 / 71374;
- SHOW SLAVE STATUS NOLOCK
(http://
- mysql --syslog option (http://
- bug 1042517 / 66560;
- bug 1132351 / 68477;
- bug 1068210 / 67504;
- bug 606811 / 54814;
- bug 1255551 / 70854;
- slow_query_
- slow_query_
- bug 1197524 / 41975;
- mysqlbinlog --rewrite-db option;
- fusionIO atomic writes;
- bug 1163439 / 68845;
- bug 1279671 / 71708;
- multiple GET_LOCK per connection feature
(http://
- bug 1238039 / 71988;
- bug 1115048 / 60682;
- bug 1331586 / 73066;
- bug 1220544 / 70228;
- max_statement_time because of max_execution_time
(http://
- Online GTID migration patch (gtid_deploymen
dynamic gtid_mode variable
(http://
- mysqldump --ignore-
- innodb_
innodb_
- INFORMATION_
PERFORMANCE_
- mutex names in SHOW ENGINE INNODB MUTEX;
- bug 1396330 / 74987;
- bug 1361568 / 73736;
- bug 1380010;
- bug 1083377 / 67685;
- bug 1262500 / 54430;
- majority of bug 878404 / 45702, except for the partial pread read
handling in my_pread;
- error number padding patch for comp_error;
- mutex names in SHOW ENGINE INNODB MUTEX output
- mysqldump --ignore-
Blueprint information
- Status:
- Complete
- Approver:
- Laurynas Biveinis
- Priority:
- Essential
- Drafter:
- Laurynas Biveinis
- Direction:
- Approved
- Assignee:
- Laurynas Biveinis
- Definition:
- Approved
- Series goal:
- Accepted for 5.7
- Implementation:
-
Implemented
- Milestone target:
-
5.7.10-1rc1
- Started by
- Laurynas Biveinis
- Completed by
- Laurynas Biveinis
Related branches
Related bugs
Sprints
Whiteboard
Completed porting prerequisites:
- A complete 5.6 tree;
- file-ids resolved;
- 5.6 port review bugs fixed:
-- commented on https:/
- merge-regression bugs fixed (https:/
-- https:/
-- https:/
-- https:/
-- https:/
-- our mysql-test/
- a 5.6/5.7 GCA branch (see the linked branches of this blueprint), check and revert patches to be dropped on that branch. 5.6.16 is fully merged to 5.7.4. 5.6.20 is fully merged to 5.7.5
Processed Jenkins:
- Do not need to define HAVE_purify anymore.
Processed features and bug fixes:
- show slave status nolock (https:/
- mysql --syslog patch (5.7.1) (reverted in the 5.6 branch)
- bug 1042517 fix (5.7.1) (reverted in the 5.6 branch)
- bug 1132351 fix (5.7.2) (reverted in the 5.6 branch. Oracle 5.7 fix is slightly different, but that's OK, PS 5.7 will be benchmarked and the fix might be adjusted as necessary).
- bug 1068210 fix (5.7.2) (reverted in the 5.6 while keeping the testcase).
- bug 606811 / bug 1186974 (5.7.4) (reverted in the 5.6 branch)
- bug 1255551 (5.7.4) (reverted in the 5.6 branch)
- slow_query_
- bug 1197524 fix (5.7.3) (reverted in the 5.6 branch, testcase disabled temporarily)
- mysqlbinlog --rewrite-db (5.7.2) (reverted in the 5.6 branch, testcase disabled temporarily).
- query timeout. (reverted in the 5.6 branch, will take the 5.7 implementation).
- trx descriptors: bug 1131189 / bug 1131187 / bug 1170103. (reverted in the 5.6 branch keeping the bug 1170103 testcase).
- fusionIO atomic writes (5.7.4 revs 7517, 7518) (reverted in the 5.6 branch)
- bug 1163439 fix. (reverted in the 5.6 branch)
- bug 1182535 fix (5.7.5). (reverted in the 5.6 branch. Will likely need 5.7-specific fixing later).
- NOT REVERTED bug 1279671 fix (5.7.5). Upstream fix incomplete, does not handle autoinc_mutex.
- Kostja's GET_LOCK patch (5.7.5) (reverted in the 5.6 branch).
- NOT REVERTED bug 1218347 fix (5.7.5). Upstream fix very conservative.
- bug 1238039 fix (5.7.5-) (reverted in the 5.6 branch)
- buffer pool mutex split. Sent to Oracle at http://
- innodb_
- HandlerSocket plugin. (reverted in the 5.6 branch).
- fake changes. (reverted in the 5.6 branch).
- http://
- http://
- http://
- Subunit MTR patch (reverted in the 5.6 branch; 5.7 Jenkins does it differently).
- bug 1299688 fix (reverted in the 5.6 branch).
- removed HAVE_purify from CMake Valgrind setup
- Move plugin-specific testcases to plugin-specific test suites
- replace ibool with bool in InnoDB patch
- bug 1220544 fix / http://
- Removed secure_file_priv extension (keep the testcase, it tests more) (upstream implemented in 5.7.6)
- Removed innodb_
- bug 1132350 / http://
- bug 1169494 / http://
- bug 1351148 / http://
- bug 1157037 / http://
- bug 1334330 / http://
- dropped our log block crc32 feature (http://
- dropped our super_read_only feature (upstream implemented in 5.7.8)
- bug 1484599 / http://
- removed innodb_
- attempted to do -DWITH_UBSAN=ON testing
- removed --source include/
- dropped various -master.opt options that match the changed defaults
- checked whether our testcases can use include/
- made plugins use MySQL plugin services API
- replaced any server fprintf(stderr) with WL #6661 changes http://
- added PFS memory allocation annotations (my_malloc, ut_*nokey)
- utility user made compatible with offline_mode
- added testcases for 74900, 74584 / bug 1384656
- tested the compatibility of SHOW CREATE USER with utility user
- tested the compatibility of ACCOUNT LOCK/UNLOCK with utility user
- added testcases for 77580, 76430
- implemented and added testcase for combination of enforced and disabled storage engines
Work Items
Dependency tree

* Blueprints in grey have been implemented.