Initial Percona Server port to MySQL 8

Registered by Laurynas Biveinis on 2015-10-09

MOVED TO https://jira.percona.com/browse/PS-269, BELOW TEXT WILL BE DELETED AFTER THE LP->JIRA MIGRATION

MySQL 8.0.3 + Percona Patches branch: https://github.com/laurynas-biveinis/percona-server/tree/mysql-8.0.3-percona-patches, also a list of upstream bugs fixed in Percona Server 8.0:
- bug 1262651 / https://bugs.mysql.com/bug.php?id=71411
- bug 1626441 / https://bugs.mysql.com/bug.php?id=71761
- bug 1313901 / https://bugs.mysql.com/bug.php?id=72475
- bug 1660339 / https://bugs.mysql.com/bug.php?id=75311
- bug 1409652 / https://bugs.mysql.com/bug.php?id=75480
- bug 1548597 / https://bugs.mysql.com/bug.php?id=76142
- bug 1657941 / https://bugs.mysql.com/bug.php?id=76418
- bug 1466414 / https://bugs.mysql.com/bug.php?id=77399
- bug 1470677 / https://bugs.mysql.com/bug.php?id=77591
- bug 1525407 / https://bugs.mysql.com/bug.php?id=79610
- bug 1549301 / https://bugs.mysql.com/bug.php?id=80496
- bug 1596845 / https://bugs.mysql.com/bug.php?id=82026
- bug 1548597 / https://bugs.mysql.com/bug.php?id=82480
- bug 1622034 / https://bugs.mysql.com/bug.php?id=82935
- bug 1539504 / https://bugs.mysql.com/bug.php?id=83003
- bug 1634932 / https://bugs.mysql.com/bug.php?id=83648
- bug 1625151 / https://bugs.mysql.com/bug.php?id=84366
- bug 1654091 / https://bugs.mysql.com/bug.php?id=84415
- bug 1728509 / https://bugs.mysql.com/bug.php?id=85205
- bug 1728512 / https://bugs.mysql.com/bug.php?id=85208
- bug 1668602 / https://bugs.mysql.com/bug.php?id=85258
- bug 1667307 / https://bugs.mysql.com/bug.php?id=85371
- bug 1679025 / https://bugs.mysql.com/bug.php?id=85835
- bug 1688698 / https://bugs.mysql.com/bug.php?id=86209

On 5.7 trunk before porting starts:
- fix compilation warning regressions (remove innobase_get_slow_log leftover prototype from srv0srv.h at the same time)
- fix the consistently failing tests (+ sanitizers & Valgrind)

In the 5.7 porting branch, assuming port to at least 8.0.2:
- revert our fix for bug 1594964 / https://bugs.mysql.com/bug.php?id=85251 (no testcase) due to 8.0 not having the bug
- remove mysql-test/include/not_ubuntu_precise.inc
- revert our fix for bug 1604462 / https://bugs.mysql.com/bug.php?id=82283 due to 8.0.2 fix
- revert our fix for https://bugs.mysql.com/bug.php?id=85088 / https://bugs.mysql.com/bug.php?id=84736 (?) / https://bugs.mysql.com/bug.php?id=84451 (?) / bug 1660591 while keeping the testcase due to 8.0.2 fix
- revert our fix for: https://bugs.mysql.com/bug.php?id=85678 due to 8.0.2 fix
- remove #ifdef HAVE_REPLICATION from our diff (f98b7bc5fdfc6777afd548b369b6646c40581eb3)
- revert https://bugs.mysql.com/bug.php?id=79894 / bug 1532747 fix due to upstream 8.0.0 fix
- revert the InnoDB fsync EINTR fix (http://bugs.mysql.com/bug.php?id=71183 / bug 1262651) as fixed in 8.0 upstream
- revert our fix for https://bugs.launchpad.net/percona-server/+bug/1561091 / https://bugs.mysql.com/bug.php?id=80288
- revert QC patches
- use CR_ client errors instead of hardcoded numbers in the testcase (s/2013/CR_SERVER_LOST etc)
- remove DTrace support (https://dev.mysql.com/worklog/task/?id=9581)
- consider implementing https://blueprints.launchpad.net/percona-server/+spec/use-innodb-metrics-for-extended-stats
- consider implementing https://blueprints.launchpad.net/percona-server/+spec/deprecate-and-obsolete-max-binlog-files (the new variable; the deprecation goes to the lower branches then)
- fix bug 1642376
- remove the scalability metrics plugin;
- revert the buffer pool mutex split patch (https://bugs.mysql.com/bug.php?id=75534 / http://dev.mysql.com/worklog/task/?id=8423)
- Remove tokudb_support_xa variable
- revert our fix for https://bugs.mysql.com/bug.php?id=79703
- remove INFORMATION_SCHEMA.THREAD_STATISTICS.CONCURRENT_CONNECTIONS column
- remove innodb_kill_idle_trx alias variable
- fix rocksdb.select test to remove deprecated PROCEDURE ANALYSE (https://jira.percona.com/browse/MYR-133)

Once MySQL 8.0 is merged:
- https://dev.mysql.com/worklog/task/?id=8971: remove have_partition.inc from the testcases, WITH_PARTITION_STORAGE_ENGINE preprocessor usage
- Review SUPER-requiring features for finer-grained dynamic privileges (https://dev.mysql.com/worklog/task/?id=8131)
- C++11'ize where beneficial (atomics, nullptr, override, auto, static_assert, noexcept, using, const_iterator, raw string literals, pass by value etc). Consider using clang-tidy with "modernize" checkers
- move percona_bug1331586 to rpl_gtid suite (http://dev.mysql.com/worklog/task/?id=9249)
- try -DWITH_UBSAN=ON build, add to Jenkins
- test MTR --no-skip option (http://dev.mysql.com/worklog/task/?id=8036)
- If mysql-8.0-template is split out of mysql-5.7-template, add --test-progress --report-unstable-tests --fail-check-testcases [--sanitize] to MTR invocation
- replace --exec rm -rf dir with force-rmdir in MTR
- likewise perl snippet at mysql-test/suite/tokudb.backup/t/innodb_use_native_aio_enabled.test
- use a an error code range for Percona Server-specific codes

Once MySQL 8.0.1 is merged:
- restore MyRocks select_for_update_skip_locked test https://jira.percona.com/browse/MYR-71
- remove absolute path --plugin-dir feature and replace its uses 95a15b435b88104929d5af9f28917c1bb6e03df0
- Create separate --only-big-tests trunk job to make the main trunk job smaller (but not for Valgrind until https://bugs.mysql.com/bug.php?id=87247 is fixed)

Once MySQL 8.0.2 is merged:
- remove explicit master_info_repository=TABLE and relay_log_info_repository=TABLE settings from the testcases to match the changed defaults

Once MySQL 8.0.3 is merged:
- Test the combination of SET STATEMENT ... FOR ... and /*+ SET_VAR(...) */ optimizer hint
- remove any explicit innodb_autoinc_lock_mode=2, log_slave_updates, log_bin, max_allowed_packet < 64M settings from the testcases
- Reconcile our backup locks and upstream LOCK INSTANCE FOR BACKUP. Our backup locks should be authorized by the same BACKUP_ADMIN privilege.
- Check if our audit API plugins should handle the new MYSQL_AUDIT_AUTHENTICATION_CLASS, add MTR tests for them with caching sha2 user connections
- For Jenkins trunk Valgrind jobs, create separate ones for small tests and --only-big-test
- Setup rpl_manygtids, special MTR suites in Jenkins

Once MySQL 8.0.4 is released:
- Verify upstream fixes for
- - https://bugs.mysql.com/bug.php?id=85993
- - https://bugs.mysql.com/bug.php?id=85371 / bug 1667307
- - https://bugs.mysql.com/bug.php?id=86209 / bug 1688698
- - https://bugs.mysql.com/bug.php?id=83003 / bug 1539504
- - https://bugs.mysql.com/bug.php?id=85258 / bug 1668602
- - https://bugs.mysql.com/bug.php?id=72475 / bug 1313901
- - https://bugs.mysql.com/bug.php?id=83814 / bug 1702903

MyRocks & TokuDB:
- Plugin ID changed in 8.0. They introduced new ID at ~30? for DB_TYPE_TEMPTABLE. In 5.7 TokuDB is 41 and MyRocks is 42.
- Investigate engine buffer estimates for optimizer https://dev.mysql.com/worklog/task/?id=7170

TokuDB:
- upgrade from 5.7 TokuDB tables with patched FRM format (clustering secondary index flag) must work (http://dev.mysql.com/worklog/task/?id=6392, http://dev.mysql.com/worklog/task/?id=6381) ** GL Note, TokuDB stores copy of .frm in status files and does not work well when .frm format changes. TokuDB upgrade needs much more investigation.
- Investigate updating TokuDB tree shape defaults to be optimal for SSD rather then slow disk.

MyRocks:

COMPLETED
MySQL 8.0.0 + Percona patches branches, retired: https://github.com/laurynas-biveinis/percona-server/tree/mysql-8.0.0-percona-patches, https://github.com/laurynas-biveinis/percona-server/tree/mysql-8.0.0-percona-patches-clean

MySQL 8.0.1 + Percona patches branches, retired: https://github.com/laurynas-biveinis/percona-server/tree/mysql-8.0.1-percona-patches, https://github.com/laurynas-biveinis/percona-server/tree/mysql-8.0.1-percona-patches-clean

MySQL 8.0.2 + Percona patches branch, retired https://github.com/laurynas-biveinis/percona-server/commits/mysql-8.0.2-percona-patches

On 5.7 trunk before porting starts:
- fixed https://bugs.launchpad.net/percona-server/+bug/1160986 to minimise diff.
- fixed https://bugs.launchpad.net/percona-server/+bug/1121072 to minimise diff.
- reviewed https://dev.mysql.com/worklog/task/?id=9354, no changes
- fixed bug 1706981

LP 8.0 series created
- set query cache bugs to Won't Fix: bug 1623040, bug 1655830
- updated statuses for BP-linked upstream bugs

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.