Comment 4 for bug 1594964

Revision history for this message
Przemek (pmalkowski) wrote :

I am able to reproduce the memory leak, on both MySQL Community 5.7 as well as Percona Server 5.7.
The important settings to make it reproducible are:
log-bin=mysql-bin
log-slave-updates
sync_relay_log_info=1
master-info-repository=table
relay-log-info-repository=table
gtid_mode=ON
enforce-gtid-consistency

With stress on low sync_relay_log_info value. With default 5.7 value of 10000, the memory leak is way less visible, and with value of 1, it grows very fast.

Quick steps to reproduce using mysqlsandbox (memory instrument just to illustrate the effect):

make_replication_sandbox --master_master Percona-Server-5.7.17-11-Linux.x86_64.ssl100.tar.gz
cd rcsandbox_Percona-Server-5_7_17
for i in {1..2}; do echo "sync_relay_log_info=1" >> node$i/my.sandbox.cnf ; done
for i in {1..2}; do echo "performance-schema-instrument='memory%=ON' " >> node$i/my.sandbox.cnf ; done

./enable_gtid

node1 [localhost] {msandbox} ((none)) > create database sbtest1;
Query OK, 1 row affected (0.02 sec)

$ sysbench --num-threads=16 --max-requests=300000 --db-driver=mysql --test=oltp --mysql-table-engine=InnoDB --mysql-db=sbtest1 --mysql-user=msandbox --mysql-password=msandbox --oltp-table-size=1000000 --mysql-socket=/tmp/mysql_sandbox15701.sock --test=/usr/share/doc/sysbench/tests/db/oltp.lua prepare
sysbench 0.5: multi-threaded system evaluation benchmark

Creating table 'sbtest1'...
Inserting 1000000 records into 'sbtest1'

$ sysbench --num-threads=16 --max-requests=300000 --db-driver=mysql --test=oltp --mysql-table-engine=InnoDB --mysql-db=sbtest1 --mysql-user=msandbox --mysql-password=msandbox --oltp-table-size=1000000 --mysql-socket=/tmp/mysql_sandbox15701.sock --test=/usr/share/doc/sysbench/tests/db/oltp.lua run
sysbench 0.5: multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 16
(...)
    total number of events: 300124
(...)

$ ps aux|grep rcsandbox_Percona-Server-5_7_17|grep .sock|awk '{print "vsz: "$5/1024" MB","rss: "$6/1024" MB",$19}'
vsz: 2336.39 MB rss: 1966.38 MB --port=15701
vsz: 732.328 MB rss: 380.527 MB --port=15702

node1 [localhost] {msandbox} ((none)) > select * from sys.memory_global_total;
+-----------------+
| total_allocated |
+-----------------+
| 1.51 GiB |
+-----------------+
1 row in set (0.01 sec)

node1 [localhost] {msandbox} ((none)) > select event_name, high_number_of_bytes_used/1024/1024 high_number_of_bytes_used_MB from performance_schema.memory_summary_global_by_event_name order by high_number_of_bytes_used desc limit 10;
+----------------------------------------------------------------------+------------------------------+
| event_name | high_number_of_bytes_used_MB |
+----------------------------------------------------------------------+------------------------------+
| memory/sql/thd::main_mem_root | 1194.45265198 |
| memory/innodb/buf_buf_pool | 133.25000000 |
| memory/innodb/log0log | 32.08451843 |
| memory/sql/XID | 19.00152588 |
| memory/performance_schema/events_statements_history_long | 13.65661621 |
| memory/innodb/fil0fil | 13.28554058 |
| memory/innodb/lock0lock | 11.08512115 |
| memory/performance_schema/events_statements_history_long.sqltext | 9.76562500 |
| memory/performance_schema/events_statements_history_long.tokens | 9.76562500 |
| memory/performance_schema/events_statements_summary_by_digest.tokens | 9.76562500 |
+----------------------------------------------------------------------+------------------------------+
10 rows in set (0.01 sec)

node1 [localhost] {msandbox} ((none)) > select thread_id tid, user, current_count_used ccu, current_allocated ca, current_avg_alloc caa, current_max_alloc cma, total_allocated from sys.memory_by_thread_by_current_bytes limit 10;
+-----+---------------------------+--------+------------+-----------+------------+-----------------+
| tid | user | ccu | ca | caa | cma | total_allocated |
+-----+---------------------------+--------+------------+-----------+------------+-----------------+
| 27 | sql/slave_sql | 530873 | 1.18 GiB | 2.33 KiB | 1.16 GiB | 12.82 GiB |
| 1 | sql/main | 3251 | 192.14 MiB | 60.52 KiB | 133.25 MiB | 224.46 MiB |
| 52 | msandbox@localhost | 51 | 718.49 KiB | 14.09 KiB | 256.00 KiB | 7.99 MiB |
| 26 | sql/slave_io | 379 | 108.16 KiB | 292 bytes | 32.01 KiB | 7.08 MiB |
| 31 | rsandbox@localhost | 51 | 69.58 KiB | 1.36 KiB | 18.07 KiB | 118.83 KiB |
| 25 | innodb/dict_stats_thread | 29 | 1.22 KiB | 43 bytes | 896 bytes | 6.62 MiB |
| 24 | innodb/buf_dump_thread | 3 | 96 bytes | 32 bytes | 96 bytes | 481.02 KiB |
| 2 | sql/thread_timer_notifier | 0 | 0 bytes | 0 bytes | 0 bytes | 0 bytes |
| 3 | innodb/io_log_thread | 0 | 0 bytes | 0 bytes | 0 bytes | 0 bytes |
| 4 | innodb/io_ibuf_thread | 0 | 0 bytes | 0 bytes | 0 bytes | 0 bytes |
+-----+---------------------------+--------+------------+-----------+------------+-----------------+
10 rows in set (0.06 sec)

node1 [localhost] {msandbox} ((none)) > STOP SLAVE SQL_THREAD; START SLAVE SQL_THREAD;
Query OK, 0 rows affected (0.18 sec)

Query OK, 0 rows affected (0.01 sec)

$ ps aux|grep rcsandbox_Percona-Server-5_7_17|grep .sock|awk '{print "vsz: "$5/1024" MB","rss: "$6/1024" MB",$19}'
vsz: 2336.39 MB rss: 445.086 MB --port=15701
vsz: 732.328 MB rss: 380.527 MB --port=15702

node1 [localhost] {msandbox} ((none)) > select * from sys.memory_global_total;
+-----------------+
| total_allocated |
+-----------------+
| 360.34 MiB |
+-----------------+
1 row in set (0.00 sec)