Drizzle

message size in DD table INNODB_REPLICATION_LOG is too small

Bug #711068 reported by Joe Daly
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Drizzle
Fix Released
Critical
Andrew Hutchings
Drizzle 2011-02-14
7.0
Fix Released
Critical
Andrew Hutchings
Drizzle 2011-02-14

Bug Description

in replication_dictionary.cc

add_field("TRANSACTION_MESSAGE_STRING", plugin::TableFunction::STRING, 2048, false);
add_field("TRANSACTION_MESSAGE_BINARY", plugin::TableFunction::VARBINARY, 2048, false);

these should be based on the gpb message size sys_var

Tags: replication

Related branches

lp:~linuxjedi/drizzle/trunk-bug-711068
Rejected for merging into lp:drizzle/7.0
Lee Bieber (community): Needs Fixing
Diff: 495 lines (+100/-51)
47 files modified
drizzled/drizzled.cc (+5/-3)
drizzled/drizzled.h (+2/-0)
drizzled/session.h (+0/-1)
drizzled/sys_var.cc (+2/-2)
drizzled/sys_var.h (+11/-0)
drizzled/transaction_services.cc (+6/-6)
plugin/innobase/handler/replication_dictionary.cc (+4/-2)
plugin/innobase/tests/r/innodb_replication_log.result (+18/-0)
plugin/innobase/tests/t/innodb_replication_log-master.opt (+1/-0)
plugin/innobase/tests/t/innodb_replication_log.test (+14/-0)
plugin/transaction_log/tests/t/alter-master.opt (+1/-1)
plugin/transaction_log/tests/t/auto_commit-master.opt (+1/-1)
plugin/transaction_log/tests/t/bad_replicator-master.opt (+1/-1)
plugin/transaction_log/tests/t/blob-master.opt (+1/-1)
plugin/transaction_log/tests/t/bug660779-master.opt (+1/-1)
plugin/transaction_log/tests/t/create_select-master.opt (+1/-1)
plugin/transaction_log/tests/t/create_table-master.opt (+1/-1)
plugin/transaction_log/tests/t/insert_on_duplicate_update-master.opt (+1/-1)
plugin/transaction_log/tests/t/insert_select-master.opt (+1/-1)
plugin/transaction_log/tests/t/multi_column_primary_key-master.opt (+1/-1)
plugin/transaction_log/tests/t/multi_table-master.opt (+1/-1)
plugin/transaction_log/tests/t/no_modification-master.opt (+1/-1)
plugin/transaction_log/tests/t/null_values-master.opt (+1/-1)
plugin/transaction_log/tests/t/rand-master.opt (+1/-1)
plugin/transaction_log/tests/t/rename-master.opt (+1/-1)
plugin/transaction_log/tests/t/replace-master.opt (+1/-1)
plugin/transaction_log/tests/t/rollback-master.opt (+1/-1)
plugin/transaction_log/tests/t/savepoint-master.opt (+1/-1)
plugin/transaction_log/tests/t/schema-master.opt (+1/-1)
plugin/transaction_log/tests/t/select_for_update-master.opt (+1/-1)
plugin/transaction_log/tests/t/slap-master.opt (+1/-1)
plugin/transaction_log/tests/t/temp_tables-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_alter-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_create-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_data_type-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_delete-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_drop-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_large_blob-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_loaddata-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_replace-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_schema-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_transaction-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_update-master.opt (+1/-1)
plugin/transaction_log/tests/t/truncate_log-master.opt (+1/-1)
plugin/transaction_log/tests/t/udf_print_transaction_message-master.opt (+1/-1)
plugin/transaction_log/tests/t/update-master.opt (+1/-1)
plugin/transaction_log/tests/t/variables-master.opt (+1/-1)
lp:~linuxjedi/drizzle/trunk-tx-log
David Shrewsbury (community): Approve
Diff: 1972 lines (+584/-372)
69 files modified
drizzled/drizzled.cc (+5/-3)
drizzled/drizzled.h (+2/-0)
drizzled/session.h (+0/-1)
drizzled/sys_var.cc (+2/-2)
drizzled/sys_var.h (+11/-0)
drizzled/transaction_services.cc (+6/-6)
plugin/information_schema_dictionary/tests/r/tables.result (+3/-0)
plugin/innobase/dict/create_replication.cc (+151/-10)
plugin/innobase/handler/ha_innodb.cc (+66/-4)
plugin/innobase/handler/replication_dictionary.cc (+10/-5)
plugin/innobase/include/create_replication.h (+10/-1)
plugin/innobase/include/read_replication.h (+2/-0)
plugin/innobase/include/srv0srv.h (+3/-0)
plugin/innobase/include/sync0sync.h (+2/-0)
plugin/innobase/include/trx0sys.h (+12/-54)
plugin/innobase/include/trx0trx.h (+10/-0)
plugin/innobase/include/trx0trx.ic (+11/-0)
plugin/innobase/log/log0recv.cc (+0/-5)
plugin/innobase/pars/pars0pars.cc (+6/-3)
plugin/innobase/srv/srv0srv.cc (+11/-0)
plugin/innobase/tests/r/innodb-system-table-view.result (+13/-6)
plugin/innobase/tests/r/innodb_replication_log.result (+19/-0)
plugin/innobase/tests/t/innodb_replication_log-master.opt (+1/-0)
plugin/innobase/tests/t/innodb_replication_log.test (+14/-0)
plugin/innobase/trx/trx0sys.cc (+20/-184)
plugin/innobase/trx/trx0trx.cc (+10/-11)
plugin/schema_dictionary/tests/r/data_dictionary.result (+35/-7)
plugin/transaction_log/tests/t/alter-master.opt (+1/-1)
plugin/transaction_log/tests/t/auto_commit-master.opt (+1/-1)
plugin/transaction_log/tests/t/bad_replicator-master.opt (+1/-1)
plugin/transaction_log/tests/t/blob-master.opt (+1/-1)
plugin/transaction_log/tests/t/bug660779-master.opt (+1/-1)
plugin/transaction_log/tests/t/create_select-master.opt (+1/-1)
plugin/transaction_log/tests/t/create_table-master.opt (+1/-1)
plugin/transaction_log/tests/t/insert_on_duplicate_update-master.opt (+1/-1)
plugin/transaction_log/tests/t/insert_select-master.opt (+1/-1)
plugin/transaction_log/tests/t/multi_column_primary_key-master.opt (+1/-1)
plugin/transaction_log/tests/t/multi_table-master.opt (+1/-1)
plugin/transaction_log/tests/t/no_modification-master.opt (+1/-1)
plugin/transaction_log/tests/t/null_values-master.opt (+1/-1)
plugin/transaction_log/tests/t/rand-master.opt (+1/-1)
plugin/transaction_log/tests/t/rename-master.opt (+1/-1)
plugin/transaction_log/tests/t/replace-master.opt (+1/-1)
plugin/transaction_log/tests/t/rollback-master.opt (+1/-1)
plugin/transaction_log/tests/t/savepoint-master.opt (+1/-1)
plugin/transaction_log/tests/t/schema-master.opt (+1/-1)
plugin/transaction_log/tests/t/select_for_update-master.opt (+1/-1)
plugin/transaction_log/tests/t/slap-master.opt (+1/-1)
plugin/transaction_log/tests/t/temp_tables-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_alter-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_create-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_data_type-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_delete-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_drop-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_large_blob-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_loaddata-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_replace-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_schema-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_transaction-master.opt (+1/-1)
plugin/transaction_log/tests/t/transaction_log_update-master.opt (+1/-1)
plugin/transaction_log/tests/t/truncate_log-master.opt (+1/-1)
plugin/transaction_log/tests/t/udf_print_transaction_message-master.opt (+1/-1)
plugin/transaction_log/tests/t/update-master.opt (+1/-1)
plugin/transaction_log/tests/t/variables-master.opt (+1/-1)
plugin/transaction_log/utilities/transaction_log_connection.cc (+6/-5)
plugin/transaction_log/utilities/transaction_log_connection.h (+1/-1)
plugin/transaction_log/utilities/transaction_reader.cc (+99/-27)
tests/r/information_schema.result (+2/-0)
tests/suite/regression/r/654219.result (+4/-0)
Joe Daly (skinny.moey)
Changed in drizzle:
importance: Undecided → Medium
tags: added: replication
Andrew Hutchings (linuxjedi)
Changed in drizzle:
assignee: nobody → Andrew Hutchings (linuxjedi)
Revision history for this message
Andrew Hutchings (linuxjedi) wrote :

I assume by "gpb message size" you mean transaction-message-threshold? If so, here is the thing, that variable is a session level variable but you are outputting data from a global scope. So do we use the session level variable or the global level? Both could truncate messages based on user settings.

Changed in drizzle:
status: New → Incomplete
Revision history for this message
David Shrewsbury (dshrews) wrote :

--transaction-message-threshold should not be settable at the session level. This is a bug and could cause all sorts of chaos.

Andrew Hutchings (linuxjedi)
Changed in drizzle:
status: Incomplete → Triaged
Revision history for this message
Joe Daly (skinny.moey) wrote :

If --transaction-message-threshold was made global in scope and read only (settable at startup only) that should solve the problem of what size to make the DD field. The message threshold has low limits so I dont see a need to truncate at all.

Revision history for this message
Andrew Hutchings (linuxjedi) wrote :

Can't merge my branch until bug #646898 is fixed. Transaction log table takes up too much RAM for our slicehost boxes with it in.

Put on 'In Progress' until bug #646898 is fixed.

Revision history for this message
Andrew Hutchings (linuxjedi) wrote :

So, new plan:

Expose internal SYS_REPLICATION_LOG table for the binary data and use the DD just for string representation of the data. This also fixes several other problems and will greatly improve performance accessing the binary data.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Related blueprints

Remote bug watches

Bug watches keep track of this bug in other bug trackers.