Testing of the drizzle transaction log

The main focus of this testing is to be certain that the transaction log is rock-solid. We will move on to other aspects of replication once this feature has a healthy set of tests established.

This will include testing with the randgen, which will be large-scale random testing.
At the moment the plan is:
* Grammar which generates DDL against a decent population of tables, using all available data types
* Validator which will verify the transaction log captures said DDL correctly
randgen testing may change as implementation occurs.

This will also include adding tests to the main test suite
There are currently a number of tests in this tree's plugin/transaction_log directory
https://code.launchpad.net/~jaypipes/drizzle/publisher

We will look at adapting these tests for use with the standard test runner, which will include functionality for tests where the server should not start (ie bad transaction log, bad command lines, etc)

UPDATE (7/7/2010) - further testing / randgen test development is on-hold while the current batch of bugs is worked on. Some test development may occur, but major testing is waiting at the moment.

We have created two sets of randgen tests for the transaction log

* Single threaded.
We have grammars set up so that each 'query' of the randgen means 10-30 queries have been sent to the database. Once the randgen has complete a 'query', the Validator (DrizzleTransactionLog) will attempt to generate SQL from the transaction log's contents (via drizzled/message/transaction_reader) and then send it to a validation server (another instance of Drizzle). After that, we call drizzledump on both servers and compare dump output.

* Multi-user
Here we use the same grammars as in single-threaded, but each 'query' of the randgen means one query is sent to the server. We let each thread execute their queries until completion and then we repeat the validation process used in the single-threaded scenario. We have to do things this way so that we don't have to worry about coordinating validation with the work of multiple threads. We use a Reporter vs. a Validator here (DrizzleConcurrentTransactionLog).

grammars:
 conf/drizzle/transaction_log_concurrent2_drizzle.yy
 conf/drizzle/transaction_log_concurrent_drizzle.yy
 conf/drizzle/transaction_log_single_thread2_drizzle.yy
 conf/drizzle/transaction_log_single_thread_drizzle.yy

gendata file:
conf/drizzle/drizzle.zz

You must have a validation server running. The Validator and Report are hard-coded with 9307 as the value, but this can be changed via the $validator_port variable. This is a bit hackish, but randgen doesn't currently provide convenient infrastructure for easily controlling other servers for purposes such as this (and it works in a pinch). The files are in randgen/lib/GenTest/Validator and randgen/lib/GenTest/Reporter.

The main server must be started with --transaction-log.enable

Example command line:
Single-user:
perl ./gentest.pl --dsn=dbi:drizzle:host=localhost:port=9306:user=root:password="":database=test --gendata=conf/drizzle/drizzle.zz --grammar=conf/drizzle/transaction_log_concurrent_drizzle.yy --queries=100 --threads=1 --debug --Validator=DrizzleTransactionLog

Concurrent:
perl ./gentest.pl --dsn=dbi:drizzle:host=localhost:port=9306:user=root:password="":database=test --gendata=conf/drizzle/drizzle.zz --grammar=conf/drizzle/transaction_log_concurrent_drizzle.yy --queries=1 --threads=2 --debug --Reporter=DrizzleConcurrentTransactionLog

NOTE: Testing is still going on, but the tests that were to be created for this blueprint have been implemented, so we are considering things complete. All that remains is some bug-hunting runs + testing of bug fixes.

https://blueprints.launchpad.net/drizzle/+spec/automate-transaction-log-tests covers adding these tests to our automated system.