Chunk Iterator
Chunking is a fundamental and crucial operation for pt-archiver, pt-table-checksum, pt-table-sync, and other potential tools that access rows. The current code is fragmented and very complex, and no two tools chunk the same (they generate the same chunks but how they get those chunks internally varies widely). There are a few of issues inherited from Maatkit about chunking: http://
I propose we write a chunk iterator that provides a simple interface to the complexities of chunking. Ideally, the chunk iter would do all the work behinds the scenes, but with hints, so a tool can simply $chunk->next(). At the tool level, we just want chunks regardless of table size, zero chunks, chunk index or columns, etc.
Unifying the chunking code would make fixing chunking bugs easier since all tools would be using the same chunking interface. A unified chunk iter would also allow us to add on other features in a nice way, like slave lag throttling, instrumentation, progress, etc.
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- Not
- Drafter:
- None
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Superseded
- Series goal:
- None
- Implementation:
- Deferred
- Milestone target:
- None
- Started by
- Completed by
- Daniel Nichter
Related branches
Related bugs
Sprints
Whiteboard
Chunk iterator has been superseded by https:/