trafodion and snapshot scan integration
This blueprint describes the integration of Trafodion with the HBase snapshot scan (TableSnapshotS
The snapshot scan was first implemented in Trafodion to work with Bulk unload and then more changes were made to make it work independently. The current implementation integrates the snapshot scan with Trafodion and also sets up the temporary space and folders before running the query. Once the query is completed the temporary space and folders are cleaned up.
In order to use snapshot scan with Trafodion we need to issue the below CQDs:
*TRAF_TABLE_
This CQD can be set to :
**NONE: (default)Snapshot scan is disabled and regular scan is used ,
**SUFFIX: Snapshot scan is enabled for the bulk unload.
**LATEST: Snapshot Scan is enabled independently from bulk unload and the latest snapshot is used if it exists. If no snapshot exists the regular scan is used and a warning is issued. For this phase of the project the user needs to create the snapshots using hbase shell or other tools. And in the next phase of the project new commands to create, delete and manage snapshots will be add. Snapshots are cached in NATable to optimize compilation time. when the user sets TRAF_TABLE_
There are some cases where snapshot scan is not supported yet in spite of setting TRAF_TABLE_
*** There is no snapshot associated with the table we are trying to scan.
***The optimizer chooses to use an index table instead of the the base table.
***The table is smaller than the threshold defined by TRAF_TABLE_
*TRAF_TABLE_
This CQD is used with bulk unload and its value is used to build the snapshot name as the table name followed by the suffix string
*TRAF_TABLE_
When the estimated table size is below the threshold (in MBs) defined by this CQD the regular scan is used instead of snapshot scan and a warning is issued. This CQD does not apply to bulk unload.
*TRAF_TABLE_
The timeout beyond which we give up trying to create the snapshot scanner
*TRAF_TABLE_
Location for temporary links and files produced by snapshot scan. Its default value is curretly set to '/bulkload/'.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- khaled Bouaziz
- Direction:
- Needs approval
- Assignee:
- khaled Bouaziz
- Definition:
- New
- Series goal:
- None
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by