SSD as L2 cache

Registered by Vadim Tkachenko

Allow to use SSD as secondary cache to buffer pool for InnoDB.

The idea is to use space on SSD as secondary (L2) cache, in additional to primary buffer_pool (L1) cache.

We pre-allocate space on SSD, which is used to store InnoDB pages.

Read logic is:

If page is missed in L1:
 - check if page in L2. if yes - read from L2 into L1 ( + mark as recently used, or perform regular LRU logic )
   If not:
 - read page to L1 from file ( regular logic)
 - schedule read of page into L2 in background thread ( Q: is it needed ? can we read in working thread without latency penalty)

Write logic:
when page from L1 is flushed we can use one of scenarios (can implement both and make as option to choose)
- write invalide ( just remove page from L2)
- write through ( write page to tablespace AND to L2)

Eviction policy for L2 (what page to delete if we need to read new page).
It should be based on current LRU logic with mid-point insertion as it is implemented in InnoDB-plugin-1.0.6

Additional comments:
1) we may have more than one device as a cache (which can be just concatenated for our sake)
2) If One of cache devices fails it should be removed from cache while second works. If both fail the system should continue working without cache.
3) The cache is WT. Discarded on reboot. This allows to enable cache on SSD drives and not to mess with changing Innodb recovery.
4) We can allocate enough memory to keep system meta data on L2 cache. So there is no IO to know if page is in cache etc.
5) We should keep cache inclusive. The pages in L1 cache should not be removed from L2 cache unless they become dirty (and so invalid)
6) We need to migrate pages to L2 in background with write throttling. It is possible we may not be able to be fast enough to migrate all pages to L2 cache it should not cause issues.
7) If page is modified in BP we need to make sure it is not read from L2 any more so it needs to be invalidated.
7) When page is flushed from BP it becomes clean and at this point it can be scheduled for being written to L2. Note I would avoid writing it at once as there is high posibility it will become dirty again. The solution (for all pages) could be to schedule pages for migration to L2 only when they made it to "old" sublist in LRU and they are clean. If they are in Young list they are not likely to be replaced soon and there is no rush to put them to L2 yet.

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Vadim Tkachenko
Direction:
Needs approval
Assignee:
None
Definition:
Obsolete
Series goal:
None
Implementation:
Deferred
Milestone target:
None
Completed by
Vadim Tkachenko

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.