Searchable Metadata
Goal: Provide a service where objects and containers can be listed by a search key matching names and other metadata.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- None
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- New
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
Wiki page: https:/
Include proposed API spec (PDF), Icehouse design summit slides, Etherpad link, and accumulated info briancline and I will collect from community feedback in the openstack-dev email list on this topic.
-lincolnt
--
This was discussed at the Icehouse summit in a design session on 7 November 2013. Lincoln Thomas at HP published the above linked spec to foster discussion and feedback from the community. SoftLayer, who currently uses a custom middleware for its metadata search capabilities, also wishes to participate in moving this blueprint forward.
Based on feedback during the design summit, there is definitely more than enough interest in the community at large in seeing a metadata search API implemented. As mentioned during the session, HP and SoftLayer hope to get broader community feedback and consensus on the proposed API prior to implementation.
-briancline
--
There are at least 2 ways to implement this API:
1) an external service (could possible be implemented in swift middleware) that reads metadata in data ingress and stores it in a searchable KV store that includes system and custom metadata for accounts, containers, and objects. This is relatively simple, and it doesn't handle existing data.
2) an internal service that adds functionality to swift to create (and delete) indexes on the account/container DBs, and adds an object DB for its metadata to be searchable too (which could be huge). This is more complicated, but it requires less external dependencies and works with existing data.
This design / impl info will be expanded in the Wiki page.
-gholt / -lincolnt
Work Items
Work items:
Post full proposed API spec from Lincoln: DONE
Post slides from Icehouse design summit from Lincoln: DONE
Agree on the API, from consensus of Stackers' feedback: INPROGRESS
Define method for backfilling/
Define reference implementation: TODO
Define the metadata store to be used for the reference implementation, with pluggable interfaces for vendors to supersede with their own: TODO
