Search Syntax Negotiation

Registered by Travis Tripp on 2015-11-19

Searchlight was very purposely built around the notion that dedicated search technologies such as ElasticSearch have solved many of the requirements for advanced searching across various types of resources. Creating an "OpenStack" common API for search in each project is difficult, time consuming, and likely to never reach the capabilities or performance of dedicated search technology. Therefore, we did not want to try to re-invent the wheel for search syntax, at least initially, but, rather, take advantage of the rich capabilities already out in the world.

All of that said, ElasticSearch is not the only search provider and even it will have different versions of its search syntax that are supported based on the deployed version of ElasticSearch backing it.

This blueprint proposes that Searchlight is capable of supporting search syntax negotiation with as transparent a pass through as possible for that actual syntax. The basics of this were started with the initial ElasticSearch plugin, but the concept has not been fully designed or constructed into the system. Just proving itself with one provider has been the priority, but we need to consider future flexibility before getting too rigid in the code and API design.

For this concept to function best, it would be beneficial if Searchlight could separate the notification listening and base enrichment from the backend that it published the data. Common data filtering configuration would be needed as well.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Low
Drafter:
Travis Tripp
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

[Malini] A little baffled here on search syntax negotiation. If SearchLight (A) and Consumer (B) both use elastic search, and have some version number, the lower of the two would work. But if the Consumer uses non-elastic search syntax, perhaps pure SQL or something even simpler, how would you interpret the query? Best effort? Also denial of service type queries such as "*" you might want to respond with "not supported" msg?

With HTTPS type stuff, the strongest of the client/server common options was selected, or a default chosen. Are you seeking something along these lines?

[TravT] Searchlight has a plugins / resource types listing capability today. I haven't fully designed this out (probably need a spec), but my thought was that each plugin / resource type could list its support syntaxes and that using basic http concepts a given client could get a list of supported syntaxes and send that through either custom headers and / or leveraging Accepts and Content-Type headers (e.g. Content-Type: application/elasticsearch.v2+json). Then when the request comes in, it gets routed to the correct plugin for doing the RBAC injection and sending on to the search back end. If a search syntax comes in that is not supported, it would respond with an appropriate response code (one of 415 Unsupported Media Type, 400 Bad Request, or 406 Not Acceptable).

FWIW, SQL would be **highly** unlikely, but quite possibly Apache Solr or Mongo DB search.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.