Use JSONSchema as our only JSON validation tool

Registered by Vincent Fran├žoise on 2016-11-18

As of now in Watcher both jsonschema and voluptuous are used to validate JSON payloads. However, the problem with voluptuous is that its structure is not standardized compare to jsonschema which means that we cannot easily expose the validation schema through our API.

As of now, voluptuous is used to:

- validate the input parameters of a Watcher action when it gets to the Applier.
- validate the efficacy indicators

Here below is an explanation of the problem from a discussion
(17:15:30) vfrancoise: https://blueprints.launchpad.net/watcher/+spec/jsonschema-validation this one is similar
(17:19:36) Yumeng__: So we need to change existed voluptuous to jsonschema
(17:19:44) vfrancoise: yes
(17:20:10) vfrancoise: because voluptuous is not really serializable
(17:20:32) vfrancoise: let me show you what bothers me in the code that has to be fixed
(17:20:51) Yumeng__: Ok, greate
(17:21:46) vfrancoise: a good definition of done is that we should be able to remove the voluptuous dependency once you've finished this BP
(17:21:59) vfrancoise: but there is one tricky part
(17:23:39) vfrancoise: give me a sec I can't remember where it is :p
(17:24:35) Yumeng__: No problem. Take your time
(17:27:35) vfrancoise: ok so that here https://github.com/openstack/watcher/blob/master/watcher/decision_engine/goal/efficacy/base.py#L65-L74
(17:29:05) vfrancoise: basically, we are collecting the voluptuous schemas from all the efficacy indicators to form one big schema that is used to validate all the inputs
(17:32:56) Yumeng__: So now we need to use jsonschema to replace the big volupous scheme?
(17:36:17) vfrancoise: yes, you need to find a way to combine many jsonschema into a big one
(17:38:49) Yumeng__: But why we need to combine schemas into a one big scheme to validate all the inputs?
(17:39:09) vfrancoise: because we want to expose it via the API later on
(17:39:41) vfrancoise: efficacy specifications are part of the /goal API
(17:40:32) vfrancoise: look at https://github.com/openstack/watcher/blob/master/watcher/api/controllers/v1/goal.py#L108-L113
(17:41:35) vfrancoise: the serialization of the schema only works because it's a valid python string so we can interprete it but that's really not a good practice
(17:41:40) Yumeng__: So the big schema is like a interface, and for a specific efficacy indicator we validate it using a specific schema in the big schema?
(17:41:54) vfrancoise: also the idea is to simplify the code so we only use jsonschema for validation
(17:43:00) vfrancoise: yes you got it
(17:43:33) vfrancoise: we do this serialization so we don't introduce a strong coupling between the watcher API and the watcher Applier
(17:43:43) vfrancoise: so we use the schema as a validation contract

Blueprint information

Status:
Complete
Approver:
Antoine Cabot
Priority:
Low
Drafter:
Vincent Fran├žoise
Direction:
Approved
Assignee:
YumengBao
Definition:
Approved
Series goal:
Accepted for pike
Implementation:
Implemented
Milestone target:
milestone icon pike-3
Started by
YumengBao on 2017-03-03
Completed by
YumengBao on 2017-06-22

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/jsonschema-validation,n,z

Addressed by: https://review.openstack.org/425204
    Remove voluptuous and Use JSONSchema as our only JSON validation tool

Addressed by: https://review.openstack.org/471327
    Replace voluptuous with JSONSchema to validate migration action

Addressed by: https://review.openstack.org/472165
    Replace voluptuous with JSONSchema to validate migration action

Addressed by: https://review.openstack.org/472218
    Replace voluptuous with JSONSchema to validate sleep action

Addressed by: https://review.openstack.org/472508
    Replace voluptuous with JSONSchema to validate resize action

Addressed by: https://review.openstack.org/472519
    Replace voluptuous with JSONSchema to validate nop action

Addressed by: https://review.openstack.org/481130
    Replace voluptuous with JSONSchema to validate change_node_power_state

Addressed by: https://review.openstack.org/486836
    Replace voluptuous with JSONSchema in BaseAction

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.