Add common framework to validate every API parameters

Registered by Ken'ichi Ohmichi

Nova has many RESTful APIs, and not all API parameters are completely validated.
We would like to propose an API validation framework to implement comprehensive validation.
The benefits of this framework will be the followings:
* Validate every API parameters.
* Unify both message format and HTTP code of error response, if the same cause.
  ex) ".. is too short.", ".. is too long.", ".. is not integer."
* Clarify the API parameter definitions.
* Clean up codes by merging error handling methods.

This framework target is Nova v3 API, because it is new and the API compatibility issues would not happen.
On Havana release, Nova v3 API is experimental and the API will become generic in Icehouse.
There are two options for common validation feature. The one is WSME validation, the other is JSONSchema validation. Through the Icehouse summit session(https://etherpad.openstack.org/p/icehouse-summit-nova-pecan-wsme), we have gotten a consensus that Nova v3 API will be moved to Pecan + JSONSchema validation.

JSONSchema validation can work for both JSON parameters and XML parameter because
* JSONSchema library validate againste dict data(not JSON data)
* the validation is executed after the deserialization.

Blueprint information

Status:
Complete
Approver:
Russell Bryant
Priority:
Low
Drafter:
Ken'ichi Ohmichi
Direction:
Approved
Assignee:
Ken'ichi Ohmichi
Definition:
Approved
Series goal:
Accepted for icehouse
Implementation:
Implemented
Milestone target:
milestone icon 2014.1
Started by
Ken'ichi Ohmichi
Completed by
Russell Bryant

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/nova-api-validation-fw,n,z

* The framework core for Nova:
  Addressed by: https://review.openstack.org/25358
      API validation framework

* The framework core for Oslo:
  Addressed by: https://review.openstack.org/25884

Addressed by: https://review.openstack.org/34289
    Add API schema for v3 fixed_ips.

Addressed by: https://review.openstack.org/34514
    Add API schema for v3 evacuate.

Addressed by: https://review.openstack.org/34519
    Add API schema for v3 keypairs.

Add some validation features to WSME
* https://bugs.launchpad.net/wsme/+bug/1245795
* https://review.openstack.org/#/c/54249/

Gerrit topic: https://review.openstack.org/#q,topic:bp/nova-v3-api,n,z

Addressed by: https://review.openstack.org/59593
    Add API schema for v3 admin_actions API

Addressed by: https://review.openstack.org/59598
    Add API schema for v3 admin_password API

Addressed by: https://review.openstack.org/59616
    Add API schema for v3 agents API

Addressed by: https://review.openstack.org/61972
    Add API schema for v3 flavor_manage API

Addressed by: https://review.openstack.org/62310
    Add API schema for v3 cells API

Addressed by: https://review.openstack.org/62538
    Add API schema for v3 extended_volumes API

Addressed by: https://review.openstack.org/62849
    Add API schema for v3 quota_classes API

Addressed by: https://review.openstack.org/63321
    Add API schema for v3 flavor_access API

Addressed by: https://review.openstack.org/64093
    Add API schema for v3 remote_consoles API

Addressed by: https://review.openstack.org/64098
    API schema for v3 get_console_output API

Addressed by: https://review.openstack.org/64134
    Add API schema for v3 flavor_manage API

Addressed by: https://review.openstack.org/64241
    Add API schema for v3 multinic API

Addressed by: https://review.openstack.org/64497
    Fix a minor API schema test bug in test_flavors

Addressed by: https://review.openstack.org/64090
    Validate flavor_extraspecs body

Addressed by: https://review.openstack.org/65413
    Add API schema for v3 quota_sets API

Addressed by: https://review.openstack.org/65846
    API schema for v3 console_output API

Addressed by: https://review.openstack.org/65912
    Define common API parameter types

Addressed by: https://review.openstack.org/66238
    Rename API schema modules with removing "_schema"

Addressed by: https://review.openstack.org/66502
    Add API schema for v3 create_backup API

Addressed by: https://review.openstack.org/66505
    Add API schema for v3 reset_server_state API

Addressed by: https://review.openstack.org/66509
    Add API schema for v3 migrate_server API

Addressed by: https://review.openstack.org/66536
    Add API schema for v3 aggregates API

Addressed by: https://review.openstack.org/66537
    Add API schema for v3 hosts API

Addressed by: https://review.openstack.org/66782
    Add a modifiable keyword for API validation

Addressed by: https://review.openstack.org/66783
    Add API schema for v3 access_ips extension

Addressed by: https://review.openstack.org/67373
    Extend API schema for API extensions

Addressed by: https://review.openstack.org/67428
    API schema for v3 server_metadata API

Addressed by: https://review.openstack.org/67781
    Define common "name" parameter for Nova v3 API

Addressed by: https://review.openstack.org/67823
    Add API schema for v3 availability_zone extension

Addressed by: https://review.openstack.org/67824
    Add API schema for v3 servers API

Addressed by: https://review.openstack.org/68338
    Add API schema for v3 block_device_mapping extension

Addressed by: https://review.openstack.org/68558
    Add API schema for v3 multiple_create extension

Addressed by: https://review.openstack.org/68560
    Add API schema for v3 scheduler_hints extension

Addressed by: https://review.openstack.org/68594
    Add API schema for v3 config_drive extension

Addressed by: https://review.openstack.org/68847
    Add API schema for v3 user_data extension

Addressed by: https://review.openstack.org/69316
    Add API schema for v3 security_groups extension

Addressed by: https://review.openstack.org/70353
    Add API schema for v3 attach_interfaces API

Addressed by: https://review.openstack.org/71465
    Generate API sample docs from API schema

Addressed by: https://review.openstack.org/72796
    Add "body=" argument to v3 API unit tests

We're deferring v3 API blueprints to Juno at this point. However, I think we should mark this blueprint as implemented for Icehouse. The framework was implemented. I think the rest of the schemas can be added later, perhaps under a new blueprint for "completing schemas". --russellb

That is a good idea, the blueprint makes the scope clear. A new blueprint is https://blueprints.launchpad.net/nova/+spec/v3-api-schema for Juno. --oomichi

Gerrit topic: https://review.openstack.org/#q,topic:bp/v3-api-schema,n,z

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.