Improve API attribute test coverage for Nova API
Now Tempest does not check API responses of Nova API in many cases.
For example, Tempest does not check what API attributes(
etc.) should be included in the response body of "create a server" API.
So we need to improve Tempest coverage from this viewpoint for blocking
any backward incompatibility change happens.
The tasks are managed on https:/
Blueprint information
- Status:
- Complete
- Approver:
- Sean Dague
- Priority:
- Not
- Drafter:
- Ken'ichi Ohmichi
- Direction:
- Needs approval
- Assignee:
- Ken'ichi Ohmichi
- Definition:
- Superseded
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
- Matthew Treinish
Related branches
Related bugs
Sprints
Whiteboard
DESIGN
Tempest validates a response body with JSONSchema.
Tempest already contains a request body schema for the negative test generator.
A response body schema also is defined in the same schema file.
PROCESS
As the first step, we need to cleanup many Nova v2/v3 API tests with blueprint nova-api-
After that, we will add this API attribute checks to clean v2/v3 API tests.
Gerrit topic: https:/
Addressed by: https:/
Verify service attributes through Nova "get services" API
Addressed by: https:/
Skip Nova API attribute tests if an XML response
Addressed by: https:/
Verify server attributes which admin can get through Nova APIs
Addressed by: https:/
Validate Volume attributes of Nova POST & GET API
Addressed by: https:/
Verify the list volume attributes of Nova APIs
Addressed by: https:/
JSON Schema validation for attach volume attribute
Addressed by: https:/
Verify service attributes through Nova os-flavor-access API
Addressed by: https:/
Verify "update a server" API response attributes
Addressed by: https:/
Verify service attributes through Nova os-quota-sets API
Addressed by: https:/
Making 'body' param optional for resp validation
Addressed by: https:/
Verify "get a server" API response attributes
Addressed by: https:/
Verify "get az list" API response attributes
Addressed by: https:/
Validate of Nova APIs Floating IPs attributes
Addressed by: https:/
Move verification of service attributes into service client
Addressed by: https:/
Verify Nova create & get Floating IP attributes
Addressed by: https:/
Check attach-detach FIP & get FIP pool attributes
Addressed by: https:/
Verify get flavor attributes of Nova V2/V3 APIs
Addressed by: https:/
Validate for list flavor attributes of V2/V3 APIs
Addressed by: https:/
Verify detail_list flavor attributes of V2/V3 APIs
Addressed by: https:/
Verify Image attributes through Nova V2 GET API
Addressed by: https:/
Validate get fixed-ips attributes of Nova V2 API
Addressed by: https:/
Verify the response attributes of 'list_hosts'
Addressed by: https:/
Verify list Image attributes through Nova V2 API
Addressed by: https:/
Check detail list Images attributes of Nova V2 API
Addressed by: https:/
Validate get keypair attributes of Nova V2/V3 API
Addressed by: https:/
Verify list agents attributes of V2/V3 APIs
Addressed by: https:/
Validate list_keypair attribute of Nova V2/V3 APIs
Addressed by: https:/
Verify the response attributes of 'show_host_detail'
Addressed by: https:/
Verify "get version" API response attributes
Addressed by: https:/
Add os-migration tests for Nova v2 API
Addressed by: https:/
Validate get limits attributes of Nova V2 API
Addressed by: https:/
Check show_hypervisor attributes of Nova V2/V3 API
Addressed by: https:/
Verify list_hypervisor
Addressed by: https:/
Validate hypervisors_servers Nova V2/V3 API
Addressed by: https:/
Verify quotas attributes through Nova "get/get default quotas" API
Addressed by: https:/
Check list_hypervisor attributes of Nova V2/V3 API
Addressed by: https:/
Verify "enable a service" API response attributes
Addressed by: https:/
Verify security_groups attributes through Nova "list security groups" API
Addressed by: https:/
Check hypervisor statistics attributes of Nova API
Addressed by: https:/
Verify hypervisor uptime attributes of Nova API
Addressed by: https:/
Verify certificate API response attributes
Addressed by: https:/
Verify the response attributes of 'list_aggregates'
Addressed by: https:/
Verify the response status of create delete Image
Addressed by: https:/
Validate image metadata attributes of Nova APIs
Addressed by: https:/
Verify "get quotas detail" API response attributes
Addressed by: https:/
Check attributes of image meta item Nova APIs
Addressed by: https:/
Verify tenant usages API response attributes
Addressed by: https:/
Define 'links' as a common parameter type
Addressed by: https:/
Check create/delete keypair attribute of Nova APIs
Addressed by: https:/
Verify attributes through Nova os-quota-sets API
Addressed by: https:/
Check reserve/unreserve fixed-ips APIs attributes
Addressed by: https:/
Verify the response attributes of 'get_aggregate'
Addressed by: https:/
Verify flavor extra specs attributes of Nova APIs
Addressed by: https:/
Verify list extensions attributes of V2/V3 APIs
Addressed by: https:/
Verify attributes through Nova os-security-groups API
Addressed by: https:/
Verify the response attributes of 'startup_host' API
Addressed by: https:/
Verify the create/delete volume APIs attributes
Addressed by: https:/
Verify "list interfaces" Nova v2/v3 API response attributes
Addressed by: https:/
Validate get_instance_
Addressed by: https:/
Validate list_instance_
Addressed by: https:/
Verify the response attributes of 'shutdown/
Addressed by: https:/
Add a response header validation
Addressed by: https:/
Verify v2 os-migration API response attributes
Addressed by: https:/
Verify "show/create interface" Nova v2/v3 API response attributes
Addressed by: https:/
Verify the response attributes of 'create-aggregate' API
Addressed by: https:/
Verify the response attributes of 'update-aggregate' API
Addressed by: https:/
Check add/remove flavor access APIs attributes
Addressed by: https:/
Check attributes of create/delete sec groups rule"
Addressed by: https:/
Verify create agent attributes of V2/V3 APIs
Addressed by: https:/
Check attributes of change/get server password API
Addressed by: https:/
Verify delete agent response of V2/V3 APIs
Addressed by: https:/
Check get_vnc_console attributes of Nova APIs
Addressed by: https:/
Verify the response attributes of 'aggregate-
Addressed by: https:/
validate server action 'console_output' attributes
Addressed by: https:/
Verify update agent attributes of V2/V3 APIs
Addressed by: https:/
Verify the response attributes of 'aggregate-
Addressed by: https:/
Verify delete quota response of V2/V3 APIs
Addressed by: https:/
Verify the response attributes of 'aggregate-
Addressed by: https:/
Verify Server Actions attributes of Nova APIs
Addressed by: https:/
Check rescue_server action attributes of Nova APIs
Addressed by: https:/
Verify list_virtual_
Addressed by: https:/
Verify delete interface response of V2/V3 APIs
Addressed by: https:/
Verify delete security group response of V2 API
Addressed by: https:/
Verify list_instance_
Addressed by: https:/
Verify get_instance_action attributes of Nova API
Addressed by: https:/
Verify the response attributes of 'set-server-
Addressed by: https:/
Verify Set/Get/Delete server meta item attributes
Addressed by: https:/
Verify set server-metadata response of V2/V3 API
Addressed by: https:/
Verify delete server attributes of Nova API
Addressed by: https:/
Check list servers attributes of Nova APIs
Addressed by: https:/
Verify delete aggregate attributes of Nova APIs
Addressed by: https:/
Verify 'list-server-
Addressed by: https:/
Verify update host Nova V2/V3 APIs attributes
Addressed by: https:/
Separate common 'start_up' definition from v2 schema
Addressed by: https:/
Verify server list_addresses V2/V3 APIs attributes
Addressed by: https:/
Verify list_addresses_
Addressed by: https:/
Check create/get/delete V2 server_group attributes
Addressed by: https:/
Verify V2 list server_group API attributes
Addressed by: https:/
Fix update_server JSON Schema for Nova V2
Addressed by: https:/
Check V2 rebuild server action API attributes
Addressed by: https:/
Verify 'update-
Addressed by: https:/
make 'server' as 'required' in server update schema
Addressed by: https:/
Correcting 'id' type in compute schema
Addressed by: https:/
Validate server detail list attribute of Nova APIs
Addressed by: https:/
Check create-
Addressed by: https:/
Verify list-Floating-
Addressed by: https:/
Verify delete-
Addressed by: https:/
Separate create server schema for admin password
Work Items
Dependency tree
* Blueprints in grey have been implemented.
