Migrate the negative tests to use auto-gen framework

Registered by David Kranz on 2014-03-04

The framework for generating negative tests from json schemas now exists. This blueprint is to track progress migrating existing negative tests to use that framework. Since the framework is json-only, the existing xml negative tests will remain. Here are the steps to migrate a negative test file:

1. Copy the file xxx_negative.py to xxx_negative_xml.py and remove everything related to JSON
2. Create a new schema in tempest/etc/schemas/.. based on the service type and api being tested
3. Replace xxx_negative.py with a class based on the auto-gen framework.

Note that there are some negative tests that cannot be expressed in the auto-gen framework. In that case, such tests should be preserved in the file being migrated.

Given the state of flux with nova api v2/v3 it would probably be better to focus on the other services. We may find a way to get schemas for nova without having to write all of them in tempest.

Blueprint information

Status:
Complete
Approver:
Matthew Treinish
Priority:
Medium
Drafter:
David Kranz
Direction:
Approved
Assignee:
Marc Koderer
Definition:
Obsolete
Series goal:
Accepted for juno
Implementation:
Implemented
Milestone target:
milestone icon juno-rc
Started by
Matthew Treinish on 2014-10-20
Completed by
Ken'ichi Ohmichi on 2016-04-03

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/autogen-negative-tests,n,z

Addressed by: https://review.openstack.org/73982
    Add multiple negative test generator support

Addressed by: https://review.openstack.org/77823
    Add support for negative tests with admin client

Addressed by: https://review.openstack.org/83063
    Deduplicate negative test calls

Addressed by: https://review.openstack.org/95658
    Add qa spec for bp/autogen-negative-tests

Addressed by: https://review.openstack.org/115258
    Change default size for strings to 1

Addressed by: https://review.openstack.org/113899
    Add functools.wraps on all function decorators

Addressed by: https://review.openstack.org/115273
    Reduce complexity during import phase

Gerrit topic: https://review.openstack.org/#q,topic:autogen,n,z

Gerrit topic: https://review.openstack.org/#q,topic:bug/1375142,n,z

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.