Add support for retargetable functional testing

Registered by Maru Newby on 2014-07-11

The current Neutron unit test suite contains a large number of tests that are actually functional - they exercise almost the full application stack through the Neutron REST API. There is also large overlap between these tests and the tests of the Neutron API managed in the Tempest repo. This blueprint proposes to refactor tests in both trees so that there can be one set of 'retargetable' API tests written to an abstract client interface. By varying the concrete client implementation, it should be possible for a given retargetable API test to run in the Neutron test suite against the programmatic plugin api, and for the same test to run in the Tempest test suite against a live Neutron service.

Blueprint information

Status:
Complete
Approver:
Kyle Mestery
Priority:
High
Drafter:
None
Direction:
Approved
Assignee:
None
Definition:
Obsolete
Series goal:
None
Implementation:
Needs Code Review
Milestone target:
None
Started by
Kyle Mestery
Completed by
Armando Migliaccio

Related branches

Sprints

Whiteboard

March-17 (mestery): Moving to Liberty, we can merge this one once it's ready and we cut the RC branch.

Jan-22 (maru): We're making progress on this initiative, but the scope is large enough (making it possible to run api tests in tree and migrating existing api tests from tempest) that I'm retargeting to kilo-3.

Jan-22 (ajo): Seems like all the reviews are merged.

December-3 (mestery): Kilo-2.

September-15 (mestery): Per discussion on the review, this is now moved out of Juno. Maru, please ping me if you think this can still make Juno this week yet.

September-2 (mestery): Per discussion with Maru, this won't make Juno-3 and will need an FFE to make Juno RC1.

16-July (mestery): Approved for Juno, marking Juno-3!

Gerrit topic: https://review.openstack.org/#q,topic:bp/retargetable-functional-testing,n,z

Addressed by: https://review.openstack.org/106290 (Merged)
    Add spec for retargetable functional testing

Addressed by: https://review.openstack.org/82775 (Merged)
    Refactor plugin setup helpers out of test.base

Addressed by: https://review.openstack.org/72585
    Add support for retargetable functional api testing

Addressed by: https://review.openstack.org/135428
    Refresh spec for retargetable functional testing

Addressed by: https://review.openstack.org/136099
    Use oslo function for parsing bool from env var

Addressed by: https://review.openstack.org/136100
    Fix base test class for functional api testing

Addressed by: https://review.openstack.org/145387
    Fix outstanding failures with Neutron API job

Addressed by: https://review.openstack.org/148408
    Switch to using abc in the retargetable client

Addressed by: https://review.openstack.org/158021
    WIP Initial copy of api tests from tempest

Addressed by: https://review.openstack.org/162257
    Initial copy of api tests from tempest

Addressed by: https://review.openstack.org/162720
    Fix tempest api test copy script

Addressed by: https://review.openstack.org/167075
    Simplify retargetable test framework

Addressed by: https://review.openstack.org/167041
    Simplify base test cases

Addressed by: https://review.openstack.org/167320
    Move API tests to neutron.test.api

Addressed by: https://review.openstack.org/168430
    Remove neutron.tests.sub_base

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.