Convert tempest from nose to testtools to speed it up

Registered by Sean Dague

remove nose from tempest and use testtools/testr instead

Blueprint information

Status:
Complete
Approver:
David Kranz
Priority:
Essential
Drafter:
None
Direction:
Approved
Assignee:
Matthew Treinish
Definition:
Approved
Series goal:
Accepted for havana
Implementation:
Implemented
Milestone target:
milestone icon havana-3
Started by
Jay Pipes
Completed by
Matthew Treinish

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/speed-up-tempest,n,z

Addressed by: https://review.openstack.org/19033
    ensure isolated test cases run with an isolated tenant

Addressed by: https://review.openstack.org/19544
    Convert these tests of ServerBasicOps into one test.

Addressed by: https://review.openstack.org/19548
    ensure setup_test_user has been called before using test_user

Addressed by: https://review.openstack.org/20013
    Ensure package-wide test init is done with testr

Addressed by: https://review.openstack.org/20364
    Use testtools as the base testcase class.

Addressed by: https://review.openstack.org/21411
    Convert to use testtools attr implementation 1/3

Addressed by: https://review.openstack.org/21413
    Convert to use testtools attr implementation 3/3

Addressed by: https://review.openstack.org/21412
    Convert to use testtools attr implementation 2/3

Addressed by: https://review.openstack.org/32204
    Use python-fixtures for image resource tracking in glance tests.

Addressed by: https://review.openstack.org/36702
    Switch to using testr as the test runner for everything non-gating.

Addressed by: https://review.openstack.org/37757
    Use subunit colorizer from nova for run_tests.sh

Addressed by: https://review.openstack.org/37758
    Add unittest like output for testr-full in tox

Addressed by: https://review.openstack.org/37785
    Remove unneeded class filter from .testr.conf

Addressed by: https://review.openstack.org/38203
    Add isolated tenants for admin tests.

Gerrit topic: https://review.openstack.org/#q,topic:admin_isolation,n,z

Addressed by: https://review.openstack.org/38856
    Sync lockutils from oslo.

Addressed by: https://review.openstack.org/38857
    Add locking to test_aggregates

Addressed by: https://review.openstack.org/38879
    Switch to using oslo logging.

Addressed by: https://review.openstack.org/39099
    Add locking for adding a ssh security group rule

Addressed by: https://review.openstack.org/39539
    Add more locks for hosts and availability zones

Addressed by: https://review.openstack.org/40975
    Rework class inheritance for scenario tests

Addressed by: https://review.openstack.org/41497
    Switch use of select() to poll()

Addressed by: https://review.openstack.org/42169
    Add tenant isolation to the swift tests

Addressed by: https://review.openstack.org/42351
    Remove identity race condition

Gerrit topic: https://review.openstack.org/#q,topic:switch-to-testr-parallel,n,z

Addressed by: https://review.openstack.org/41797
    Switch gating tox jobs to testr parallel

(?)

Work Items

Work items:
Investigate test failures when testr is run is serial mode: DONE
Investigate test failures when testr is run in parallel mode: DONE
Investigate why testr finds fewer tests than nose: DONE
Remove nose dependencies: DONE
unittests to testtools conversion: DONE
Confirm all SKIPS we're seeing with testr are valid: DONE
Need testr/testtools equivalent of smoke tests attribute: DONE
Patch testr to add group scheduling: DONE
Debug race conditions and failures when using testr: DONE
Move run_tests and tox over to testr: DONE

This blueprint contains Public information 
Everyone can see this information.