Testing: call tempest function from tox

Registered by Michael Gugino on 2016-09-29

Currently, we are deploying tempest into a container and executing tempest by instructing ansible to issue a shell command. This works, and may be appropriate for the AIO and tempest roles, but is not optimal for individual role repository testing.

I propose we install tempest on the local host (via ansible in the usual way), but then have another command in tox that actually executes the tempest check directly. This will produce far better logs for examining tracebacks when tempest fails.

Potential negative impact: There will be more to maintain in each role's tox.ini, but this should be offset by less to maintain in each role's plays.

Alternative: Log tempest results somewhere and cat them out at the end of a failed run.

Blueprint information

Status:
Not started
Approver:
None
Priority:
Undefined
Drafter:
Michael Gugino
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

My alternative suggestion is to implement something like https://review.openstack.org/371753 inside the tempest role, with the execution enabled by a var. The role can then optionally also pull the results back to the deployment host in a file which we can upload with the rest of the test resulting logs.

I think uploading the results into logs is a good approach. I think should call tempest from the roles directly, via ansible or otherwise, so we don't have to do too much templating of the tempest role.

Gerrit topic: https://review.openstack.org/#q,topic:bp/testing-direct-tempest,n,z

Addressed by: https://review.openstack.org/382211
    [WIP] Execute tempest using updated method

Gerrit topic: https://review.openstack.org/#q,topic:382193,n,z

Addressed by: https://review.openstack.org/387906
    Execute tempest using updated method

Addressed by: https://review.openstack.org/398923
    Using updated tempest method for neutron.

Addressed by: https://review.openstack.org/398915
    Using updated tempest method for ceilometer.

Addressed by: https://review.openstack.org/398910
    Using updated tempest method for aodh api.

Addressed by: https://review.openstack.org/398884
    Using updated tempest method for cinder.

Addressed by: https://review.openstack.org/398987
    Using updated tempest method for keystone.

Addressed by: https://review.openstack.org/398838
    Using updated tempest method for trove api.

Addressed by: https://review.openstack.org/398916
    Using updated tempest method for horizon.

Addressed by: https://review.openstack.org/398846
    Using updated tempest method for designate api.

Addressed by: https://review.openstack.org/370638
    [WIP] Add multiple scenarios for testing

Addressed by: https://review.openstack.org/408734
    Execute tempest using updated method

Addressed by: https://review.openstack.org/408738
    Execute tempest using updated method

Addressed by: https://review.openstack.org/466670
    Execute tempest using updated method

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.