Guarantee patrole tempest gate run times to under 1 hour

Registered by Felipe Monteiro

QA, at the PTG, requested that our Patrole gates run underneath an hour. Right now, we are encroaching upon that threshold. Thus, various strategies should be used to decrease the overall gate run time.

1) As part of this blueprint, a list needs to be compiled with the slowest tests. Many of those tests will probably include compute tests. Currently, there are several compute rbac test files that can be merged together, to reduce the number of servers that are booted. For example, test_lock_server_rbac and test_suspend_server_rbac can probably be merged together in the same file, i.e. test_server_actions_rbac.

The same sort of approach may need to be taken for volume tests, which are very slow, particularly those involving backups.

2) The test regex used to kick off tempest tests should be modified to look something like:
tempest run --regex (?!.*\[.*\bslow\b.*\])(^patrole_tempest_plugin\.(api|scenario))

It may be needed to tag slow tests just like Tempest does (see regex above), and run those slow tests using a separate command. Additional tempest gates can be added to run those slow tests on their own, if need be.

3) Only tests included in the big tent services [0] should be included in Patrole; tests belonging to other services should create a Patrole plugin.

Additionally, we probably need to hold off on implementing any further slow tests (i.e. shelve compute tests) until we have completed this blueprint.

[0] https://github.com/openstack/tempest/tree/master/tempest/api

Blueprint information

Status:
Complete
Approver:
Samantha Blanco
Priority:
Medium
Drafter:
Felipe Monteiro
Direction:
Needs approval
Assignee:
Rick Bartra
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Felipe Monteiro
Completed by
Felipe Monteiro

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/decrease-tempest-gate-runtime,n,z

Addressed by: https://review.openstack.org/443287
    Add @test.attr(type='slow') to slow tests to reduce test run time.

Addressed by: https://review.openstack.org/443288
    Add negative lookahead to post_test_hook to skip slow tests.

Addressed by: https://review.openstack.org/438203
    Add role validation/run time efficiency to Patrole framework.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.