Fuel for OpenStack

Describe system tests using a set of templates instead of testcases with hardcoded configs and scenarios

Registered by Dennis Dmitriev

Currently, we are using a fixed cluster configuration with a fixed scenario for each system test, slightly correcting some parameters with environment variables.

There are several contexts that could be specified by templates:
- fuel admin node configuration,
- cluster settings,
- roles and hardware requirements for nodes in cluster,
- additional configs for plugins
- test scenario as a set of independent actions.

Using different template for any of the context gives a lot of variations for running system tests without adding a new hardcoded cases. Any new environment can be tested with all set of, for example, destructive tests; all cluster configurations can be tested for testing a specified plugin; just combining different templates for a test run.

Blueprint information

Status:
Complete
Approver:
Nastya Urlapova
Priority:
Undefined
Drafter:
Dennis Dmitriev
Direction:
Approved
Assignee:
Dmitry Tyzhnenko
Definition:
New
Series goal:
Accepted for mitaka
Implementation:
Implemented
Milestone target:
milestone icon 9.0
Started by
Timur Nurlygayanov
Completed by
Dmitry Tyzhnenko

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/template-based-testcases,n,z

Addressed by: https://review.openstack.org/197628
    Template based tests with yaml configs

Gerrit topic: https://review.openstack.org/#q,topic:feature/template-based-testcases,n,z

Addressed by: https://review.openstack.org/239895
    Template based test cases

Addressed by: https://review.openstack.org/240016
    Add new runner for system test

Addressed by: https://review.openstack.org/238105
    Move to template-based description of devops environments

Addressed by: https://review.openstack.org/242582
    Add decorator @action

Addressed by: https://review.openstack.org/244616
    Add custom groups for template base cases

Addressed by: https://review.openstack.org/244732
    Add nested action to template tests

Addressed by: https://review.openstack.org/247026
    Improvements of TestRail reporter for system_tests

Addressed by: https://review.openstack.org/249356
    Extends system tests to use devops template

Gerrit topic: https://review.openstack.org/#q,topic:feature/system_test_plugin_example,n,z

Addressed by: https://review.openstack.org/243225
    Add test for example plugins into system test

Gerrit topic: https://review.openstack.org/#q,topic:feature/bdd/class-setup-teardown,n,z

Addressed by: https://review.openstack.org/260017
    Add Setup and Teardown method for cases

Addressed by: https://review.openstack.org/261309
    Move define_custom_groups to system_test package

Addressed by: https://review.openstack.org/265899
    Add several common actions and tests

Addressed by: https://review.openstack.org/282404
    Refactoring of system_test classes

Addressed by: https://review.openstack.org/283187
    Add validator for test groups

Gerrit topic: https://review.openstack.org/#q,topic:feature/devops30,n,z

Addressed by: https://review.openstack.org/292977
    Add support devops 3.0 into fuel-qa

Addressed by: https://review.openstack.org/304680
    Replace django DoesNotExist exception with custom devops exception and provide full information

Addressed by: https://review.openstack.org/349503
    Add support devops 3.0 into fuel-qa

(?)

Work Items

Nearby

This blueprint contains Public information 
Everyone can see this information.