Tempest in Rally. Refactoring, improvements and adding new features

Registered by Yaroslav Lobankov

Rally is able to install Tempest, generate the config file and run tests. Installation of Tempest works fine, but generation of the config file doesn't work correctly. We should fix, refactor and improve the existing code related to Tempest in Rally. In addition, we need to make some context that will create all resources needed for Tempest on the fly. Also, Rally needs some features that will be very useful for users. For example, generate the config file, not running Tempest; show the location of the config file, etc.

Blueprint information

Status:
Complete
Approver:
Andriy Kurilin
Priority:
Medium
Drafter:
Yaroslav Lobankov
Direction:
Needs approval
Assignee:
Yaroslav Lobankov
Definition:
Approved
Series goal:
None
Implementation:
Implemented
Milestone target:
None
Started by
Yaroslav Lobankov
Completed by
Yaroslav Lobankov

Related branches

Sprints

Whiteboard

- Possibility to generate tempest.conf, not running tempest (https://review.openstack.org/#/c/225130/)
- Improve the `rally verify import` command (https://review.openstack.org/#/c/228398/, https://review.openstack.org/#/c/230456/)
- Make consistency in logging (https://review.openstack.org/#/c/230756/)
- Update incorrect Tempest test sets in Rally (https://review.openstack.org/#/c/230759/)
- Rework generation of the Tempest config file (https://review.openstack.org/#/c/226652/)
- Make a context for creating all resources needed for Tempest (https://review.openstack.org/#/c/226652/)
- Add creation of network resources to the existing context (https://review.openstack.org/#/c/232134/)
- Add command to show tempest.conf (https://review.openstack.org/#/c/234126/)
- Add possibility to run a set of tests listed in a file (https://review.openstack.org/#/c/230087/)
- Add "x-fail" mechanism for Tempest tests (https://review.openstack.org/#/c/235857/)

(?)

Work Items

Work items:
- Possibility to generate tempest.conf, not running tempest: DONE
- Improve the `rally verify import` command: DONE
- Make consistency in logging: DONE
- Update incorrect Tempest test sets in Rally: DONE
- Rework generation of the Tempest config file: DONE
- Make a context for creating all resources needed for Tempest: DONE
- Add creation of network resources to the existing context: DONE
- Add command to show tempest.conf: DONE
- Add possibility to run a set of tests listed in a file: DONE
- Add "x-fail" mechanism for Tempest tests: DONE

This blueprint contains Public information 
Everyone can see this information.