Service level testing
As openstack-salt is being deployed on various systems (rhel, ubuntu) and platforms (vm, baremetal, docker containers) there is a need to properly test service formulas (or openstack component formulas) test individually on these platforms. This blueprint covers approach for standardized developer local test environment as well as it aims to be a part of gerrit/jenkins review process.
This blueprint covers individual formula test scenarios, test & dev tools and approaches for formula validation.
Be aware there is complement blueprint being created. Aimed for system level & integration testing of all salt formula in system context.
Blueprint information
- Status:
- Started
- Approver:
- Jakub Pavlik
- Priority:
- Undefined
- Drafter:
- Petr Michalec
- Direction:
- Needs approval
- Assignee:
- Petr Michalec
- Definition:
- New
- Series goal:
- None
- Implementation:
-
Good progress
- Milestone target:
- None
- Started by
- Aleš Komárek
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
There were several approaches to be considered, however there are few important requirements to satisfy:
* formula unit validation
* python/jinja/yaml syntax validation
* salt best practices, style validation
* formula quick smoke tests for integrity
* standardized dev/test environment to be able to reproduce bugs or work on features
* formula integration test on various operating systems
* formula integration test on various operating platforms
Log your statements and proposals at:
https:/
More to read:
https:/
# Implementation
## Unit tests
## Syntax, Best practices
Syntax and best practices formula vaidation (ie: for Chef there is an http://
Part of the best practices/style validation is a gerrit review process.
## Development and integration testing
(A) - Unit tests
(B) - Quick, smoke tests
There is an Makefile in formula repo with `make test` target. That executes `tests/
(C) - Integration tests
More on this topic at: https:/
(will open for writing if interested)
Gerrit topic: https:/
Addressed by: https:/
Introduce Test Kitchen
Addressed by: https:/
service, endpoint name to contain region id
Addressed by: https:/
Introduce test-kitchen
Addressed by: https:/
Introduce test-kitchen
Addressed by: https:/
introduce test-kitchen
Addressed by: https:/
introduce test-kitchen
Addressed by: https:/
introduce test-kitchen
Addressed by: https:/
introduce test-kitchen
Addressed by: https:/
introduce test-kitchen
Addressed by: https:/
Revert "introduce test-kitchen"
Addressed by: https:/
Revert "introduce test-kitchen"
Addressed by: https:/
Revert "introduce test-kitchen"
Addressed by: https:/
Revert "introduce test-kitchen"
Addressed by: https:/
Revert "introduce test-kitchen"
Addressed by: https:/
Revert "Introduce test-kitchen"
Work Items
Work items:
kitchen yml test suite specification per openstack salt formula: DONE
basic verification scripts per openstack salt formula: INPROGRESS
jenkins and gerrit integration to exec kitchen tests: TODO
Dependency tree
* Blueprints in grey have been implemented.
