Automatic verification of test environment

Registered by Dmitry Kaigarodеsev

Currently to add new node to Jenkins's worker pool DevOps engineer has to manually verify that this node has been properly configured to run this particular job. Which is not reliable and doesn't scale well. Creating the system of formal criteria for every job gives us the way to automate this process.

Moreover, If we categorize test types by similar requirements, we will be able to speed up verification process by eliminating unnecessary duplication: "if server runs BVT test properly, then it can run all the other system tests", "if server is capable of building MOS mirror, it can build iso as well", so we don't need to test twice.

Acceptance Criteria:

- there is a database (YAML file) of different Jenkins job types: generic system test, huge system test, nailgun unit test, iso build, mirror build...

- for each type there is a verification scenario (specific Jenkins job), which checks the node for - - --
support of this particular type of tests,

- there is a CI check for fuel-infra/jenkins-jobs/ repository, which verifies that all new jobs which are added to Jenkins configuration have type and verification scenario attached,

- there is a verify_for_type Jenkins job which runs verification scenario for specified job type on a specified node and on success sets <job_type>_ready label on the node

- there is a CI check for fuel-infra/puppet-manifests repository, which verifies that puppet manifests used to deploy Jenkins slaves of different roles pass verification scenarios
each job type has a description, and there is a script/tool to compile those descriptions into documentation in HTML form

- there is a CI job to publish that documentation on docs.fuel-infra.org site

Links:
Initial implementation for verification job. Works only for particular type of jobs on Fuel CI: https://bugs.launchpad.net/fuel/+bug/1440079

Note: Moving unit and functional tests to Docker containers will significantly decrease amount of verification required to add node to the worker pool, but will not eliminate the need for categorized verification system. We will just move parts of verification process to the stage where we build Docker containers.

Blueprint information

Status:
Not started
Approver:
Aleksandra Fedorova
Priority:
Undefined
Drafter:
Dmitry Kaigarodеsev
Direction:
Needs approval
Assignee:
Dmitry Kaigarodеsev
Definition:
Discussion
Series goal:
None
Implementation:
Unknown
Milestone target:
milestone icon next

Related branches

Sprints

Whiteboard

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.