Tempest support for testing against multiple keystone API versions

Registered by Andrea Frittoli on 2013-06-25

Tempests needs to be able to choose which version of keystone API shall be used.

At the moment tempest clients, tenant isolation classes and tests themselves rely on getting access to a specific version of an identity client, as well as of a specific set of credentials, which makes it difficult to switch between v2 and v3 and newer versions in the future.

A number of refactors are required to achieve this and make sure we don't need to change test code again when moving to different keystone API versions:

Authentication shall be factored out in an authentication provider .
Credentials shall be stored in a dedicated class, and provided to tests by a credential manager, being them static or dynamic ones.
Clients managers shall receive credentials and be the sole responsible for instantiating clients and provide them to the tests. At the moment client managers instantiate all available clients when created. This is unnecessary, and it's an overkill when just a single client is needed for a specific set of credentials. So client managers shall be changed to lazy instantiation of clients.

Different kind of tests are to be addressed: API tests, scenario tests, CLI tests, 3rd party tests, stress test framework. For scenario and CLI tests, tempest uses official clients. The adoption to keystone v3 tests is limited to the availability of keystone v3 compatibility in the official clients.

Blueprint information

Status:
Complete
Approver:
Giulio Fidente
Priority:
High
Drafter:
Andrea Frittoli
Direction:
Approved
Assignee:
Andrea Frittoli
Definition:
Approved
Series goal:
Accepted for kilo
Implementation:
Implemented
Milestone target:
None
Started by
Andrea Frittoli on 2014-01-07
Completed by
Andrea Frittoli on 2015-04-01

Related branches

Sprints

Whiteboard

This should probably be implemented with a config option.

Working assumption is that all non-keystone tests only uses a single keystone domain. There is no need for multiple domains for the non-keystone tests. ATM we have username and project name in tempest.conf. With this assumption it would be sufficient to add a domain name, which would apply for both user domain scope as well as project domain scope.

https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3.md

Created https://bugs.launchpad.net/python-cinderclient/+bug/1262843. Support for token / url combination in cinder / nova bindings would be enough to use all python bindings with keystone v3 API.

Gerrit topic: https://review.openstack.org/#q,topic:bp/multi-keystone-api-version-tests,n,z

Addressed by: https://review.openstack.org/65311
    Multiversion authentication part1

Addressed by: https://review.openstack.org/65966
    Fixes handling of arrays in XML to JSON conversion

Addressed by: https://review.openstack.org/71493
    Multiversion authentication part2

Gerrit topic: https://review.openstack.org/#q,topic:bp/part3,n,z

Addressed by: https://review.openstack.org/73704
    Refactor Managers to a common base class

Addressed by: https://review.openstack.org/74387
    Defines a Credentials class

Gerrit topic: https://review.openstack.org/#q,topic:bp/unit-tests,n,z

Addressed by: https://review.openstack.org/77594
    Enforces the use of Credentials

Addressed by: https://review.openstack.org/80244
    Object storage tests to use default auth_provider

Addressed by: https://review.openstack.org/80245
    Configuration for domain name

Addressed by: https://review.openstack.org/80246
    Get credential IDs from Credentials class

Addressed by: https://review.openstack.org/81294
    add bp:multi-keystone-api-version-tests

Addressed by: https://review.openstack.org/81872
    Define V3 Credentials

Addressed by: https://review.openstack.org/82111
    Use auth data to fill credentials

Addressed by: https://review.openstack.org/82112
    Enforces the use of Credentials (part2)

Addressed by: https://review.openstack.org/82113
    Access credential fields as attributes

Addressed by: https://review.openstack.org/82911
    Official client based auth

Addressed by: https://review.openstack.org/81896
    Add config fixture support to unit tests

Addressed by: https://review.openstack.org/90166
    Use auth provider in official client manager

Addressed by: https://review.openstack.org/93894
    Get tenant_id from credentials

Addressed by: https://review.openstack.org/133029
    Support v3 in credential providers and subclasses

Addressed by: https://review.openstack.org/161259
    Use the configured version of identity in stress

Addressed by: https://review.openstack.org/161262
    Skip ec2 and s3 tests without identity v2

Addressed by: https://review.openstack.org/161266
    Skip CLI tests is identity v2 is not available

Addressed by: https://review.openstack.org/163895
    Copy the default params dict to avoid race

Addressed by: https://review.openstack.org/167601
    Support identity_version in credential provider wrappers

Addressed by: https://review.openstack.org/166327
    Non-admin token tests for Keystone API

Addressed by: https://review.openstack.org/173335
    mv multi-keystone-api-version-tests to implemented

(?)

Work Items

Work items:
Define exact scope: DONE
Move auth from rest_client to auth provider: DONE
Provide unit tests for the new auth class: DONE
Refactor Manager, Credentials class everywhere: DONE
Tenant isolation support for V3: INPROGRESS
Provide multi auth-version for API tests: DONE
Provide multi auth-version for scenario tests: DONE
Skip 3rd part tests in case of v3 auth: DONE
Provide multi auth-version for stress framework: DONE
Setup an experimental keystone v3 check: DONE

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.