Move functional tests into their own repository

Registered by Adam Young

Keystone is kindof unique amongst open stack in that it has no other Open Stack service dependencies. Tie means that it is really easy for us to change something that breaks the rest of Open Stack. And thus, we are paranoid.

One side effect of that paranoia is that many of our "Unit tests" are really functional tests. For example, then entir v3 API requires spinning up a Keystone server and making an HTTP call. This is really a functional test, not a unit test.

Running the tests inside of Keystone takes a long time....too long. Unit tests are a key component of development, and need to be run constantly. If we make tyhe unit tests too slow, we slow down development.

I'd like to split off the functional tests from the unit tests. Now, in the past we had suggested moving these to Tempest, but I think that puts the burden of review on people that do not understand Keystone concepts as well as the core team. While we can certainly make use of the Tempest infrastructure, I think the right place for the Keystone functional tests is a repo managed by the Keystone team, but not in either the Keystone nor the Keystone client repositories. This repository would be responsible for any tests that:

Make a web call
Use any version of the Keystone client to talk to Keystone
Require diffferent "real backends" to run tests (LDAP, PostgreSQL, MySQL, Cassandra)

The existing Keystone server repositority would maintain a set of unit tests that exercise the backends against in memory KVS.,FakeLDAP and SQLite (or the MySQL based replacement for in memory SQL). These tests could be called from the keystone-testing repository with the alternative backends.

Changes to Keystone and Keystone client would both gate on the keystone-testing repository.

Blueprint information

Status:
Complete
Approver:
None
Priority:
Undefined
Drafter:
Adam Young
Direction:
Needs approval
Assignee:
None
Definition:
Superseded
Series goal:
None
Implementation:
Unknown
Milestone target:
None
Completed by
Morgan Fainberg

Related branches

Sprints

Whiteboard

We aren't going to move the functional testing out of tree at this time. There is no reason to do so with the current structure. We can revisit this if needed at a later point.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.