Remove duplicate code using Data Driven Tests (DDT)

Registered by Bhagyashri Shewale on 2016-07-26

Many times, there are a number of data sets that we have to run the same tests on. And, to create a different test for each data set values is time-consuming and inefficient.

Data Driven Testing [1] overcomes this issue. Data-driven testing (DDT) is taking a test, parametrizing it and then running that test with varying data. This allows you to run the same test case with many varying inputs, therefore increasing coverage from a single test, reduces code duplication and can ease up error tracing as well.

DDT is a third party library needs to be installed separately and invoke the module when writing the tests. Several openstack projects [2] are using DDT for test optimization.

DDT generates the test names automatically which can be problematic to trace out for which test data failure occurred. To overcome this limitation we can pass descriptive name using annotate function as shown below:

 @ddt.data(annotated('member_key', {'mem': TENANT1}),
                              annotated('member_empty', {'member': ''}),
                              annotated('list_return_error', [TENANT1]))
         def test_create_invalid_member(self, value):
        pass

it generates following test names:

 test_create_invalid_member_1_member_key
 test_create_invalid_member_2_member_empty
 test_create_invalid_member_3_list_return_erro

This will help to find out failure against particular test data.

Same is implemented in 'openstack/zaqar' [3].

[1] http://ddt.readthedocs.io/en/latest/index.html
[2] http://codesearch.openstack.org/?q=ddt%3E%3D1.0.1&i=nope&files=&repos=
[3] https://github.com/openstack/zaqar/blob/master/zaqar/tests/functional/wsgi/v1/test_queues.py#L87

Blueprint information

Status:
Started
Approver:
Nikhil Komawar
Priority:
Undefined
Drafter:
Bhagyashri Shewale
Direction:
Needs approval
Assignee:
Bhagyashri Shewale
Definition:
Pending Approval
Series goal:
None
Implementation:
Slow progress
Milestone target:
None
Started by
Bhagyashri Shewale

Related branches

Sprints

Whiteboard

[2016-07-27 nikhil]

Hi Bhagyashri, drafting BP is not sufficient. You'd keep this as a placeholder and then file a spec for a full discussion. See more on: https://github.com/openstack/glance/blob/master/doc/source/contributing/blueprints.rst

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.