Stress test framework should track and report api success/failures

Registered by Walt Boring on 2013-09-24

When a stress test calls an api, say volume_client.create_volume(), the stress framework should track
that api call and note that it was successfull or failed. When the stress test is done, it should have the option
of dumping out the api calls and their success/fail rate.

Something like this (as a cinder test example)

[2013-05-24][10:12:28][A:668,E:2] =========================================
[2013-05-24][10:12:28][A:668,E:2] Test performed 668 actions.
[2013-05-24][10:12:28][A:668,E:2] Test performed 75 create volume actions.
[2013-05-24][10:12:28][A:668,E:2] Test performed 75 delete volume actions.
[2013-05-24][10:12:28][A:668,E:2] Test performed 108 create snapshot actions.
[2013-05-24][10:12:28][A:668,E:2] Test performed 108 delete snapshot actions.
[2013-05-24][10:12:28][A:668,E:2] Test performed 151 attach volume actions.
[2013-05-24][10:12:28][A:668,E:2] Test performed 151 detach volume actions.
[2013-05-24][10:12:28][A:668,E:2] Test observed 2 errors.
[2013-05-24][10:12:28][A:668,E:2] Test observed 0 create volume errors.
[2013-05-24][10:12:28][A:668,E:2] Test observed 0 delete volume errors.
[2013-05-24][10:12:28][A:668,E:2] Test observed 0 create snapshot errors.
[2013-05-24][10:12:28][A:668,E:2] Test observed 0 delete snapshot errors.
[2013-05-24][10:12:28][A:668,E:2] Test observed 2 attach volume errors.
[2013-05-24][10:12:28][A:668,E:2] Test observed 0 detach volume errors.
[2013-05-24][10:12:28][A:668,E:2] Total test time: 0:43:46.122799

Blueprint information

Status:
Complete
Approver:
Matthew Treinish
Priority:
Medium
Drafter:
Walt Boring
Direction:
Approved
Assignee:
Ramy Asselin
Definition:
Obsolete
Series goal:
Accepted for juno
Implementation:
Unknown
Milestone target:
milestone icon juno-3
Completed by
Ramy Asselin on 2015-08-18

Related branches

Sprints

Whiteboard

3/13/2014: Sample output of a 3.5 hour cinder CHO test: http://paste.openstack.org/show/73469/

Gerrit topic: https://review.openstack.org/#q,topic:stress-api-tracking,n,z

Addressed by: https://review.openstack.org/90449
    Track API call success & failures in stress tests

Gerrit topic: https://review.openstack.org/#q,topic:bp/stress-api-tracking,n,z

Addressed by: https://review.openstack.org/97589
    Add qa spec for bp:stress-api-tracking

(?)

Work Items

Work items:
[walter-boring] API Statistics Wrapper: DONE
[ramy-asselin] Stabilize API Statistics Wrapper: DONE
[walter-boring] Cinder Stress Test: DONE
[ramy-asselin] Stabilize Cinder Stress Tests: INPROGRESS
[ramy-asselin] Cinder CHO Test: DONE
[ramy-asselin] Cinder CHO Stabilize: INPROGRESS
[ramy-asselin] Extend Cinder tests: TODO
[ramy-asselin] Code Cleanup: TODO

This blueprint contains Public information 
Everyone can see this information.