Anonymous statistics performance testing

Registered by Igor Shishkin on 2014-10-13

It's required to measure performance for anonymous statistics server side application:


Load tests should be performed for Collector service which is used for gathering and storing raw Fuel usage statistics information:

Yandex-tank will be used as testing tool for that. It requires third-party utility for creating random requests data (e.g. JSON schema with Fuel action logs or installation info).

Load test will be performed manually using specially created parameterized Jenkins' job:

RPS (requests per second) will be used as the metric. This measures the approximate number of Fuel installations from that collector service can receive statistics.

There are three stages of performance testing that should be performed:

1) Get threshold RPS value:
 - produce load on collector which growths linearly from 1 to some huge unreal RPS value (e.g. 10,000) with +1 step;
 - get the RPS max value (Threshold) which collector can handle correctly: less than 0,1% of 502 or connection timeout errors;
 - assume that collector service passed this stage of tests if 50% Threshold value is higher than expected max RPS produced by real Fuel installations (for example we can use 100,000 as greatest possible Fuel installations number, so expect up to 48 requests per seconds to collector from them according to normal (Gaussian) probability distribution, because Fuel will send statistics not more than once per hour).

2) Run long-term load test using the following test plan:
 - produce constant load using RPS = 50% of Threshold value;
 - periodically linearly change RPS value for +/- 25% for short period of time (< 5 minutes);
 - consider that collector service passed this stage of tests if not more than 0,01% requests to it failed.

3) Run stress tests:
 - perform 2nd step again using huge or invalid requests (JSON data);
 - measure response latency after adding 1, 5 and 10 millions of records to the collector database;
 - assume that collector service passed this stage of tests if not more than 1% of valid requests failed and average latency is less that timeout value.

Blueprint information

Alexander Kislitsky
Igor Shishkin
Andrey Sledzinskiy
Series goal:
Accepted for 6.1.x
Milestone target:
milestone icon 6.1
Started by
Artem Panchenko on 2014-12-10
Completed by
Andrey Sledzinskiy on 2015-04-16

Related branches



Gerrit topic:,topic:bp/anonymous-statistics-performance-testing,n,z

Addressed by:
    Add requests templates for performance testing

Addressed by:
    Add request for OSWL collector


Work Items

This blueprint contains Public information 
Everyone can see this information.


No subscribers.