Workload Fingerprint learning modules

Registered by Mrittika Ganguli on 2016-01-27

In a data center, the workloads utilize server hardware resources to varying degrees of demand. From an end-user perspective, there is a need to provide consistent performance which can be measured in industry known metrics and with standard or other popular benchmarks. Modern server hardware, right from the microprocessor to network controllers, is an increasingly complex system providing various features that are configurable to benefit the target usage scenarios.

Modifying one control attribute in isolation can adversely affect the state of other resources which makes tuning a complex optimization problem. Resource tuning requires a proactive analysis based on insights derived from statistical analysis of data collected while running a collection of well known benchmarks. Proactive self-tuning by advanced knowledge of future states and application behavior enables the system to adapt to changing environments autonomously.

Understanding application behavior has been a key source of insight for striking a balance between reducing operational cost while improving the workload performance. Profiling an application reveals its time-varying behavior that repeats in some defined patterns over its lifetime. A program phase defines a discontinuity in time where observable characteristics fluctuates in a distinctive manner to trigger a measurable system impact. Data mining techniques when employed to the runtime trace of a program, can discover the program operational phases and its corresponding properties. The workload characteristics integrated through the sequence of known (or dynamically identified) phase are represented as a fingerprint. These fingerprints ascertain future behavior at relatively low computational cost.

We will define the following modules in Watcher:
1. Learning modules for Adaboost
2. Phase detection module
3. Prediction of a NCU metric using phases
4. using the data in 3 to select the right filter to schedule in Nova

Blueprint information

Status:
Not started
Approver:
Antoine Cabot
Priority:
Undefined
Drafter:
Mrittika Ganguli
Direction:
Needs approval
Assignee:
None
Definition:
New
Series goal:
None
Implementation:
Unknown
Milestone target:
None

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/fingerprint,n,z

Addressed by: https://review.openstack.org/448396
    Support workload fingerprinting

Addressed by: https://review.openstack.org/448402
    Support workload fingerprint generation

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.