Autoscaling API for Heat

Registered by Duncan McGreggor

Heat has done an enormous amount of work towards feature-parity with AWS AutoScaling and CloudWatch. The intent of this session is to discuss what work remains left to be done, to fully explore the scope of that work, and to come to a consensus on a high-level implementation plan.

In particular, there is interest in creating an API for manipulating scaling groups, responding to alerts, scaling up/down, etc. There are many ways an API could be provided, and at least two different places it could live. This session aims to resolve any ambiguities around this.

Ideally, the outcome of these session would be a well-defined approach for 1) any additional implementation needed in Heat, and 2) a plan for creating an autoscaling API that a consensus agrees is the best approach.

This summit discussion should result in several blueprints detailing whatever specific work needs to be done in the course of the coming release cycle.

Blueprint information

Status:
Complete
Approver:
Steve Baker
Priority:
High
Drafter:
Christopher Armstrong
Direction:
Approved
Assignee:
Christopher Armstrong
Definition:
Obsolete
Series goal:
None
Implementation:
Unknown
Milestone target:
None
Completed by
Steve Baker

Related branches

Whiteboard

(shardy) We already do everything described in this BP, see the AutoScalingMultiAZSample example template, description should be updated or this should be marked obsolete

(oubiwann) I've updated the description... it's not immediately clear to me whether the template you mention (this one? https://github.com/openstack/heat/blob/master/templates/AutoScalingMultiAZSample.template) does everything outlined in the updated description. Thanks!

(shardy) Please spend some time evaluating our existing AS functionality - we do almost everything mentioned in this BP already (with the exception of the separate AS API, which could be added), so the scope of this BP is too broad - can you work out what features are missing which you require, then raise a separate BP for each bit of missing functionality? (e.g "Add API to Autoscaling"). Drop into #heat so we can discuss your ideas and requirements further, thanks! :)

Resources which implement this functionality:
AWS::AutoScaling::AutoScalingGroup :
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html
implemented in : heat/engine/resources/autoscaling.py

AWS::AutoScaling::LaunchConfiguration:
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-launchconfig.html
implemented in : heat/engine/resources/autoscaling.py

AWS::AutoScaling::ScalingPolicy
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-policy.html
implemented in : heat/engine/resources/autoscaling.py

AWS::CloudWatch::Alarm
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cw-alarm.html
implemented in : heat/engine/resources/cloud_watch.py

The metric collection is currently performed by an in-instance agent (a clone of the cfn-push-stats tool, which pushes metrics to our CloudWatch (heat-api-cloudwatch) API, via cron-jobs in each instance, this is a basic mechanism, which will eventually be replaced or supplemented by ceilometer metrics and alarms, but currently ceilometer monitoring functionality is not complete enough for us to adopt it (see https://blueprints.launchpad.net/heat/+spec/watch-ceilometer)

So as you can see, we already implement a large proportion of what you're proposing, so a more constructive way forward is to evaluate the gaps in our existing implementation, rather than proposing a wholesale replacement with some new service (I understand you're working on something, but we have no visibility of it)

(shardy) Further note re AS API - although I used this as an example, I personally don't see this as a priority, since we support UpdateStack for the AS resources, so the preferred method of manipulating AS resources is by updating your stack template then updating the stack via heat-api or heat-api-cfn. We could certainly add a separate AS API in future, but right now I'm not sure if the effort involved can be justified.

(oubiwann) shardy, thanks SO MUCH for all the detail you provided. Absolutely brilliant. Over the next week, we will be digging into the resources that you have identified and assess functionality against our use cases and then refine the proposed discussion to cover only that which has no overlap with the amazing amount of work that has already been done in Heat. Thanks again for going above and beyond, and so clearly stating the exact points of implementation!

(stevebaker) Obsoleting, I've found the actual autoscaling blueprints

(?)

Work Items