Add support for Glance v2 to Horizon

Registered by Julie Pichon

Summary
=======

Detect whether Glance v1 or v2 is available and use the correct version
to make the API calls accordingly.

Motivation
========

Features that require v2 only should not be displayed when v2 is
in-use. We can also safely assume that v1 will be deprecated at some
point, and we should make sure Horizon fully supports v2.

Description
=========

Horizon should be able to detect whether Glance v1 or v2 is available
and use the correct version. Features that require v2 should not be
displayed (e.g. image custom properties [1]) if only v1 is available.

See the Keystone and Cinder API files for examples of how this is
achieved with other services in Horizon.

[1] cf. https://blueprints.launchpad.net/horizon/+spec/manage-image-custom-properties

UX
===

There shouldn't be any major UX impact.

 * There will be a new setting for the version of "image" service, just
   like identity, volume data_processing currently do [1].
 * Some actions may not be displayed depending on the Glance version,
   similar to the access controls currently in place for other panels,
   etc.

[1] https://github.com/openstack/horizon/blob/a69f712d94/openstack_dashboard/local/local_settings.py.example#L29

Note: The copy_from attribute, to create an Image from a URL is not currently implemented in v2 (see bug 1419040). We probably don't want to promote using v2 until this is the case (e.g. with the HORIZON_IMAGES_ALLOW_UPLOAD it wouldn't be unusual to a deployment to offer *only* the option to upload via URL). I think it shouldn't stop us getting the work underway until the feature is available though.

[2] http://docs.openstack.org/developer/horizon/topics/settings.html#horizon-images-allow-upload

(Related note to the note: don't let yourself be confused by all the "locations" stuff in the v2 client, it is unrelated to this and shouldn't be relied upon.)

Testing
======

Set the "image" version in the local settings to 1 or 2:

OPENSTACK_API_VERSIONS = {
   "image": 2,
}

When using v1:

In the Project Images panel: see the image list, image details, create an image from file upload or URL, create volume, edit, delete.
In the Admin panel: list images, see details, create Image, edit, delete. The metadata options shouldn't be visible.

When using v2:

Same as v1, except:
- Metadata menu available for images in the admin panel
- Possibly it won't be possible to upload an image via URL (until bug 1419040 is resolved)

Outside Dependencies
==================

The Glance client already supports a "version" parameter [1] since at
least 2012 (python-glanceclient 0.1.0).

[1] https://github.com/openstack/python-glanceclient/blob/master/glanceclient/client.py

Bug 1419040 needs to be implemented in Glance and glanceclient before we can allow upload via URLs.

Requirements Update Required
========================

No requirement updates expected, as we already rely on a recent Glance
client (except, see bug 1419040 for copy_from / URL upload)

Doc Impact
=========

Expected documentation changes:

* Settings:
 - New key for the OPENSTACK_API_VERSIONS dictionary

* Default behaviour changes:
 - Some actions that were displayed to users even though they were
   unusable (due to Glance v2 absence), may now be hidden by default

Blueprint information

Status:
Complete
Approver:
David Lyle
Priority:
High
Drafter:
Julie Pichon
Direction:
Approved
Assignee:
Brad Pokorny
Definition:
Approved
Series goal:
Accepted for 10.0.0-newton
Implementation:
Implemented
Milestone target:
milestone icon newton-rc1
Started by
Julie Pichon
Completed by
Rob Cresswell

Whiteboard

[jpichon 2015.01.13] Flesh out a bit the description, using the new template.

[travis-tripp 2015.01.22] This work in the metadata definitions patches and impacts the REST API patches. (Below)

https://review.openstack.org/#/c/104063/59/openstack_dashboard/api/glance.py

https://review.openstack.org/#/c/104063/59/openstack_dashboard/dashboards/admin/metadata_defs/panel.py

https://review.openstack.org/#/c/141273/15/openstack_dashboard/api/rest/glance.py

[travis-tripp 2015.01.23] Notes from call with Flavio (flaper87) who is converting Nova to v2:

 - Nova patch doing this work: https://review.openstack.org/#/c/144875/26/nova/image/api.py,cm
 - No longer a specific field for properties
 - Properties must be a string
 - Custom properties are enabled in the server by default, meaning you can send any property. Can check using image_schema['properties']['additionalProperties'] or something like that. However, if somebody enables custom properties on the Glance server, then only certain properties will be allowed and the Glance client will throw an exception. It is painful, because every time you try to add the property, you'll get the exception from the client. (see https://github.com/openstack/glance/blob/master/etc/schema-image.json)
 - is_public has been changed from being a boolean to being an enum.
 - if you want to filter on a property, you don't send property-
 - purge properties keyword is gone
 - image size is now None by default. In version 1, it was 0 (zero)
 - Rely on keystone to determine the version.
 - Version is no longer a discrete version, just pass in URL that is parsed
 - Can find out from client what version instantiated (e.g.)
   - if client.version >= 2:
 - There are image fields that are readonly. In v1, the client would just ignore it and move on if you tried to update it. In v2, the client will throw an exception. For example direct_url. If you get the full image details and then update part of it and then you need strip out those fields. See line 329 of the above review. Copied below:

    if client.version >= 2:
        # NOTE(flaper87): Hold your breath, don't freak out and
        # do not call the police. This works as expected (TM).
        # Yeah... right. Anyway, turns out that glance's images
        # in v2 don't have a `properties` field. Instead, they
        # have properties that are custom and defined using jsonschema.
        # In order to keep this backwards compatible, we need to
        # build that good ol' `properties` dictionary with the
        # custom properties.
        #
        # Oh, why didn't I do it the other way around? Why didn't
        # I translated v1 images into v2-like ones? The reason is
        # that in v1 we don't have a good way to know what properties
        # are custom unless you have the image. This is a problem
        # for `POST` operations where we don't have neither a v2
        # client instance nor a v1, trust-worthy, not created by
        # nova, image that we can use to move properties around.
        #
        # Yes, yes. This too shall pass ;)
        props = image_schema['properties']
        base_props = [prop for prop in props
                      if props[prop].get('is_base', True)]
        for attr in image.keys():
            if attr not in base_props:
                output['properties'][attr] = image.get(attr)

In v1 to v2, uploading images changed. See line 562.

[jpichon 2015.02.06] Updated blueprint testing, UX and requirements update based on bug 1419040. I don't think that missing feature should block this blueprint ; I think we might want to wait until it's available to promote v2 but it shouldn't prevent us from adding support to v2 in the places where we can already.

Gerrit topic: https://review.openstack.org/#q,topic:bp/horizon-glance-v2,n,z

Addressed by: https://review.openstack.org/150084
    /!\ WIP /!\ Support for Glance v2

Gerrit topic: https://review.openstack.org/#q,topic:bp/edit-server-metadata,n,z

Addressed by: https://review.openstack.org/246486 - Merged
    Add version check for listing namespaces

Addressed by: https://review.openstack.org/256627 - Abandoned
    Add version check for listing namespaces

Addressed by: https://review.openstack.org/249531 - Abandoned
    Fix issue when using glance v2 api

Addressed by: https://review.openstack.org/320039 - Needs Review
    [WIP] Support for Glance v2

Gerrit topic: https://review.openstack.org/#q,topic:glance-v2,n,z

Addressed by: https://review.openstack.org/334097 - Merged
    Allow metadata to be shown whether v1 or v2 glance data (NG)

Addressed by:
    Fix the loss of JSON types when using multipart/form-data - Merged

Gerrit topic: https://review.openstack.org/#q,topic:bp/horizon-glance-v2-patch41,n,z

Addressed by: https://review.openstack.org/364316 - Needs Review
    another try to deal with metadata nicely

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information 
Everyone can see this information.