Add a third image type for infra images

Registered by Mark Goddard

We currently have two image types in kolla: binary and source. This makes sense for our OpenStack images which can use either distro packages or source tarballs.

For other images this makes less sense though, and we end up with images which should be identical, but are not due to their base image content. For example, we have both source and binary cron images.

This feature proposes to add a third image type - 'infra'. These images will not be affected by the source/binary type.

Infra examples: cron, rabbitmq, fluentd, mariadb

Some things to think about:

* Base image content for infra images
* Minimising image size
* Optimising for overall network bandwidth in CI publish jobs
* Optimising for individual image size (often conflicts with previous)
* New CI publish job for infra, or use one of binary/source?

Blueprint information

Status:
Started
Approver:
Mark Goddard
Priority:
High
Drafter:
Mark Goddard
Direction:
Needs approval
Assignee:
None
Definition:
Approved
Series goal:
Accepted for wallaby
Implementation:
Started
Milestone target:
milestone icon 12.0.0
Started by
Marcin Juszkiewicz

Related branches

Sprints

Whiteboard

Addressed by: https://review.opendev.org/698753
    WIP: Create 'infra' type of images

Gerrit topic: https://review.opendev.org/#/q/topic:wip/infra-images

Addressed by: https://review.opendev.org/707599
    WIP: introduce non-infra-base image

One of patches introduces 'non-infra-base' image which only holds binary/source environment variables. This allows us to have 'base' image the same for binary/source builds and due to that all infra images are also the same nevermind which 'install type' was used during build.

When both types of images are built on one fresh system then first build (let assume 'binary') creates Infra type of images from scratch and also builds Binary images. Then next Source build can be run and Infra images will be recreated from Docker's cache while Source ones will be fully built.

So if we run binary/source builds as one CI job then we get set of binary/infra/source images using shared layers ('base' will be the same for each image). But that's enlarge build time. Also there is more data to push inside one CI job (but smaller amount then running two separate CI jobs).

https://etherpad.opendev.org/p/Kolla-infra-images has notes

Addressed by: https://review.opendev.org/738072
    Add 'infra' install type.

Addressed by: https://review.opendev.org/739711
    introduce 'infra_image_prefix' for infra images

Gerrit topic: https://review.opendev.org/#/q/topic:infra-images

Addressed by: https://review.opendev.org/739935
    fluentd: migrate to td-agent 4 where possible

Addressed by: https://review.opendev.org/738994
    ubuntu: move to 20.04 Focal

Addressed by: https://review.opendev.org/744032
    Rename infrastructure images by default

Gerrit topic: https://review.opendev.org/#/q/topic:bp/infra-images

Addressed by: https://review.opendev.org/752928
    create 'add_binary_source_envs' macro and use it

Gerrit topic: https://review.opendev.org/#/q/topic:bp/infra-images/split

(?)

Work Items

Work items:
Build binary/infra/source images for CentOS: DONE
Build binary/infra/source images for Debian: DONE
Build binary/infra/source images for Ubuntu: DONE
Deploy all-in-one locally: DONE
Deploy all-in-one on CI: INPROGRESS

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.