unit tests fail as non-root on maas deployed system

Bug #1635350 reported by Joshua Powers
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Medium
Unassigned
cloud-init (Ubuntu)
Fix Released
Medium
Unassigned
Xenial
Fix Released
Low
Unassigned
Yakkety
Fix Released
Low
Unassigned

Bug Description

=== Begin SRU Template ===
[Impact]
Running cloud-init's unit test cases on a system deployed by MAAS would
fail. The reason is that the non-root user would not be able to read
files with MAAS node credentials in /etc/cloud/cloud.cfg.d

We want this change SRU so that an attempt to build and run tests on a
system deployed by maas will work rather than fail due to unit test failure.

[Test Case]
Run unit tests on a system deployed by maas, or even just with:
  f=/etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg
  sh -c 'mkdir -p "${1%/*}" && touch "$1" && chmod ugo-r "$1"' -- "$f"
  tox -e py3

[Regression Potential]
This was just to fix a build break or unit tests being run.
Changes are only to unit tests.
=== End SRU Template ===

Observed Behavior:

On a system deployed by MAAS I checked out master and then tried to immediately build it:
> git clone https://git.launchpad.net/cloud-init
> cd cloud-init
> ./packages/bddeb

I get a number of errors around permission issues around this file:
PermissionError: [Errno 13] Permission denied: \'/etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg\'

See: https://paste.ubuntu.com/23354559/
 or formatted better: http://paste.ubuntu.com/23374383/

If I run as root however, it build as expected.

Expected Behavior:
Running bddeb works as a non-root user.

Related branches

Revision history for this message
Scott Moser (smoser) wrote :

just as a work around, you can actually build with:
 DEB_BUILD_OPTIONS=nocheck

and then it wont run the tests.

obviously good reason to run the tests, but you may be able to assume it ran already.

description: updated
Revision history for this message
Ryan Harper (raharper) wrote :

<rharper> nTraceback (most recent call last):\n File "/tmp/tmpvkejmell/cloud-init-0.7.8-19-g7ae2011/tests/unittests/test_data.py", line 375, in test_unhandl

Scott Moser (smoser)
Changed in cloud-init:
status: New → Confirmed
importance: Undecided → Medium
Scott Moser (smoser)
Changed in cloud-init:
status: Confirmed → Fix Committed
summary: - bddeb does not work on a system deployed by maas
+ unit tests fail on maas deployed system
summary: - unit tests fail on maas deployed system
+ unit tests fail as non-root on maas deployed system
Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
status: New → Fix Released
importance: Undecided → Medium
Changed in cloud-init (Ubuntu Xenial):
status: New → Confirmed
Changed in cloud-init (Ubuntu Yakkety):
status: New → Confirmed
Changed in cloud-init (Ubuntu Xenial):
importance: Undecided → Low
Changed in cloud-init (Ubuntu Yakkety):
importance: Undecided → Low
description: updated
Scott Moser (smoser)
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Joshua, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-47-gb6561a1-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Joshua, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-49-g9e904bb-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Scott Moser (smoser)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Scott Moser (smoser) wrote :

I've marked this as verification done.
It really was only a unit test issue, so not really related to SRU.
That said, I verified that, on a maas deployed system.

$ ls -l /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg
-rw------- 1 root root 309 Apr 11 2016 /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg
$ id
uid=1003(smoser) gid=1003(smoser) groups=1003(smoser),110(lxd),117(sbuild),1004(libvirt),1005(kvm),1008(curtin)

## So the file /etc/cloud/cloud.cfg.d/90_dpkg_maas.cfg is not readable
## by my user.

$ git clone git://git.launchpad.net/cloud-init cloud-init
$ cd cloud-init
$ git checkout ubuntu/0.7.8-49-g9e904bb-0ubuntu1
$ tox -e py3
...
Ran 750 tests in 22.480s

OK (SKIP=6)
____________________ summary ____________________
  py3: commands succeeded
  congratulations :)

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (4.0 KiB)

This bug was fixed in the package cloud-init - 0.7.8-49-g9e904bb-0ubuntu1~16.04.1

---------------
cloud-init (0.7.8-49-g9e904bb-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.postinst: update /etc/fstab on Azure to fix
    future resize operations. (LP: #1611074)
  * New upstream snapshot.
    - Add activate_datasource, for datasource specific code paths.
      (LP: #1611074)
    - systemd: cloud-init-local use RequiresMountsFor=/var/lib/cloud
      (LP: #1642062)

cloud-init (0.7.8-47-gb6561a1-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/cloud-init.templates: enable DigitalOcean by default [Ben Howard]
  * New upstream snapshot.
    - systemd/cloud-init-local.service:
      + replace 'Wants' and 'After' on local-fs.target with more granular
        After=systemd-remount-fs.service and RequiresMountsFor=/var/lib
        and Before=sysinit.target.
        This is done run sufficiently early enough to update /etc/fstab.
        (LP: #1611074)
      + add Before=NetworkManager.service so that cloud-init can render
        NetworkManager network config before it would apply them.
    - systemd/cloud-init.service:
      + add Before=sysinit.target and DefaultDependencies=no (LP: #1611074)
      + drop Requires=networking.service to work where networking.service is
        not needed.
      + add Conflicts=shutdown.target
      + drop unnecessary Wants=local-fs.target
    - net: support reading ipv6 dhcp config from initramfs [LaMont Jones]
      (LP: #1621615)
    - dmidecode: Allow dmidecode to be used on aarch64, and only attempt
      usage on x86, x86_64, and aarch64. [Robert Schweikert]
    - disk-config: udev settle after partitioning in gpt format.
      (LP: #1626243)
    - Add support for snap create-user on Ubuntu Core images. [Ryan Harper]
      (LP: #1619393)
    - Fix sshd restarts for rhel distros. [Jim Gorz]
    - Move user/group functions to new ug_util file [Joshua Harlow]
    - update Gentoo initscripts to run in the correct order [Matthew Thode]
    - MAAS: improve the debugging tool in datasource to consider
      config provided on kernel cmdline.
    - lxd: Update network config for LXD 2.3 [Stéphane Graber] (LP: #1640556)
    - Decode unicode types in decode_binary [Robert Schweikert]
    - Allow ephemeral drive to be unpartitioned [Paul Meyer]
    - subp: add 'update_env' argument which allows for more easily adding
      environment variables to a subprocess call.
    - Adjust mounts and disk configuration for systemd. (LP: #1611074)
    - DataSources:
      + Ec2: protect against non-dictionary in block-device-mapping.
      + AliYun: Add new datasource for Ali-Cloud ECS, that is
        available but not enabled by default [kaihuan.pkh]
      + DigitalOcean: use meta-data for network configuration and
        enable data source by default. [Ben Howard]
      + OpenNebula: replace parsing of 'ip' command with similar function
        available in cloudinit.net. This fixed unit tests when running
        in environment with no networking.
    - doc changes:
      + Add documentation on stages of boot.
      + make the RST files consistently formated and other improvements.
     ...

Read more...

Changed in cloud-init (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Chris Halse Rogers (raof) wrote : Please test proposed package

Hello Joshua, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.8-49-g9e904bb-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Yakkety):
status: Confirmed → Fix Committed
tags: removed: verification-done
tags: added: verification-needed
Revision history for this message
Joshua Powers (powersj) wrote :

Tested using integration tests [1] and manually downloaded and installed deb locally.

[1] https://jenkins.ubuntu.com/server/job/cloud-init-integration-y/16/consoleText

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.8-49-g9e904bb-0ubuntu1~16.10.1

---------------
cloud-init (0.7.8-49-g9e904bb-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/cloud-init.templates: enable DigitalOcean by default [Ben Howard]
  * debian/cloud-init.postinst: update /etc/fstab on Azure to fix
    future resize operations. (LP: #1611074)
  * New upstream snapshot.
    - systemd/cloud-init-local.service:
      + replace 'Wants' and 'After' on local-fs.target with more granular
        After=systemd-remount-fs.service and RequiresMountsFor=/var/lib
        and Before=sysinit.target.
        This is done run sufficiently early enough to update /etc/fstab.
        (LP: #1611074)
    - systemd/cloud-init.service:
      + add Before=sysinit.target and DefaultDependencies=no (LP: #1611074)
      + drop Requires=networking.service to work where networking.service is
        not needed.
      + add Conflicts=shutdown.target
      + drop unnecessary Wants=local-fs.target
    - net: support reading ipv6 dhcp config from initramfs [LaMont Jones]
      (LP: #1621615)
    - dmidecode: Allow dmidecode to be used on aarch64, and only attempt
      usage on x86, x86_64, and aarch64. [Robert Schweikert]
    - disk-config: udev settle after partitioning in gpt format.
      (LP: #1626243)
    - Add support for snap create-user on Ubuntu Core images. [Ryan Harper]
      (LP: #1619393)
    - Fix sshd restarts for rhel distros. [Jim Gorz]
    - Move user/group functions to new ug_util file [Joshua Harlow]
    - update Gentoo initscripts to run in the correct order [Matthew Thode]
    - MAAS: improve the debugging tool in datasource to consider
      config provided on kernel cmdline.
    - DataSources:
      + Ec2: protect against non-dictionary in block-device-mapping.
      + AliYun: Add new datasource for Ali-Cloud ECS, that is
        available but not enabled by default [kaihuan.pkh]
      + OpenNebula: replace parsing of 'ip' command with similar function
        available in cloudinit.net. This fixed unit tests when running
        in environment with no networking.
    - doc changes:
      + Add documentation on stages of boot.
      + make the RST files consistently formated and other improvements.
      + fixed example to not overwrite /etc/hosts [Chris Glass]
      + fix spelling / typos in ca_certs and scripts_vendor.
      + improve HACKING.rst file
      + Add documentation for logging features. [Wesley Wiedenmeier]
    - code style and unit test changes:
      + pep8: fix style errors reported by pycodestyle 2.1.0
      + pyflakes: fix issue with pyflakes 1.3 found in ubuntu zesty-proposed.
      + Add coverage dependency to bddeb to fix package build.
      + Add coverage collection to tox unit tests. [Joshua Powers]
      + do not read system /etc/cloud/cloud.cfg.d (LP: #1635350)
      + tests: silence the Cheetah UserWarning about NameMapper C version.
      + Fix python2.6 things found running in centos 6.

 -- Scott Moser <email address hidden> Tue, 22 Nov 2016 17:04:36 -0500

Changed in cloud-init (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

This is fixed in cloud-init 0.7.9.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.