support running apt with eatmydata

Bug #1236531 reported by Scott Moser
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
Undecided
Unassigned
cloud-init (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

During instance boot, cloud-init installs packages (if instructed by the user).
During instance boot, there is no state of the system that the user cares about.

I'd like to use eatmydata for apt-get install at that point, as it is dramatically faster than apt-get even with --force-unsafe-io.

From an ubuntu perspective, this change would mean:
 a.) depending on (or recommending) eatmydata
 b.) MIR for eatmydata
 c.) fixing eatmydata's relationship with sysvinit scripts. The thing to fix there is that if you 'eatmydata apt-get install some-service' and some-service starts a daemon, the daemon will inherit the LD_PRELOAD which is most likely not desired. (bug 1257036)

Related bugs:
 * bug 1257036: services started with eatmydata should remove eatmydata by default

Related branches

Scott Moser (smoser)
Changed in cloud-init (Ubuntu):
milestone: none → ubuntu-14.02
status: New → Confirmed
Changed in cloud-init:
status: New → Confirmed
Scott Moser (smoser)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in libeatmydata (Ubuntu):
status: New → Confirmed
Scott Moser (smoser)
description: updated
no longer affects: libeatmydata (Ubuntu)
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.5~bzr902-0ubuntu1

---------------
cloud-init (0.7.5~bzr902-0ubuntu1) trusty; urgency=medium

  * debian/control: Build-Depend on python-jsonpatch as #717916 is now fixed.
  * debian/control: Recommend eatmydata (LP: #1236531)
  * New upstream snapshot.
    * support invoking apt with 'eatmydata' (LP: #1236531)
    * add a message in log about dynamic import failures
  * New in '0.7.4' release.
    * fix reading of mount information on kernels < 2.6.26 (LP: #1248625)
    * SmartOS: change 'region' to 'datacenter_name' to address change
      in data provided to instance (LP: #1249124)
    * support calling 'add-apt-repository' for 'cloud-archive:' entries
      (LP: #1244355)
    * DataSourceAzure: fix incompatibility with python 2.6 (LP: #1232175)
    * fix bug mounting first partition of a alias'd name. (LP: #1236594)
    * SmartOS: fix bug with hostname due to trailing whitespace (LP: #1236445)
    * fix creation of partitions on Azure (LP: #1233698)
    * cc_growpart: respect /etc/growroot-disabled (LP: #1234331)
    * ubuntu config: add default user to 'sudo' group (LP: #1228228)
    * Fix usage of libselinux-python when selinux is disabled
    * add OpenNebula datasource
 -- Scott Moser <email address hidden> Tue, 17 Dec 2013 16:51:30 -0500

Changed in cloud-init (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Scott Moser (smoser) wrote :

So, very unscientific. But 2 systems launched with eatmydata enabled on trusty on azure. 2 systems without. Systems here were 'extrasmall' (670M memory).
cloud-config to do this is:
| system_info:
| apt_get_wrapper:
| enabled: False

The /var/log/cloud-init.log will have something like:
Jan 9 15:18:27 sm-eatdata0 [CLOUDINIT] util.py[DEBUG]: apt-install [eatmydata apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install bzr pastebinit ubuntu-dev-tools ccache bzr-builddeb vim-nox git-core lftp] took 105.916 seconds

Jan 9 15:26:20 sm-realsave0 [CLOUDINIT] util.py[DEBUG]: apt-install [apt-get --option=Dpkg::Options::=--force-confold --option=Dpkg::options::=--force-unsafe-io --assume-yes --quiet install bzr pastebinit ubuntu-dev-tools ccache bzr-builddeb vim-nox git-core lftp] took 88.697 seconds

eatmydata: 145.542 105.916
no-eatmydata: 88.697, 78.097

So... some interesting data there. I suspect that is a result of memory pressure on the vfs or something. Something to look at though.

Revision history for this message
Scott Moser (smoser) wrote :

ok. so on a 'small' instance type (1678M memory), I see:
eatmydata: 94.597, 103.672
no-eatmydata: 137.641, 115.919

All of the above were done on
3.12.0-7-generic (linux-image-3.12.0-7-generic 3.12.0-7.15).

Revision history for this message
Scott Moser (smoser) wrote :

bah. i think my data is garbage. i'll try to get better data. The issue I see is that my tests had run my 'apt-go-fast' (http://sn.im/apt-go-fast) as a bootcmd which makes /usr/bin/apt a wraper around eatmydata /usr/bin/apt.real anyway.

so really, the above should have all been being run with eatmydata anyway.
odd.

better numbers coming.

Revision history for this message
Scott Moser (smoser) wrote :

ok. better tests and results using the provided launcher script here.
I launched 3 small , 3 extrasmall both with and without eatmydata involved. sm-save-extrasmall-2 failed to launch (api/service failure). Below are the results, obtained with with the shell below.

sm-eat-extrasmall-0 104.547 16 88
sm-eat-extrasmall-1 106.096 12 94
sm-eat-extrasmall-2 127.666 35 92
sm-eat-small-0 103.512 10 93
sm-eat-small-1 128.128 24 104
sm-eat-small-2 101.121 5 96
sm-save-extrasmall-0 342.229 6 336
sm-save-extrasmall-1 332.766 5 327
sm-save-small-0 307.896 36 271
sm-save-small-1 290.326 28 262
sm-save-small-2 276.366 26 250

Revision history for this message
Scott Moser (smoser) wrote :
Revision history for this message
Scott Moser (smoser) wrote :

fixed in 0.7.5

Changed in cloud-init:
status: Confirmed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.