Apt Improvements

Registered by Scott Moser on 2012-04-23

= Rationale =
As described in bug 972077, apt's on-disk format makes it difficult or impossible to update the archive without exposing a race condition to a client. While in theory potentially very small, in reality it makes 'apt' a very weak link in a automated install scenario.

= Goal =
Make apt hashsum mismatch errors go away permanently, and remove racey conditions when updating apt repositories.

Blueprint information

Status:
Not started
Approver:
Dave Walker
Priority:
Medium
Drafter:
Ubuntu Server
Direction:
Approved
Assignee:
Robie Basak
Definition:
Approved
Series goal:
Accepted for raring
Implementation:
Not started
Milestone target:
milestone icon ubuntu-13.04-feature-freeze

Related branches

Sprints

Whiteboard

Etherpad: http://pad.ubuntu.com/uds-q-servercloud-q-apt-improvements
Blueprint: https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-apt-improvements
Proposed spec: https://wiki.ubuntu.com/AptByHash
Discussion: https://lists.ubuntu.com/archives/ubuntu-devel/2012-July/035487.html

= User Stories =
 * Jim is using JuJu, and adds a node. He does not want to see apt fail due to a race condition.
 * Marley is using the Ubuntu installer for network based installs against a local mirror that is updated from a remote mirror. He does not want installs to fail due to the mirror being updated.
 * Sally wants to use a http proxy to cache deb downloads and does not want to be vulnerable to caching mismatched files that would cause apt failures.

= Assumptions =
None

= Test Plans =
Test will be conducted in 2 ways:
 a.) implied test: much of our automated infrastructure relies on apt and previously failed quite commonly due to apt. These errors should be erased.
 b.) explicit test: we should ideally have some canary that would ensure our public mirrors do not fail. One example starting point is [1]
   [1] https://code.launchpad.net/~smoser/+junk/check-archive/

= Release Note =
Ubuntu's public mirrors now contain an improved archive format. Coupled with updates to apt transient apt-update errors are a thing of the past.

Note: feature freeze is upon us, and I think this work is too invasive for an FFe. My priorities mean that I can't work on this for the next few weeks, so it looks like this will have to slip to R. --racb

[21-Sep-2012] Postponing remaining work items until R cycle per Robie's bove note. -a.rosales

I see no reason why this needs more sessions/discussion, this should probably not be scheduled for UDS-R --adconrad

(?)

Work Items

Work items:
[racb] debootstrap production code: POSTPONED
[racb] apt production code: POSTPONED
[utlemming] merge s3aptmirror production code: POSTPONED
apt-ftparchive by-hash generator command: POSTPONED
apt-ftparchive modifications to automatically generate by-hash: POSTPONED
ubumirror patch: POSTPONED
debmirror patch: POSTPONED
Fix squid-deb-proxy cache settings as appropriate: POSTPONED
[smoser] Pin RR DNS in squid-deb-proxy: POSTPONED
Testing: POSTPONED
Implement InRelease in archive.ubuntu.com mirrors: POSTPONED
Add by-hash generation to archive.ubuntu.com: POSTPONED
Add by-hash generation to Launchpad PPAs: POSTPONED
[racb] apt proof of concept: DONE
[racb] debootstrap proof of concept: DONE
[racb] s3aptmirror patch: DONE
[racb] Live S3 mirror: DONE

Dependency tree

* Blueprints in grey have been implemented.