New database format for USN info

Registered by Marc Deslauriers on 2011-10-05

The current USN database is a python pickle, which is less than an ideal database format for importing into other projects. This session will discuss what alternative database format the security team could offer.

Blueprint information

Status:
Complete
Approver:
Jamie Strandboge
Priority:
High
Drafter:
Marc Deslauriers
Direction:
Approved
Assignee:
Marc Deslauriers
Definition:
Approved
Series goal:
Accepted for precise
Implementation:
Implemented
Milestone target:
None
Started by
Jamie Strandboge on 2012-04-23
Completed by
Jamie Strandboge on 2012-04-26

Related branches

Sprints

Whiteboard

Note: jdstrand> now that we serve over HTTPS, shipping a hash of the database is enough and we can skip signing the db and/or the hash. We will create a monitoring script for unauthorized changes to the usn database
Note: jdstrand> json exports located in https://usn.ubuntu.com/usn-db/database[-all].json[.bz2] with sha256sum files with .sha256 suffix

Work items:
[jdstrand] add export as JSON code (1): DONE
[jdstrand] investigate best way to sign (eg, dedicated key) (0.5): DONE
[jdstrand] investigate how to distribute public key (0.5): DONE

= Notes from etherpad =
The current USN database is a python pickle, which is less than an ideal database format for importing into other projects. This session will discuss what alternative database format the security team could offer.
History
- landscape, usn-website and security team are consumers
- was on people.canonical.com (http)
- now on www.ubuntu.com (https)
Do we ever remove it?
- only happened once--
Preferences from landscape
- single file with current stable releases only
- https good enough? no. should also be gpg signed
- yaml can't be used. same types of problems as pickle:
 * http://en.wikipedia.org/wiki/YAML#Security
 * http://pyyaml.org/wiki/PyYAMLDocumentation
 * probably best to avoid yaml and potential pitfalls and deliver somehting safe
- smaller size is preferrable, but not huge. compression is probably enough when using xml or json
[ACTION] invetigate json, yaml and xml (see the size difference)
[ACTION] investigate yaml for safe load
[ACTION] timeframe to obsolete pickle file (landscape says maybe 6 months, but will get back to us)
[ACTION] investigate best way to sign (eg, dedicated key)
[ACTION] investigate how to distribute public key

(?)

Work Items