Crash Database

Registered by Jason Warner on 2011-10-18

Build a crash database for Ubuntu.

Quoting Matthew:

Microsoft, Apple, and Mozilla have dedicated crash tracking systems, separate from their bug trackers. This has multiple benefits, when compared with Launchpad and apport:
- people don't need a sign-on account, of any sort, to report crashes
- developers aren't spammed with e-mail notifications about duplicate crashes
- non-developers aren't spammed with e-mail they don't understand after reporting crashes
- the error message, explaining that a window just disappeared because the application crashed, appears in release versions of the software -- not just alphas and betas
- contributors can easily see which are the most common causes of crashes
- contributors can easily see how reliable their software is compared with previous versions.

For these reasons, Ubuntu should have an error tracker too.

Overview of crash reporters: http://en.wikipedia.org/wiki/Crash_reporter
Public statistics from Mozilla: https://crash-stats.mozilla.com/products/Firefox
Previous blueprint: https://blueprints.launchpad.net/ubuntu/+spec/crash-tracker
Previous blueprint: https://launchpad.net/ubuntu/+spec/crash-reporting

Blueprint information

Status:
Started
Approver:
Steve Langasek
Priority:
Low
Drafter:
Evan Dandrea
Direction:
Needs approval
Assignee:
Evan Dandrea
Definition:
Approved
Series goal:
Accepted for precise
Implementation:
Good progress
Milestone target:
milestone icon ubuntu-12.04
Started by
Evan Dandrea on 2011-11-07

Related branches

Sprints

Whiteboard

Work items:
Decide whether 0-byte .upload files are a sufficient way to separate crash reports to be uploaded, or if we should modify the existing report: DONE
Rewrite the crash reporter daemon in C: DONE
Look into using apport-unpack to simply crash file parsing: POSTPONED
Find an optimal compression format for the core dumps: POSTPONED
Handle compressed and base64 encoded fields in crash reports: POSTPONED
Build a test harness and unit tests for the crash reporting daemon: DONE
Flatten crash reports before sending them to the crash database to make them more amenable to Cassandra and searching: POSTPONED
Clean up upload files and crash files after ~14 days: DONE
Implement the new apport user interface: DONE
Ensure NetworkManager is bringing up the interfaces as early as possible: DONE
Ensure Ubuntu has an Internet-connectivity check (GNetworkMonitor): DONE
Build out the oops-repository API to cover the post, additional information, update, and get information out operations: DONE
Craft a schema for the crashes column family: DONE
Integrate reporting to txstatsd into oops-repository: DONE
Wire txstatsd to Graphite: DONE
Build a small Django webservice for management functions: INPROGRESS
Build retracing queues on top of RabbitMQ: DONE
Create retracing services, one for each architecture, that pull off RabbitMQ, feed through apport-retrace, and post back to Cassandra: DONE
Add posting to retracing RabbitMQ queue and Cassandra back up queue to oops-repository: DONE
Create a top crashes report, generated in hourly, daily, and weekly increments: INPROGRESS
Create system tests for end-to-end functionality, using the Canonicloud to bring up test services: INPROGRESS
Stop cleaning up ddebs after 7 days (requires major changes): POSTPONED
Shepherd ddeb support in Launchpad (RT 45785): POSTPONED
Create fractured deployment functionality to turn on crash reporting for end users gradually: DONE
Evaluate existing hardware database, launchpad-results micro service, and HEXR as the location for the hardware profile submissions: POSTPONED
Glue the selected hardware database to the crash database, so that queries can operate across both: POSTPONED
Send a hardware profile on first Internet connection (install or post-install): POSTPONED
Investigate using Datastax's Brisk/Enterprise with Pig or Hive to query over existing crash reports: POSTPONED
Evaluate using Socorro instead of oops-repository (Cassandra) for the backend: POSTPONED
Move retracing entirely into the crash database and provide as a web service for Launchpad to consume: POSTPONED
Link Launchpad bugs to crashes in the database with the same signature: POSTPONED
Provide an "update available that fixes this issue" response from oops-repository when the respective bug is closed by an upload: POSTPONED
Use an upstart inotify watch to drive the crash report submission: POSTPONED
Provide a facility to, in subsequent reports, automatically attach additional information as defined by a developer on the server-side: POSTPONED
Backport the crash reporting client to the previous Ubuntu version: POSTPONED
Backport the crash reporting client to the previous Ubuntu LTS: POSTPONED
[mpt] Design admin UI for disabling crash reporting <https://wiki.ubuntu.com/ErrorTracker#settings>: DONE
[jdstrand] security review for MIR (2) (high): DONE

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.