Server QA workflow

Registered by Chuck Short on 2010-04-26

The server team is spending alot of time triaging incoming bugs and keeping on top of the number of bugs can become difficult. This session will be a discussion on how we can improve the workflow for our team members and have a better response to our user base.

Blueprint information

Status:
Started
Approver:
Jos Boumans
Priority:
High
Drafter:
C de-Avillez
Direction:
Approved
Assignee:
Chuck Short
Definition:
Approved
Series goal:
Accepted for maverick
Implementation:
Good progress
Milestone target:
milestone icon ubuntu-10.10-beta
Started by
Thierry Carrez on 2010-06-09

Related branches

Sprints

Whiteboard

Status:
Work on Eucalyptus blocked this blueprint.

Complexity:
[hggdh2] maverick-alpha-3: 1
[hggdh2] ubuntu-10.10-beta: 1

Roadmap Notes:
Wiki updates can be moved to the beta phase if short on time

Work items for maverick-alpha-3:
[hggdh2] Update wiki DebuggingSamba: DONE
[hggdh2] Update wiki DebuggingMySQL: DONE
[hggdh2] Update wiki DebuggingOpenldap: DONE
[hggdh2] Update source_samba.py to refuse a bug if /etc/smb.conf is missing: DONE
[hggdh2] create pre-set search for 'Incomplete with response (Server team kndowledgebase)': DONE
[hggdh2] create pre-set search for 'Incomplete without response' (Server team kndowledgebase): DONE
[hggdh2] Update per package bug filling description to point to Debugging* pages (samba, mysql, openldap): DONE
[hggdh2] define a tag for work-intensive bug determination, and document it under Bugs/Tags (new tag 'work-intensive'): DONE
[hggdh2] review and adjust (as needed) server-team package subscription: POSTPONED
[hggdh2] introduce the new tools during an Ubuntu Server Team meeting: POSTPONED
[hggdh2] Make New,Undecided daily bug triage results more public: POSTPONED

Work items for ubuntu-10.10-beta:
[hggdh2] Make New,Undecided daily bug triage results more public: POSTPONED
[hggdh2] review and adjust (as needed) server-team package subscription: POSTPONED
[hggdh2] introduce the new tools during an Ubuntu Server Team meeting: POSTPONED

====

Reviewers: ttx + mathiaz

mathiaz review / 20100526:
 - update some WI

ttx review / 20100527:
 - Missing a spec, but looks good from the WI
 - Suggested assignee: hggdh
 - Complexity: 3
 - Milestone: A2

wait on LP for "more information provided"

=== UDS M notes ===
 * making turn out good on bug.
   * Daily bug triage
 * review Bug triaging process.
   * Subscribe to incomplete bugs ?
 * Introduce QA Best practices:
   - Look into CI.
 * Access to UEC for spinning images.
 * Prioritization / escalation
(GobbyDoc server-m-qa-workflow)
== QA Workflow ==

=== Objective: Setting bug importance asap ===
Answer: Daily bug triage:
 - review New,Undecided bugs: set importance (mandatory), update status (optional).
 - some days are worse than others, sometime the day before would not be done
 - leveraging bugsquad?
   - seems not to be as responsible as one would expect
 - currently only one per day (small team)
 - easier to have community approaching via specialisation areas instead of day-of-week

Measure daily,new bugs.

Measure how many bug reports are time intensive.

DebugSamba: document importance.

Update pkgs guided bug filing instructions with links to Debugging*:
 - samba
 - openldap
 - mysql

Positive reinforcement loop:
  More public stats out ? Graphing ? Daily bug triager hall of {f,sh}ame ?

More DebuggingXXX wikidocs

=== Objective: don't let "incomplete with an answer" linger ===
Answer:?
Subscribe to incomplete bugs and see them to completion ?
Work in progress in LP to autounset "incomplete"

=== Other QA best practices ===
 * Look into CI.
 * Access to UEC for spinning images.
 * BugSquad DebuggingWhatever wiki pages (Samba, MySQL, etc)
 * write a bug pattern for samba upgrade failures (missing smb.conf?)
 * JSON for specific scenarios?
   - Incomplete with response
   - Incomplete without response
   - etc

=== Prioritization / escalation ===
 * Flagging/Tagging ?
  * use a tag for identifying work-intensive reproductions
  * set Importance as understood at the moment. Do not default to Medium.

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.