Android QA

Registered by Zach Pfeffer on 2011-10-10

In this session we will talk about the Linaro Android QA process.

Description
------------------
The Android QA process tests high-level functionality, characterizes failures and files bugs against likely subsystems.

The current set of tests are listed here:

ADB
ADB over USB
ADB over Ethernet
All ADB commands

Boot and Basics
u-boot serial support
android console output over serial
android console shell over serial
android boots to console shell
android boots to UI

Audio
HDMI audio
Audio output on jacks
Audio input on jacks

Video Out - HDMI
All resolutions work
Monitor recognized
Best resolution automatically chosen
Monitor hot-plug

Video Out - DVI
All resolutions work
Monitor recognized
Best resolution automatically chosen
Monitor hot-plug

Graphics
Not-Accellerated
Accellerated 2D
Accellerated 3D

Ethernet
Ethernet configurable from command line
Ethernet auto-configured at boot
Ethernet integrated into Android connection manager

Wireless
Onboard wireless configurable from the command line
Onboard wireless auto-configured at boot
Onboard wireless integrated into Android connection manager

Bluetooth
Bluetooth available from command line
Bluetooth integrated into Android Bluetooth framework
Scanning devices in UI works
Pairing with headset in UI works

Modem
USB modem works for the command line
USB modem works with the connection manager

SD
SD-card mounted through fstab
SD-card can be accessed through gallery

Camera
Capture works
Save works
Gallery display

Video Playback
Software Playback
Hardware accellerated playback
Playback all supported formats

Powermanagement
Suspend and resume
Wakelock works
clock tree in debugfs
P-states (cpufreq driver)
C-states (cpuidle driver)
powertop

App Support
Can install apps
Browser
Calculator
Calendar
Camera
Clock
Contacts
Email
Gallery
Messaging
Phone

USB Host
USB Keyboard
USB Mouse

USB Device
Gadget driver loaded
Mass storage function (sdcard export)
Ethernet function (tethering)

Agenda
------------
1. Introduce the current QA process
2. Brainstorm

Goal of this Session
-----------------------------
To share the current QA practice and get input on how to make it better.

Blueprint information

Status:
Complete
Approver:
Zach Pfeffer
Priority:
Essential
Drafter:
Abhishek Paliwal
Direction:
Approved
Assignee:
Abhishek Paliwal
Definition:
Approved
Series goal:
Accepted for 2011q4
Implementation:
Implemented
Milestone target:
milestone icon connect-q4.11
Started by
Zach Pfeffer on 2012-02-28
Completed by
Zach Pfeffer on 2012-02-28

Related branches

Sprints

Whiteboard

[pabhishek 2011/01/01]: FInd Slides for this session at: https://docs.google.com/a/linaro.org/open?id=0B0xwyUNxNaAaZDc1YzYxN2EtZGFiNy00MWQ5LWEyNTQtNWM0ODhlNDliOTll
[zpfeffer 2011/10/31]
Possible action items:
Integrate CTS manually and in automation
See possible action items in https://blueprints.launchpad.net/linaro-android/+spec/linaro-platforms-lc4.11-android-benchmarking
Generate https://docs.google.com/a/linaro.org/spreadsheet/ccc?key=0AnpUtxWjZbP9dGFDUk5kNXBoeWZDb3MyUmJ4cnBHTEE&hl=en_US#gid=0 more easily
    Fix all the yellow and red boxes

Notes from session:
Android QA

Suggestions
AI: Need interaction tests, play music
AI: Pull the SD card out

NoJS complaining about

AI: Get CTS in

Tony: I have a concern about automation. You should try to automate the process.

Alexander: Audio, USB storage device

Zach: High speed video capture

Speech recognition

AI: Abhishek: Sign up

AI: Sign wave test, Amit Pundir

AI: Auto testing Ethernet and WiFi

AI: USB Storage

Botao: sl4a Android Scripting Language, for doing Video capture testing
    Test scripts

Zach: Need to make setting up LAVA tests easier, web interface
AI: Set up session with LAVA to talk about making things easier

AI: not tested, bug

AI: look at bugs as driving work

AI: Tony: when was fix commited?

AI: Tony to look for launchpad lib

(?)

Work Items