Firmware Test Suite Improvements for 12.10

Registered by Colin Ian King

List of features and improvements for the Firmware Test Suite for Ubuntu 12.10

Blueprint information

Chris Van Hoof
Colin Ian King
Canonical Hardware Enablement
Series goal:
Accepted for quantal
Milestone target:
milestone icon ubuntu-12.10-beta-1
Started by
Chris Van Hoof
Completed by
Chris Van Hoof

Related branches



= Discuss new features and fixes for fwts for 12.10 =

== notes from the minds of cking and vanhoof ==
 * update logging output formats, json, xml, etc [AR]
 * update to core backend for UEFI specific tests outside of the OS [AR]
   * define tests first
   * evaluate work required to implement
 * harvest results.log systematically as BIOS/Firmware revisions update in enablement to verify fixes and changes delivered [AR]
  * Will prioritize this next cycle [vanhoof]
 * $profit

== Core Discussion ==
* ACPI 5.0 support [AR]
 * what's the delta - what do we need to change? ~2-3 days to evaluate delta
* Finish off the method test - lots more objects to sanity check, for example: Full parsing and sanity checking of the _CRS resource structures. Currently if these are broken the kernel aborts and misses subsequent structures in the _CRS buffer which may be considered sub-optimal by some ;-)
  * ~30 more tests to complete -- nice to have
* Re-work logging - it really could do with a re-write to allow different formatting, e.g. structured logs in json. [AR]
  * ~1-2w estimate
* Remove the leading logging cruft "lineno testname " from log by default, or just for the "-" option.
  * survey says kill it [AR]
* expand the fwts-test test
  * ~30% covered now -- Shoot for 50% this cycle [AR]

Discuss wishlist items:
* LP#853875 battery test should sanity check _BTP via /proc/acpi/battery/*/alarm ~ 15% of machines had this control, so probably worth adding [alexhung]

* LP#859568 method test could give references to ACPI specification when it detects non-compliant AML - nice to have feature [jk/vanhoof/cking/alex to chat]

* LP#863175 should examine the PCC tables - not sure if it is worth doing
  * Not being used presently, continue to monitor

* LP#872325 ASPM PCI-express incorrect BIOS setting detection - d2f2534ecbe9603369c5e0e23af92bb17a7aa8ac does some ASPM checking - is that now complete?
  * ~1w, good progress

* Escalation path for fwts failures found during certification testing (who to report bugs to etc.)
  * in bugs per usual process, Critical/High

* Should we port fwts to uefi (framework + UEFI specific tests). Considerable effort - need to add OS abstraction layer and write a UEFI OS interface.

== UEFI Specific (courtesy of Harry @ Intel ==
* OS runtime UEFI calls from OS runtime
  * See chapt 7 from UEFI 2.3.1 spec on

* All of these tests needed to be performed on UEFI runtime services that maybe relocated above 4gig (to make sure it is 64bit safe).

=== 7.2 variables services:===
* Getvariable
  * Getnextvariablename
  * Setvariable
* Needs to test enough nvram (ie 16k? or some minimal size of memory that an OS will need to install)

=== 7.2.1 ===
* EFI_variable_authentication

 === 7.3 ===
* Time services
  * Gettime, settime, getwakeuptime, setwakeuptime

=== 7.4 virtual memory services ===
* Setvirtualaddressmap
* Convertpointer
   * Used by loader so maybe done in loader before virtual mode handoff to kernel. Important that bioses do this correctly for the full memory range
   * Above 4gig and at key address boundries (32gig etc and near top of memory). Avoid <1meg where legacy bios compat. Is required (check memory map).

=== 7.5 ===
* Reset system
* Get next high monotonic count
* Update capsule (not supported yet. To be included in future release in

=== Misc Tests ===
* Chapt 5 GPT support (bios routines can actually address > 2.2Tb of disk space)
* Chapt 4 EFI system table (has one and is correctly formed)

== Update 2012-09-20 ==
Updated remaining items to reflect current status and migrated any items not implemented this cycle to 13.04:


Work Items

Work items:
[alexhung] test battery trip point (_BTP): DONE
[colin-king] remove leading line number + test name from log: DONE
[colin-king] re-work logging to allow for structured output (json): DONE
[colin-king] add xml logging output: DONE
[colin-king] add html logging output: DONE
[colin-king] add multi-logging output: DONE
[colin-king] survey for new ACPI 5.0 control methods: DONE
[colin-king] add ACPI 5.0 support to acpidump: DONE
[colin-king] update to latest ACPICA: DONE
[vanhoof] write a fwts json diff tool: POSTPONED
[alexhung] alex to define/scope UEFI tests prior to evaluation of fwts backend changes: POSTPONED
[vanhoof] move fwts-live images to Precise userspace: DONE
[colin-king] cking review changes to backend of fwts to support native UEFI testing: DONE
[vanhoof] investigate what it would take to have fwts/fwts-frontend on install cd: POSTPONED
[vanhoof] check into file formatting issues of results.log for readability on other OSes: DONE
[vanhoof] check in on nautilus automagically opening right path on usb drive on mount: POSTPONED
[vanhoof] investigate the potential to ask users to submit results.log to Launchpad via fwts-live: POSTPONED
[vanhoof] add serial console support to images: DONE
[vanhoof] update fwts-frontend to include --log-type plaintext,html: DONE
[vanhoof] have result.html preserved in casper persistence file: DONE
[cr3] investigate the potential to integrate into Ubuntu Friendly: POSTPONED
[] check that kernel does not use BIOS reserved memory used for Trusted Execution Technology: POSTPONED

Dependency tree

* Blueprints in grey have been implemented.