Firmware Test Suite Improvements for 12.10

= Discuss new features and fixes for fwts for 12.10 =

== notes from the minds of cking and vanhoof ==
 * update logging output formats, json, xml, etc [AR]
 * update to core backend for UEFI specific tests outside of the OS [AR]
   * define tests first
   * evaluate work required to implement
 * harvest results.log systematically as BIOS/Firmware revisions update in enablement to verify fixes and changes delivered [AR]
  * Will prioritize this next cycle [vanhoof]
 * $profit

== Core Discussion ==
* ACPI 5.0 support [AR]
 * what's the delta - what do we need to change? ~2-3 days to evaluate delta
* Finish off the method test - lots more objects to sanity check, for example: Full parsing and sanity checking of the _CRS resource structures. Currently if these are broken the kernel aborts and misses subsequent structures in the _CRS buffer which may be considered sub-optimal by some ;-)
  * ~30 more tests to complete -- nice to have
* Re-work logging - it really could do with a re-write to allow different formatting, e.g. structured logs in json. [AR]
  * ~1-2w estimate
* Remove the leading logging cruft "lineno testname " from log by default, or just for the "-" option.
  * survey says kill it [AR]
* expand the fwts-test test
  * ~30% covered now -- Shoot for 50% this cycle [AR]

Discuss wishlist items:
* LP#853875 battery test should sanity check _BTP via /proc/acpi/battery/*/alarm ~ 15% of machines had this control, so probably worth adding [alexhung]

* LP#859568 method test could give references to ACPI specification when it detects non-compliant AML - nice to have feature [jk/vanhoof/cking/alex to chat]

* LP#863175 should examine the PCC tables - not sure if it is worth doing
  * Not being used presently, continue to monitor

* LP#872325 ASPM PCI-express incorrect BIOS setting detection - d2f2534ecbe9603369c5e0e23af92bb17a7aa8ac does some ASPM checking - is that now complete?
  * ~1w, good progress

* Escalation path for fwts failures found during certification testing (who to report bugs to etc.)
  * in bugs per usual process, Critical/High

* Should we port fwts to uefi (framework + UEFI specific tests). Considerable effort - need to add OS abstraction layer and write a UEFI OS interface.

== UEFI Specific (courtesy of Harry @ Intel ==
* OS runtime UEFI calls from OS runtime
  * See chapt 7 from UEFI 2.3.1 spec on http://www.uefi.org/specs/

* All of these tests needed to be performed on UEFI runtime services that maybe relocated above 4gig (to make sure it is 64bit safe).

=== 7.2 variables services:===
* Getvariable
  * Getnextvariablename
  * Setvariable
* Needs to test enough nvram (ie 16k? or some minimal size of memory that an OS will need to install)

=== 7.2.1 ===
* EFI_variable_authentication

 === 7.3 ===
* Time services
  * Gettime, settime, getwakeuptime, setwakeuptime

=== 7.4 virtual memory services ===
* Setvirtualaddressmap
* Convertpointer
   * Used by loader so maybe done in loader before virtual mode handoff to kernel. Important that bioses do this correctly for the full memory range
   * Above 4gig and at key address boundries (32gig etc and near top of memory). Avoid <1meg where legacy bios compat. Is required (check memory map).

=== 7.5 ===
* Reset system
* Get next high monotonic count
* Update capsule (not supported yet. To be included in future release in tianocore.org)

=== Misc Tests ===
* Chapt 5 GPT support (bios routines can actually address > 2.2Tb of disk space)
* Chapt 4 EFI system table (has one and is correctly formed)

== Update 2012-09-20 ==
Updated remaining items to reflect current status and migrated any items not implemented this cycle to 13.04:
 * https://blueprints.launchpad.net/ubuntu/+spec/hardware-r-fwts-features

Work items:
[alexhung] test battery trip point (_BTP): DONE
[colin-king] remove leading line number + test name from log: DONE
[colin-king] re-work logging to allow for structured output (json): DONE
[colin-king] add xml logging output: DONE
[colin-king] add html logging output: DONE
[colin-king] add multi-logging output: DONE
[colin-king] survey for new ACPI 5.0 control methods: DONE
[colin-king] add ACPI 5.0 support to acpidump: DONE
[colin-king] update to latest ACPICA: DONE
[vanhoof] write a fwts json diff tool: POSTPONED
[alexhung] alex to define/scope UEFI tests prior to evaluation of fwts backend changes: POSTPONED
[vanhoof] move fwts-live images to Precise userspace: DONE
[colin-king] cking review changes to backend of fwts to support native UEFI testing: DONE
[vanhoof] investigate what it would take to have fwts/fwts-frontend on install cd: POSTPONED
[vanhoof] check into file formatting issues of results.log for readability on other OSes: DONE
[vanhoof] check in on nautilus automagically opening right path on usb drive on mount: POSTPONED
[vanhoof] investigate the potential to ask users to submit results.log to Launchpad via fwts-live: POSTPONED
[vanhoof] add serial console support to images: DONE
[vanhoof] update fwts-frontend to include --log-type plaintext,html: DONE
[vanhoof] have result.html preserved in casper persistence file: DONE
[cr3] investigate the potential to integrate into Ubuntu Friendly: POSTPONED
[ivan.hu] check that kernel does not use BIOS reserved memory used for Trusted Execution Technology: POSTPONED