Refactor Unity accessibility support

Registered by Luke Yelavich

Discuss and outline the work required to fully enable Unity 3D for accessibility and LDTP/at-spi based automated testing.

Blueprint information

Not started
Jason Warner
Needs approval
Series goal:
Proposed for raring
Milestone target:

Related branches



Unity's accessibility support has had a somewhat checkered history. When unity was introduced in Natty, some work was done to implement assistive technology/accessibility support in unity, using the atk library. For various reasons, it was decided at the time for unity to use atk directly, rather than the underlying widget toolkit, Nux.

At the same time, a version of Unity that didn't require 3D acceleration, known as Unity 2D, was developed, using the QT toolkit. At the same time, work was ongoing in the upstream QT community to add support for at-spi, the now cross-desktop accessibility infrastructure framework that allows assistive technologies such as Orca, as well as automated testing frameworks like LDTP to query an application's widgets, and their current state. Given the new at-spi support for QT 4.6+, work was done to improve unity-2d accessibility in parallel. Due to having toolkit support for accessibility already, the extra work required to extend unity-2d to work with Orca wasn't that much.

In the meantime, Unity 3D was getting new features such as the HUD. Due to the current partial support for accessibility in Unity 3D, and given that Unity used atk directly, the HUD wasn't, and still isn't, accessible out of the box. Now that unity 2D is no longer developed, work needs to e done to have full atk support for Unity 3D for 14.04, both for assistive technology use eg Orca, and for automated testing via the various frameworks that can work with at-spi, such as LDTP.

It is therefore worthwhile exploring the question of whether Unity's atk support should be re-architected around Nux. In other words, whether we should move all atk support code from Unity over to Nux. This brings about several advantages:
* Any other application using Nux will automatically get accessibility, and automated testing support with no extra effort on the app author's part.
* When Unity makes use of a Nux widget, it gets accessibility and automated testing support, and if extra information needs to be conveyed to the user, that extra code only needs to reside in Unity, and will not affect Nux's base atk code.

NOTE: This discussion doesn't affect the unity panel service, as the unity panel service currently uses atk directly, and asside from a few needed improvements, works well already.

The current atk code in unity 3D can be used as a starting point, but it too needs extending to support the full atk/at-spi support, as required for each widget. At a minimum, on-screen location, actions, and states need to be implemented.


Work Items

Work items:
Port the already supported widget set in Unity's atk code over to nux: TODO
Extend nux initialization code to load and use the atk bridge: TODO
Extend current atk code to cover the full atk API as required for the various widgets: TODO
Support any remaining inaccessible nux widgets: TODO
Write tests, both automated, and manual using Orca, to make sure widgets remain accessible, and report their correct states: TODo[themuso] [themuso] Define and perform a series of tests over the lifecycle of the porting work, to make sure user expectations are met: TODO

This blueprint contains Public information 
Everyone can see this information.