Windows

W indows A
Accessibility
ccessibility
ET
.N

Volume 5 / Issue 4

What's New in Windows 7

Automation API
Testing Tools
vs. Accessibility
Guidelines
Internet Explorer 8
New Accessibility
Features
TABLE OF CONTENTS
Features
6 A Letter from the Director of Accessibility
at Microsoft
“Our vision is to create innovative technology that is accessi-
ble to everyone and that adapts to each person’s needs. Acces-
sible technology eliminates barriers for people with disabilities
and it enables individuals to take full advantage of their
capabilities.” —Bill Gates, Chairman, Microsoft Corporation
Rob Sinclair
8 Open Accessibility
Open accessibility is a term that describes both the need for
and the realization of interoperability in assistive technolo-
gies. This article describes open accessibility and the ways
Microsoft and other software development organizations,
like Novell and the AIA, are working together to help user
access technology across platforms.
Norm Hodne
10 Accessibility 101
What is accessibility? Learn about the fundamentals of
accessibility, the people that accessible technologies help,
and the things you can do to help them interact smoothly
and successfully with each other.
Jennifer Linn, Annuska Perkins
16 Windows Automation API 3.0 Overview
The ecosystem of Windows® automation technologies
includes Microsoft® Active Accessibility and Windows
implementations of the UI Automation specification, which
provides more flexible programmatic access to UI elements.
Masahiko Kaneko
22 What's new in Windows 7 Automation API
Windows® 7 offers end–to-end accessibility with better
performance, seamless interoperability, and improved
framework design.
Nazia Zaman, Matthew Karr
28 Creating Accessibility-aware Silverlight 2
Content
This article provides you with guidelines for taking advan-
tage of Silverlight(TM) 2 features and building an accessible
Silverlight application.
Mark Rideout
US subscriptions are US $29.99 for one year. Subscriptions outside the US pay US $44.99. Payments should be made in US dollars drawn on a US
bank. American Express, MasterCard, Visa, and Discover credit cards are accepted. Bill me option is available only for US subscriptions. Back issues
are available. For subscription information, email subscriptions@code-magazine.com or contact customer service at 832-717-4445 ext 10.
Subscribe online at www.code-magazine.com
CoDe Component Developer Magazine (ISSN # 1547-5166) is published bimonthly by EPS Software Corporation, 6605 Cypresswood Drive., Suite
300, Spring, TX 77379. POSTMASTER: Send address changes to CoDe Component Developer Magazine, 6605 Cypresswood Drive., Suite 300,
Spring, TX 77379.
4 Table of Contents
36 Making Custom Controls Accessible
While there are many ways to make custom controls accessi-
ble, choosing the right solution can be tricky. With Microsoft®
technologies, you can implement UI Automation, create or
override properties with Dynamic Annotation, or close the gap
between Microsoft Active Accessibility and UI Automation.
LeAnne Fossmeyer, Michael Bernstein, David Tseng, Vidhya Sriram
42 Microsoft Accessibility Testing Tools
The few informal tests described in this article can expose
a plethora of usability and accessibility shortcomings, over-
sights, and other issues in your application. But how do you
test, assess, and rectify them?
Karl Bridge
48 Internet Explorer 8 New Accessibility
Features
Windows® Internet Explorer® 8 has a lot of cool new
features that make Web page browsing more accessible,
including Caret Browsing, Zoom version 2, high DPI, and
support for ARIA and UI Automation.
JP Gonzalez-Castellan
56 A Pragmatic Approach to WPF Accessibility
A Microsoft MVP shares his experience addressing ac-
cessibility with Windows® Presentation Foundation and
adaptive technology vendors.
Alvin Bruney
62 Writing a UI Automation Provider
UI Automation providers retrieve data about UI elements
for assistive technologies and automation tools. This article
walks through code samples for a UI Automation provider
for a Win32-based custom control.
Brendan McKeon
70 Creating UI Automation Client Applications
UI Automation enables a client application to investigate and
manipulate the UI of all running applications on the system.
This article walks through codes sample that demonstrate the
basic features of UI Automation for client applications.
Matthew Karr
Departments
7 Code Compilers
59 Advertisers Index
www.code-magazine.com
ONLINE QUICK ID 0811012
A Letter from the Director of
Accessibility at Microsoft
“Our vision is to create innovative technology that is accessible
to everyone and that adapts to each person’s needs. Accessible
technology eliminates barriers for people with disabilities and it enables
individuals to take full advantage of their capabilities.”
—Bill Gates, Chairman, Microsoft Corporation
t Microsoft®, our commitment to creating
A the most accessible development platform
in the world runs deep. In 1997, Microsoft
Active Accessibility® was released, providing a
solid foundation for the first generation of ac-
6 A Letter from the Director of Accessibility at Microsoft
cessible applications. Eleven years later, Micro-
soft Active Accessibility still powers millions of
applications around the globe. UI Automation
extends the legacy of Microsoft Active Accessi-
bility and provides support for managed applica-
tion development through the Windows Presen-
tation Foundation (WPF). Windows 7 introduces
additional accessibility features for end users, as
well as UI Automation improvements in perfor-
mance, extensibility, and expanded platform sup-
port.
In November of 2007, Microsoft entered into an
agreement with Novell to extend UI Automation
support to the Linux Accessibility Toolkit. This
agreement, entered into under the Microsoft Com-
munity Promise Specification initiative, extends
UI Automation support to the Linux platform, in-
creasing interoperability and further demonstrat-
ing Microsoft’s commitment to the accessibility
community.
Within these pages you’ll find a number of techni-
cal articles describing the state of accessible ap-
plication development, as well as overviews of the
latest platform support for accessibility in some of
Microsoft’s most exciting products, including Win-
dows Presentation Foundation (WPF), Silverlight,
and Internet Explorer 8. For those interested in
learning the basics, this issue provides a thorough
overview of accessible application development,
intended for those who are completely new to the
space.
Our mission at Microsoft is to help people every-
where meet their full potential. Our commitment
to accessibility is core to who we are—it’s a first-
class service in every product that we ship, and a
set of investments we continue to grow year after
year, product after product. Quite simply, making
the computer easier to see, hear, and use is what
we’re all about.
Rob Sinclair
Director of Accessibility
Microsoft Corporation
www.code-magazine.com
 CODE COMPILERS
component developer magazine

Volume 5 Issue 4

Group Publisher
Markus Egger
Associate Publisher
Rick Strahl
Managing Editor
Ellen Whitney
Content Editors
Kevin Fansler
Writers In This Issue
Michael Bernstein Karl Bridge
Alvin Bruney LeAnne Fossmeyer
JP Gonzalez-Castellan Norm Hodne
Masahiko Kaneko Matthew Karr
Jennifer Linn Brendan McKeon
Annuska Perkins Mark Rideout
Rob Sinclair Vidhya Sriram
David Tseng Nazia Zaman
Technical Reviewers
Markus Egger
Art & Layout
King Laurin GmbH
info@raffeiner.bz.it
Production
Franz Wimmer
King Laurin GmbH
39057 St. Michael/ Eppan, Italy
Printing
Fry Communications, Inc.
800 West Church Rd.
Mechanicsburg, PA 17055
Advertising Sales
Lake Quincy Media LLC
330-677-2440
ssmith@lakequincy.com
msmith@lakequincy.com
Sales Managers
Erna Egger
+43 (664) 151 0861
erna@code-magazine.com
Tammy Ferguson
832-717-4445 ext 26
tammy@code-magazine.com
Circulation & Distribution
General Circulation: EPS Software Corp.
Newsstand: Ingram Periodicals, Inc.
Media Solutions
Worldwide Media
Subscriptions
Circulation Manager
Cleo Gaither
832-717-4445 ext 10
subscriptions@code-magazine.com

US subscriptions are US $29.99 for one year.

Subscriptions outside the US are US $44.99.
Payments should be made in US dollars drawn
on a US bank. American Express, MasterCard,
Visa, and Discover credit cards accepted. Bill
me option is available only for US subscriptions.
Back issues are available. For subscription
information, email subscriptions@code-magazine.com
or contact customer service at 832-717-4445 ext 10.

Subscribe online at
www.code-magazine.com

CoDe Component Developer Magazine

EPS Software Corporation / Publishing Division
6605 Cypresswood Drive, Ste 300, Spring, Texas
77379
Phone: 832-717-4445
Fax: 832-717-4460
ONLINE QUICK ID 0811022

Open Accessibility
Creating a natural user interface requires designers, testers,
and developers working in concert to develop the right support
that makes multi-modal access to an operating system and
applications possible. To assist in this work through the Accessibility
Interoperability Alliance (AIA), Microsoft® released its UI Automation
Specifications with a Community Promise and released testing tools as
open source projects via CodePlex. Microsoft is committed to interoperable
accessibility.
Norm Hodne
normanho@microsoft.com,
+1 (425) 704-7324 Making Technology Accessible number of users access technology in their daily lives.
and Interoperable By making UI Automation freely available Microsoft
Norm Hodne is the Lead
hopes to increase the accessibility of many informa-
Program Manager in the
The purpose of the AIA, formed recently by a coali- tion and communication technologies and to drive
Windows Experience Group tion of the world’s leading information and assistive cross-platform interoperability.
responsible for the accessibility technology (AT) companies, is to foster industry-wide
APIs, accessibility applications, collaboration aimed at reducing many of the barri-
and the speech commanding ers that people with disabilities frequently encounter Microsoft UI Automation:
and dictation features shipped when they try to access information and communi-
in Windows. An Innovative Accessibility Model
cation technologies, including Web sites.
Norm also is a representative UI Automation, a next-genera-
to the INCITS V2 standards To create accessible technol- tion accessibility programming
ogy products today, developers Fast Facts model, simplifies development
organization where he works
on standards related to work across multiple platforms, Novell will implement and reduces costs for AT devel-
application models, and types of UI Automation for Moonlight opers as well as IT application
accessibility.
hardware. To address interoper- developers who want to make
that will provide accessibility
ability issues arising from this their software compatible with
current environment, the AIA is capabilities for Silverlight AT products, such as screen
using a two-part strategy: First, applications on Linux. readers for people who are
the group is working to harmo- blind. UI Automation, which
nize current technologies to in- is available for Windows® XP,
®
teroperate more easily, reducing costs for developers Windows Vista , Windows 7, and is built into the
and creating more usable products for customers. Windows Presentation Foundation (WPF), Silver-
Second, it is working to create a unified accessibility light™ 2 and in additional products, offers developers
model that will serve the entire industry. In order to a richer and more efficient way to convey user inter-
support backwards compatibility and bridging tech- face commands and behaviors to a person through
nology to a full UI Automation their AT product.
implementation, Microsoft re-
leased the UI Automation Speci- UI Automation represents what the interface can
UI Automation fication with the UI Automation, do rather than how it is presented visually, making
allows developers to Microsoft Active Accessibility, it easier for both the application developer and as-
and IAccessibleEx specifications. sistive technology developers to create rich experi-
create assistive technologies ences for their users, including people who are blind.
across all platforms and Microsoft also released the UI Au- AT products built using UI Automation will require
tomation Specification to the AIA, fewer updates and fewer customizations for specific
help a larger number of users an engineering working group applications, because they can intelligently interact
access technology dedicated to making it easier for with user interfaces and controls that have yet to be
in their daily lives. developers to create software, invented.
hardware, and Web-based prod-
ucts that are accessible to people
with disabilities. The UI Auto- Linux Implementation
mation specification describes the latest accessibility
framework technology developed by Microsoft, and Microsoft developed UI Automation from the start
will help developers include advanced accessibility to be compatible for implementation on platforms
functionality into implementations designed for use other than Windows. Starting in 2008, a Novell
on any operating system. team is implementing UI Automation for Linux and
will support SUSE, Ubuntu, and Red Hat and make
UI Automation allows developers to create assistive the code available to other Linux distributions. Their
technologies across all platforms and help a larger implementation work has several goals:

8 Open Accessibility www.code-magazine.com


…Microsoft has agreed to
grant a royalty-free license
for any patents necessary to
implement required portions
of the UI Automation
Specification…
• Make Windows Forms accessible
• Make Moonlight accessible
• Implement UI Automation technologies
Mono is an open source project sponsored by No-
vell which implements software to develop and run
.NET client and server applications on Windows, Li-
nux, Solaris, Mac OS X, and UNIX. Based upon the
Mono Web site, Novell believes that Mono is well
positioned to become the leading application devel-
opment platform for Linux.
Implementing UI Automation on Linux retains the
existing Linux accessibility frameworks. UI Automa-
tion will integrate with the AT-SPI and ATK acces-
sibility frameworks by implementing the client inter-
faces in a bridge that will act as a translation layer
between the frameworks.
Open Source Tools
Keeping with its commitment to help the community
build better experiences for people that use assistive
technology, Microsoft released two new testing tools
as open source on the CodePlex shared code site. The
UI Accessibility Checker (AccChecker) a Microsoft
Active Accessibility® testing tool and UI Automation
Verify (UIA Verify), a UI Automation testing tool,
were released in March 2008. These tools are targeted
at both developers and testers to provide support for
the development process to create and ship products
that meet the needs of assistive technology users.
UI Accessibility Checker
AccChecker enables testers, without prior experience
with accessibility testing, to easily discover accessi-
bility problems with Microsoft Active Accessibility
and other user interface (UI) implementations. Acc-
Checker was born from the realization that existing
Microsoft Active Accessibility tools, such as Inspect
and AccExplorer, provided in-depth details on the
implementation, but no information on the correct-
ness of the accessibility implementation.
UI Automation Verify Test
Automation Framework
UIA Verify is the second accessibility-testing tool
available on CodePlex. UIA Verify is a test automa-
www.code-magazine.com
tion framework that features the User Interface Auto-
mation Test Library (UIA Test Library) and Visual UI
Automation Verify (Visual UIA Verify), the graphical
user interface (GUI) tool. The framework facilitates
manual and automated testing of the Microsoft UI
Automation Provider implementation of a control
or application. The majority of the UIA Verify func-
tionality is provided through a dynamic link library
(such as UIATestLibrary.dll) that
contains the code for testing spe-
cific UI Automation functionality
and supports logging of the test Microsoft is continuing
results. its commitment to accessibility
by creating innovative
About the accessibility APIs and testing
Community Promise tools and releasing them
As a member of the AIA, Micro- with a Community Promise or
soft has agreed to grant a royal-
ty-free license for any Microsoft
as open source projects.
patents necessary to implement
required portions of the UI Au-
tomation Specification, as the specification may
be modified and eventually published by the AIA.
Companies also can implement the latest version of
the UI Automation Specification, which is publicly
available from Microsoft. The Community Promise
that accompanies the UI Automation Specifica-
tion permits royalty-free access to Microsoft patent
claims necessary to implement required portions of
both mandatory and optional parts of the UI Auto-
mation Specification.
AIA Membership
Committed to Open Accessibility The Accessibility
Interoperability Alliance offers
Microsoft is continuing its commitment to accessi- membership to IT and AT
bility by creating innovative accessibility APIs and companies, as well as to non-
testing tools and releasing them with a Community governmental organizations
Promise or as open source projects. This open ap- (NGOs).
proach to accessibility technology allowed Novell
to start development of UI Automation for Linux; The AIA is committed to
it should also enable the development of lower-cost, developing and enhancing
interoperable solutions that ultimately make technol- standards and promoting the
ogy accessible to greater numbers of people through- adoption of the standards to
out the world. support the estimated one in
four computer users that can
benefit from AT applications.
Resources
Accessibility Interoperability Alliance
http://www.accessinteropalliance.org/
AIA Steering
UI Automation Community Promise Committee
http://msdn.microsoft.com/en-us/accessibility/
bb892136.aspx The Steering Committee for the
Accessibility Interoperability
Novell UI Automation Accessibility Project Alliance includes five members.
http://www.mono-project.com/Accessibility The Steering Committee
includes Qualilife and
CodePlex project hosting Web site GWMicro in the AT member
http://codeplex.com/ category, IBM and Adobe
in the IT member category,
Norm Hodne and Microsoft as the at-large
member.
Open Accessibility 9
ONLINE QUICK ID 0811032

Jennifer Linn
jenlin@microsoft.com
Jenny is a writer in the
Windows Experience group
and has been working with
Accessibility for 8 years.
Currently her main focus is
on the MSDN Accessibility
Developers Center
(http://msdn.microsoft.com/en-
us/accessibility/default.aspx)
and internal accessibility
guidelines for product groups.
In her spare time, she enjoys
walking homeless dogs at the
Seattle Humane Society
(http://www.seattlehumane.org)
and reading about history.
Accessibility 101
We know what you’re thinking: Why should I read an article
about the fundamentals of accessibility? Well, if you already know
the percentage of computer users who have disabilities, can name at least
ten different categories of assistive technologies, and can describe the key
concepts involved in designing an accessible application, then you can
probably skip to the next article. However, if you’re unsure what accessible
technology is, then take a few minutes and keep reading. You’ll learn about
the main concepts around accessible technology, the people they help,
and things you can do to help them interact smoothly and successfully
with each other.
What We Mean When We Talk of solutions that makes computer use more comfort-
able for some people and possible for others.
about Accessibility
Accessible technology is a piece of software or hard- As a developer, one of your main concerns is to en-
ware that makes it easier for someone to see, hear, sure that you provide programmatic access to your
and use a technology product, such as a computer application’s user interface elements. This enables the
or a mobile device. It can be an assistive technology assistive technologies to get information about the UI
(AT) product, a specially designed piece of software elements and expose that information to the assistive
or hardware that accommodates technologies, which in turn relays
someone’s disability (or multiple Fast Facts the information to the user. For
disabilities) and enables them to example, important information
interact with a computer. These 57% of computer users that an AT needs includes type,
products are developed to work are likely or very likely to name, location, and current state
with a computer's operating sys- benefit from the use of of a UI element. They also need
tem and software. For example, a accessible technology. to know when changes in the
blind computer user may use as- UI occur. For information about
44% of computer users
sistive technology called a screen how to provide programmatic
reader to navigate an applica-
use some form of access using Microsoft’s acces-
tion’s interface and a computer accessible technology. sibility frameworks, Microsoft
user with ALS may use a word Users seek solutions UI Automation and Microsoft
prediction program to help fa- to make their computers Active Accessibility, see the Ac-
cilitate communicating thoughts easier to use, not for cessibility Developer Center on
and ideas. solutions based on their MSDN (http://msdn.microsoft.
health or disability. com/en-us/accessibility/default.
Many companies offer hardware aspx).
devices, accessories, aids, and Making accessibility options
software applications that fall easier to discover and Programmatic access is just one
under the umbrella of assistive use will result in computers of the key concepts of creating
technology. For an overview of that are easier to use, accessible applications. The
assistive technology categories, more convenient, following section presents ad-
see Descriptions of Assistive and more comfortable for ditional concepts about acces-
Technology Products at the end computer users. sible design.
of this article.

Accessible technology can also be a feature built into
a product that allows someone to adjust the settings
to meet their needs. Examples of accessibility fea-
tures include those that allow a user to increase font
size, change font settings, or choose different colors
for their computer screen display. Other examples
are the option for users to receive announcements
from their computer through sound notifications (a
"ding" when new e-mail messages arrive), or visual
notifications (a modal dialog flashes and beeps when
the user tries to click away from it). So overall it is
helpful to think of accessible technology as a range
Who Benefits from
Accessible Technology?
Although most accessible technology was originally
intended and designed for individuals with severe
disabilities, accessible technology is widely used by
computer users of all abilities today. Among adult
computer users in the United States:
• 1 in 4 has a visual disability.
• 1 in 4 has a dexterity disability.
• 1 in 5 has a hearing disability.

10 Accessibility 101 www.code-magazine.com

Although many accessible technologies are de-
signed to help people with disabilities optimize
their abilities, research shows that the majority of
all computer users can benefit from adjusting their
display, mouse, keyboard, and sound settings. Us-
ers can find accessible technologies helpful when,
for example, recovering from shoulder surgery.
Another example is using a mobile device when
driving a car or riding a bus, in which case, voice
recognition or output can be helpful. According to
a two-part study commissioned by Microsoft and
conducted by Forrester Research (The Market for
Accessible Technology—The Wide Range of Abili-
ties and Its Impact on Computer Use: http://www.
microsoft.com/enable/research/phase1.aspx and
Accessible Technology in Computing—Examining
Awareness, Use, and Future Potential: http://www.
microsoft.com/enable/research/phase2.aspx), the
majority of computer users can benefit from using
accessible technology. Figure 1 shows that 57%
(74.2 million) of computer users are likely or very
likely to benefit from the use of accessible tech-
nology due to having mild or severe disabilities.
Specifically:
• 40% (51.6 million) of computer users are likely
to benefit from the use of accessible technol-
ogy due to mild difficulties/impairments.
• 17% (22.6 million) of computer users are
very likely to benefit from the use of accessible
technology due to severe difficulties/impair-
ments.
What does that mean for you? That’s a lot of people
who will depend on the accessibility of your appli-
cation. Accessible technology has the potential to
improve computer use for a wider audience because
it makes computers easier to use. A key component
of encouraging the use of accessible technology is to
make it easier to find and highlight the functionality
and benefits rather than the impairments they seek
to ameliorate.
To get a more personal view of how accessible tech-
nology helps people, here is a look at some profiles
of computer users today. The following profiles de-
scribe how people with common types of disabili-
ties might use accessible technology to help them in
their job and everyday life. Each profile is followed
by design and development considerations for cre-
ating accessible applications.
Computer Users with Vision Disabilities
Vision disabilities include low vision, color blindness,
and blindness. Among adult computer users in the
United States, 1 in 4 (27%) have a vision disability.
There are many options for individuals with vision
disabilities to modify the computer displays and ap-
pearance so it is more legible, or receive information
through sound or touch. Those who are blind can-
not use a computer monitor, but have the option to
receive information from their computers through
hearing or touch offered through screen readers and
Braille displays.
www.code-magazine.com
Profile: Low Vision
Olivia, a managing editor, has had low vision as a
result of glaucoma. Low vision is a term commonly
used to mean partial sight, or sight that isn't fully
correctable with surgery, pharmaceuticals, contact
lenses, or glasses. It includes moderate vision im-
pairment, such as tunnel vision or blind spots, as
well as legal blindness and almost total blindness.
Sometimes it involves a lack of acuity, meaning that
objects appear blurred. Other times, it involves a re-
duced ability to distinguish colors, see contrasts, or
determine spatial relationships among objects.
Olivia can see type and images on her computer
screen when they are enlarged to about 1.5 inches in Annuska Perkins
height. As her glaucoma advances, print will appear
Annuska Perkins has called
faded and words will be difficult to read. By using a
Microsoft home since 1998.
screen enlargement program, Olivia can effectively
view and interact with documents and tools that are
She began as a program
an important part of her job. In addition, she uses manager for MSN Side-
high DPI for sharpness and turns on the high con- walk.com, an online city guide.
trast accessibility setting with the enlarger because Afterwards, she joined the
having the magnified screen displayed in black on Accessibility group, where her
white improves the readability for her. role evolved from program
manager, to product planner,
While she is currently able to use a mouse, she to User Experience expert.
knows she needs to learn more about keyboard ac- Check out her blog and projects
cess because she will have to switch to keyboard use on the MSDN Accessibility site:
as her vision continues to deteriorate. Even with her http://msdn.microsoft.com/
high level of magnification, Olivia tends to sit ex- accessibility.
tremely close to the monitor. In addition, due to the
increased level of magnification, she needs to scroll
horizontally and vertically many times in order to
see what other customers will see all at once on the
screen. She has to work to remember where items
are in order to retain context, for example, where
controls and text are meant to be seen together but
aren’t due to her level of magnification. These factors
make her prone to headache, eyestrain, and fatigue.
Figure 1: This chart also shows the percentages of computer users who are likely or very likely
to benefit from the use of accessible technology due to a range of mild to severe difficulties and
impairments.
Accessibility 101 11

Accessibility
Resources
Whether you’re developing
accessibility technology or
designing an accessible Web
site, you’ll need a bounty
of resources. The following
are tools, specifications,
guidelines, and resources you
can use to make computer-
based content and technology
accessible to all.
Tools
accChecker is a Microsoft ® years, ever since a car accident. He can see a little
Active Accessibility® API test
tool with additional accessibility
test automation features. http://
www.codeplex.com/AccCheck
UIA Verify is a UI Automation
API test tool with additional
accessibility test automation
features. http://www.codeplex.
com/UIAutomationVerify
Fujitsu ColorDoctor simulates
display content according
to grayscale and color
characteristics. http://design.
fujitsu.com/en/universal/
assistance/colordoctor/
Wat-C Color Contrast Analyser
tests whether foreground and
background color combinations
provide good color visibility.
It also simulates certain
visual conditions such as
color blindness. http://www.
paciellogroup.com/resources/
contrast-analyser.html
12 Accessibility 101
Design and Development Considerations:
Low Vision
To ensure that a low vision computer user’s experience
is equal to that of users not requiring assistive technol-
ogy, keep the following design guidelines in mind:
• Ensure that individual objects in the UI can
easily be distinguished and that each object has
a clear, unique, meaningful label that makes
sense when read out of context.
• Respect the Windows user system settings re-
lated to accessibility, such as color choices and
icon sizes.
• Do not specify hard-coded fonts. Instead, en-
able users to change the font size and face. Use
default font size of 10 points or larger.
• Ensure that users can set color settings on a sys-
tem-wide basis and have them respected within
their applications.
• Check for appropriate use of color in fore-
ground and background combinations.
• Ensure programmatic access to UI elements.
• Provide full keyboard access to your applica-
tions functionality via keyboard focus, access
keys, and shortcut keys.
• Ensure that the UI adapts well to high DPI set-
tings.
Profile: Blindness
Jorge, a financial analyst, has been blind for five
light but no images and uses a guide dog to navigate
within buildings and outside.
After his car accident, Jorge went through extensive re-
habilitation to learn how to use assistive technology to
continue working. He uses a screen reader to relay in-
formation from his computer, along with an earphone
so that he can work without disturbing co-workers. He
also uses a refreshable Braille display to supplement
the voice information he receives from the screen
reader. He uses Braille copies of the firm’s reports and
analyses when needing to work with hardcopy.
Jorge also relies on the keyboard to navigate and on
memorization of spatial relationships and keyboard
shortcuts. Consistency and predictability within and
across applications are great aids to his productivity.
Design and Development Considerations: Blind
Computer Users
To ensure that a blind computer user’s experience is
equal to that of users not requiring assistive technol-
ogy, keep the following design guidelines in mind:
• Provide programmatic access to the UI for as-
sistive technologies such as screen readers and
Braille displays.
• Use text that is easily understood and that
makes sense when read out loud with no visual
association. Supply alternative text for images.
• Group information logically and place the main
idea of each paragraph in the first sentence.
• Ensure that your feature can be used without
a mouse. Provide powerful, intuitive keyboard
shortcuts.
• Support prefix navigation (typing the first few
characters to reach a known destination).
• Ensure tab orders are logical.
Computer Users with Hearing Disabilities
Hearing disabilities encompass a wide range of con-
ditions—from slight hearing loss to deafness. People
who have hearing disabilities might be able to hear
some sound, but might not be able to distinguish
words. Among adult computer users in the United
States, 1 in 5 (21%) have a hearing disability.
Profile: Deafness
Tim is a network architect and has been deaf from
birth. He works very effectively with his computer; in
addition to using e-mail, he uses instant messaging to
communicate with his colleagues and co-workers.
Since he is deaf, he does not need sound information
from his computer so he has turned sound off com-
pletely. To be sure that he gets information visually
that would otherwise be conveyed by sound, he uses
SoundSentry to provide visual warnings for system
sounds, ShowSounds to display captions for speech
and sounds, and Visual Notification to provide visual
warnings when features are turned on or off. He has
turned on the captioning features where available in
products.
Tim also uses a TTY (Telephone Typewriter) device
(replacing a handset phone) and a text pager with
the keyboard set to vibrate attached to his belt. This
pager does text to voice for outgoing messages and
voice to text for incoming messages.
Design and Development Considerations: Deaf
Computer Users
To ensure that a deaf computer user’s experience is
equal to that of users not requiring assistive technol-
ogy, keep the following design guidelines in mind:
• If your application plays video, provide a way
to turn on and off captioning. Use the Show-
Sounds system setting as a cue to the user’s
preference.
• Be sure that your feature works with Visual
Notification (Sound Sentry) system setting so
that users receive visual cues in lieu of sound
notifications.
• Providing visual customizations can greatly en-
hance the customer experience. For example,
provide ways for users to change the size, color,
and flashing characteristics of sound alternatives.
Computer Users with Dexterity Disabilities
Individuals with dexterity disabilities experience
pain, discomfort, or complete loss of feeling in their
www.code-magazine.com
fingers, hands, wrists, or arms, making it difficult to
use a standard keyboard or mouse. Among adult
computer users in the United States, 1 in 4 (26%)
have a dexterity difficulty. Dexterity difficulties and
impairments can be caused by a wide range of com-
mon illnesses and accidents such as carpal tunnel,
arthritis, stroke, cerebral palsy, Parkinson's disease,
multiple sclerosis, loss of limbs or digits, spinal cord
injuries, and repetitive stress injury, among others.
Profile: Repetitive Stress Injury
Jason is a medical researcher, studying molecular bi-
ology. In high school, Jason played tennis and has
been playing regularly for the last 15 years. Unfortu-
nately, years of playing have been hard on his joints
and he recently had surgery for a rotator cuff injury.
Jason is undergoing physical therapy to strengthen
the muscles of his rotator cuff. He is able to con-
duct his work by using a speech recognition program
installed on his computer. He also answers phone
calls right on his computer and has a mobile phone
equipped with voice command.
Design and Development Considerations: Dexter-
ity Disabilities
To ensure that a computer user’s experience is equal
to that of users not requiring assistive technology,
keep the following design guidelines in mind:
• Provide full keyboard access to your applica-
tions functionality via keyboard focus, access
keys, and shortcut keys.
• Enable programmatic access.
Computer Users with Learning Disabilities
Learning disabilities can range from conditions such
as dyslexia and attention deficit disorder to intellec-
tual disabilities. Some individuals with learning dis-
abilities are better able to process information when
it is presented to them in a form and at a pace that is
appropriate to them individually. During the learning
process, many people with learning disabilities ben-
efit from having a multisensory experience of audio
speech paired with a visual representation. Reduc-
ing visual and auditory distractions can also aid the
learning process for many people.
Profile: Dyslexia
Amelia is a fifth grade student who wants to be a
photographer when she grows up. A bright and ca-
pable student, she struggles with dyslexia and admits
that she used to be scared of reading books aloud in
class. Fortunately, Amelia's parents and teacher rec-
ognized the dyslexia early, and the school’s assistive
technology specialist helped find a software program
that helps her read text in class and at home.
Amelia's computer is now outfitted with a reading
and writing software program. The program high-
lights text as it reads the text aloud and includes a
www.code-magazine.com
talking dictionary, phonetic spell-
checking, word-prediction, and
homophone support to address Microsoft’s Commitment
key aspects of reading and writ-
ing processes. In addition, she has to Accessibility:
turned on features in the operat- “Our vision is to create
ing system to help her focus on
tasks, and she sometimes uses a innovative technology that is
headset to block out background accessible to everyone and that
noise.
adapts to each person's needs.
Design and Development Con- Accessible technology eliminates
siderations: Learning Disabili-
ties barriers for people with disabilities
and it enables individuals
To ensure that a computer user
with a learning disability has an to take full advantage of their
end-user experience equal to that capabilities.”
of users not requiring assistive
technology, keep the following
—Bill Gates, Microsoft Corporation
design guidelines in mind:
• Enable users to perceive information in mul- Microsoft® Internet Explorer®
tiple ways, such as visually and aurally. Developer Toolbar provides
• Avoid unnecessary animations and distracting a variety of tools for quickly
UI elements. Provide a way to turn off anima- creating, understanding,
tions. and troubleshooting Web
• Adhere to common UI visual guidelines to pro- pages. http://www.microsoft.
vide a consistent and intuitive experience. For com/downloads/details.
more information, see the Windows Vista User aspx?FamilyID=e59c3964-
Experience Guidelines on MSDN (http://msdn.
672d-4511-bb3e-
microsoft.com/en-us/library/aa511258.aspx).
2d5e1db91038&DisplayLang=en
Aging Computer Users
ARIA Resources
Do you have trouble seeing things on your com- Web Accessibility Initiative WAI
puter screen? By the time we reach our fifties, two- provides guidelines, resources,
thirds of us have vision, hearing, or dexterity im- and strategies to make the
pairments that will impact our use of the computer. Web accessible to people with
Some functional losses are accelerated by the onset disabilities.
of age-related degenerative diseases and ailments, http://www.w3.org/WAI/
including hypertension, osteoporosis, diabetes, and
macular degeneration. Disabling conditions, includ- WAI-ARIA Primer provides
ing arthritis and orthopedic impairments resulting background on the accessibility
from sports, vehicle, and occupational injuries ex- problem space solved with ARIA.
perienced earlier in life tend to manifest themselves http://www.w3.org/TR/wai-aria-
as the body ages. primer/
ARIA Specification
Profile: Aging Population
defines the ARIA syntax.
Fran is a retired librarian and spends her mornings http://www.w3.org/TR/wai-aria/
hosting a book club and taking an aerobics class. She WAI-ARIA Best Practices
and her husband, Roy, bought a new computer when describes how to develop
their kids started sharing photos and videos of their accessible rich Web applications
families. using ARIA. http://www.w3.org/
TR/wai-aria-practices
Both Fran and Roy have difficulty reading text on the
computer screen and prefer different color schemes
and settings. Roy prefers larger icons and a slower
mouse cursor, while Fran prefers photos of her
grandchildren on the desktop, large blue text, and a
stylized mouse pointer for easier visibility. They each
set up a profile with their preferred settings so they
can share files while personalized settings make it
easier for each to read text on the computer screen.
Accessibility 101 13

Among adult computer users
in the United States:
• 1 in 4 has a vision difficulty
• 1 in 4 has a dexterity difficulty
• 1 in 5 has a hearing difficulty
Microsoft Resources
Profiles of Accessibility
in Action details inspiring
profiles of accessibility in
action among five individuals
using accessibility options and
assistive technology products
on their computers. http://
www.microsoft.com/enable/
profiles/default.aspx
Microsoft® Accessibility
Developer Center offers
guidance, essential
information, and tools
for developing accessible
applications and writing
accessible code. http://msdn.
microsoft.com/accessibility
Microsoft Active Accessibility
on MSDN Library describes
Microsoft Active Accessibility
in detail. http://msdn.
microsoft.com/en-us/library/
ms697707(VS.85).aspx
UI Automation Specification
and Community Promise
http://msdn.microsoft.com/
en-us/accessibility/bb892133.
aspx
Microsoft Accessible
Technologies http://www.
microsoft.com/enable/
14 Accessibility 101
They now use the computer for nearly every aspect
of their lives including banking and filling prescrip-
tions while traveling, managing digital photos, and
communicating with family members using e-mail
and video phone conferencing.
Design and Development Con-
siderations:
The Aging Population
To ensure that customers like
Fran and Roy have an end-user
experience equal to that of users
not requiring assistive technolo-
gy, keep the design guidelines for
low vision, dexterity, and cogni-
tive changes in mind.
High-level Concepts for Creating
Accessible UI
In addition to providing programmatic access and
following the considerations described above, what
are the high-level concepts to consider to ensure that
your application’s UI is accessible?
• Ensure that all functionality of the UI is usable
by people with vision, dexterity, hearing, and
cognitive limitations.
• Make your application’s default UI accessible.
For example, chose a color palette with con-
trasting colors and avoid color combinations
that are not perceivable by people with color-
blindness.
• Make your UI flexible and customizable so that
users can optimize the UI. As the user specifies
their preferences through Windows settings en-
sure your application responds appropriately.
(The user’s settings are discoverable program-
matically through the Windows SystemParam-
etersInfo() API: http://msdn.microsoft.com/
en-us/library/ms724947(VS.85).aspx.)
• Make the UI easy to navigate regardless of the
input device. Ensure that it can be accessed
whether a customer uses a mouse, keyboard,
on-screen keyboard, or speech recognition.
Descriptions of Assistive Technology
Products
Assistive technology products are designed to pro-
vide additional accessibility to individuals who
have physical or cognitive difficulties, impairments,
and disabilities. When selecting assistive technol-
ogy products, it is crucial to find products that are
compatible with the computer operating system and
programs on the particular computer being used.
More than 100 companies offer hardware devices,
accessories, aids, and software applications that fall
under the umbrella of assistive technology. These al-
ternative input products include speech recognition
systems, alternative keyboards, electronic pointing
devices, sip-and-puff systems, wands, sticks, joy-
sticks, trackballs, and touch screens; and alternative
output systems such as speech synthesizers, Braille
embossers and displays, and screen readers.
Below are descriptions of the various types of assis-
tive technology products that are currently available
on the market today. For more information, you can
search the catalog of assistive technology products
for products compatible with the Windows operat-
ing system on the Microsoft.com/enable site.
Screen magnifiers (or screen enlargers) work like
a magnifying glass for the computer by enlarging a
portion of the screen, increasing legibility, and mak-
ing it easier to see items on the computer. Some
screen magnifiers allow a person to zoom in and
out on a particular area of the screen.
Screen readers are used to verbalize, or "speak," items
on the screen including text, graphics, control buttons,
and menus into a computerized voice that is spoken
aloud. In essence, a screen reader transforms a graphic
user interface (GUI) into an audio interface. Screen
readers are essential for computer users who are blind.
Speech recognition, or voice recognition, pro-
grams allow people to give commands and enter
data using their voices rather than a mouse or key-
board. Voice recognition systems use a microphone
attached to the computer, which can be used to cre-
ate text documents such as letters or e-mail mes-
sages, browse the Internet, and navigate among ap-
plications and menus by voice.
Refreshable Braille displays provide tactile output
of information represented on the computer screen.
A Braille "cell" is composed of a series of dots. The
pattern of the dots and various combinations of the
cells are used in place of letters. Refreshable Braille
displays mechanically lift small rounded plastic or
metal pins as needed to form Braille characters. The
user reads the Braille letters with his or her fingers,
and then, after a line is read, can refresh the display
to read the next line.
Braille embossers transfer computer generated
text into embossed Braille output. Braille transla-
tion programs convert text scanned-in or generated
via standard word processing programs into Braille,
which can be printed on the embosser.
On-screen keyboards provide an image of a stan-
dard or modified keyboard on the computer screen
that allows the user to select keys with a mouse,
touch screen, trackball, joystick, switch, or elec-
tronic pointing device. On-screen keyboards often
have a scanning option that highlights individual
keys that can be selected by the user. On-screen
keyboards are helpful for individuals who are not
able to use a standard keyboard due to dexterity or
mobility difficulties.
Alternative input devices allow individuals to con-
trol their computers through means other than a
standard keyboard or pointing device. Examples
include:
www.code-magazine.com
 • Alternative keyboards feature larger- or hear what they are typing and also provide a spo-
smaller-than-standard keys or keyboards, al- ken voice for individuals who cannot communicate
ternative key configurations, and keyboards orally, but can communicate their thoughts through
for use with one hand. typing.
• Electronic pointing devices are used to con-
trol the cursor on the screen without use of Talking and large-print word processors are soft-
hands. Devices used include ultrasound, infra- ware programs that use speech synthesizers to pro-
red beams, eye movements, nerve signals, or vide auditory feedback of what is typed. Large-print
brain waves. word processors allow the user to view everything
• Sip-and-puff systems are activated by inhal- in large text without added screen enlargement.
ing or exhaling.
• Wands and sticks are worn on the head, held
in the mouth or strapped to the chin and used Conclusion
to press keys on the keyboard.
• Joysticks are manipulated by hand, feet, chin, Whether you know it or not, many people with
etc., and used to control the cursor on screen. disabilities probably use your applications. Under-
• Trackballs are movable balls on top of a standing their needs is the first step to designing ac-
base that can be used to move the cursor on cessible applications that can be used by everyone.
screen.
• Touch screens allow direct selection or activa- Jennifer Linn
tion of the computer by touching the screen, Annuska Perkins
making it easier to select an option directly
rather than through a mouse movement or
keyboard. Touch screens are either built into
the computer monitor or can be added onto a
computer monitor.
• TTY/TDD conversion modems are connected
between computers and telephones to allow
an individual to type a message on a computer
and send it to a TTY/TDD telephone or other
Baudot-equipped device.
• Light signaler alerts monitor computer sounds
and alert the computer user with light signals.
This is useful when a computer user cannot
hear computer sounds or is not directly in
front of the computer screen. As an example, a
light can flash alerting the user when a new e-
mail message has arrived or a computer com-
mand has completed.

Keyboard filters are typing aids such as word pre-

diction utilities and add-on spelling checkers that
reduce the required number of keystrokes. Key-
board filters enable users to quickly access the let-
ters they need and to avoid inadvertently selecting
keys they don't want.

Reading tools and learning disabilities programs

include software and hardware designed to make
text-based materials more accessible for people
who have difficulty with reading. Options can in-
clude scanning, reformatting, navigating, or speak-
ing text out loud. These programs are helpful for
those who have difficulty seeing or manipulating
conventional print materials; people who are devel-
oping new literacy skills or who are learning Eng-
lish as a foreign language; and people who compre-
hend better when they hear and see text highlighted
simultaneously.

Text-to-Speech (TTS) or speech synthesizers re-

ceive information going to the screen in the form
of letters, numbers, and punctuation marks, and
then "speak" it out loud in a computerized voice.
Using speech synthesizers allows computer users
who are blind or who have learning difficulties to

www.code-magazine.com Accessibility 101 15

ONLINE QUICK ID 0811042
Masahiko Kaneko
mkaneko@microsoft.com
Masahiko Kaneko is a Senior
Program Manager on the
User Interface Platform Team
in the Windows Experience
Division. A program manager
in accessibility at Microsoft
for more than 10 years, he
has been involved with several
releases of the Windows
Operating System as well as
many other Microsoft products.
Masahiko enjoys road biking,
XC skiing, music, and movies
in his free time but most of it is
now dedicated to his first son
who is turning 18 months in
September.
Figure 1: Microsoft Active Accessibility uses OLEACC.dll to communicate between clients, like
screen readers, and servers, such as Windows applications.
16 Windows Automation API 3.0 Overview
Windows Automation
API 3.0 Overview
While general accessibility requirements (such as font colors
in UI rendering) are important, programmatic access to the
graphical user interface (GUI) is a crucial element to improving
accessibility. On the Windows® operating system, Microsoft® Active
Accessibility® and User Interface (UI) Automation support this programmatic
access. This article provides a quick overview of Windows Automation API
3.0 featured in Windows 7.
Windows Automation API 3.0: cessibility refers to the application or control offering
the UI for accessibility as the server, while UI Auto-
a Bit of History
mation refers to this as the provider.
Today, the Windows operating system offers two
application programming interfaces (API) specifi- Microsoft Active Accessibility offers a single COM
cations for user interface accessibility and software interface with a fixed, small set of properties. UI Au-
test automation. The legacy API, Microsoft Active tomation offers a richer set of properties, as well as a
Accessibility, was introduced to set of extended interfaces called
Windows 95 as a platform add- Fast Facts control patterns to manipulate
on in 1996. The new API is a automation elements in ways
Windows implementation of UI Automation is designed Microsoft Active Accessibility
the User Interface Automation for a modern and evolving cannot.
specification called UI Automa- user experience. The latest
tion. UI Automation was intro- Windows 7 implementation While UI Automation previ-
duced in Windows Vista® and ously had both managed and
helps applications be
.NET Framework 3.0. unmanaged API for providers,
compatible with existing the original release had no un-
Although the two technologies accessibility applications managed interfaces for clients.
are different, the basic design while fully supporting new With Windows Automation API
principles are similar. Both ex- user interface paradigms. 3.0, you can finally write UI Au-
pose the UI object model as UI Automation is actually tomation clients entirely in un-
a tree hierarchy rooted at the faster and more reliable managed code.
desktop. Microsoft Active Ac-
than its predecessor,
cessibility represents individual The new API also provides sup-
UI elements as accessible ob-
Microsoft Active Accessibility. port for transitioning from Mi-
jects, and UI Automation rep- crosoft Active Accessibility serv-
resents them as automation elements. Both refer to ers to UI Automation providers. The IAccessibleEx
the accessibility tool or software automation pro- interface enables legacy Microsoft Active Accessibil-
gram as the client. However, Microsoft Active Ac- ity servers to add support for specific UI Automation
patterns and properties without rewriting their whole
implementation. The specification also allows in-pro-
cess Microsoft Active Accessibility clients to access UI
Automation provider interfaces directly, rather than
through UI Automation client interface.
The ecosystem of Windows automation technologies,
now called Windows Automation API, includes
classic Microsoft Active Accessibility and Windows
implementations of the UI Automation specifica-
tion. At Microsoft, the UI Automation specification
is implemented on Windows Vista, Windows Server
2008, Windows Presentation Foundation (WPF),
XPS Viewers, and many other upcoming Microsoft
products. Windows 7, Windows Internet Explorer 8,
and Silverlight 2.0 are joining the pack soon.
www.code-magazine.com
The Architecture:
Microsoft Active Accessibility,
UI Automation, and Interoperability
The goal of Microsoft Active Accessibility is to ex-
pose basic information about custom controls such
as control name, location on screen, and type of con-
trol, as well as state information such as visibility and
enabled/disabled status. The UI is represented as a
hierarchy of accessible objects; changes and actions
are represented as WinEvents. The following com-
ponents comprise the Microsoft Active Accessibility
architecture:

• Accessible Object—A logical UI element (such

as a button) that is represented by an IAcces-
sible COM interface and an integer ChildID. Figure 2: User Interface Automation (UI Automation) uses the UI Automation Core to communicate
• WinEvents—An event system that enables serv- between clients and providers and uses proxies to communicate with legacy implementations.
ers to notify clients when an accessible object
changes.
• OLEACC.dll—run-time, dynamic-link library
that provides the Microsoft Active Accessibility
API and the accessibility system framework.

For Microsoft Active Accessibility, the system com-

ponent of the accessibility framework (OLEACC.
dll) helps the communication between accessibility
tools and applications (Figure 1). The applications
(Microsoft Active Accessibility servers) provide UI
accessibility information to tools (Microsoft Active
Accessibility clients), which interact with the UI on
behalf of users. The code boundary can be a pro-
grammatic or process boundary.

The goal of UI Automation is similar but broader,

as described later in this article. From an archi- Figure 3: UIA-to-MSAA Bridge enables Microsoft Active Accessibility clients to access UI
tecture point of view, UI Automation loads the UI Automation providers.
Automation Core component into both the acces-
sibility tools’ and applications’ processes (Figure
2). This component manages cross-process com-
munication and provides higher level services. This
core component enables bulk fetching or caching
of properties, which improves the cross-process
performance over Microsoft Active Accessibility
implementation.

Interoperability between
Microsoft Active Accessibility-based and
UI Automation-based Applications
The UIA-to-MSAA Bridge enables Microsoft Active
Accessibility clients to access UI Automation provid-
ers by converting the UI Automation object model to
a Microsoft Active Accessibility object model (Fig- Figure 4: The MSAA-to-UIA Proxy enables UI Automation clients to access Microsoft Active
ure 3). Similarly, the MSAA-to-UIA Proxy (Figure 4) Accessibility servers.
translates Microsoft Active Accessibility-based server
object models for UI Automation clients.
Limitations of
Now with IAccessibleEx, you can also improve ex- Microsoft Active Accessibility
isting Microsoft Active Accessibility server imple-
mentations by adding only required UI Automation Microsoft designed the Microsoft Active Acces-
object model information. The MSAA-to-UIA Proxy sibility object model about the same time as Win-
takes care of incorporating the added UI Automa- dows 95 released. The model is based on “roles”
tion object model. defined a decade ago, and you cannot support new

www.code-magazine.com Windows Automation API 3.0 Overview 17

 UI behaviors or merge two or more roles togeth- is broader in scope than just an interface definition.
er. There is no text object model, for example, to UI Automation provides:
help assistive technologies deal with complex Web
content. • An object model and functions that make it easy
for client applications to receive events, retrieve
Another limitation involves navigating the object property values, and manipulate UI elements.
model. Microsoft Active Accessibility represents the • A core infrastructure for finding and fetching
UI as a hierarchy of accessible objects. Clients navi- across process boundaries.
gate from one accessible object to another using • A set of interfaces for providers to express tree
interfaces and methods available structure and some general properties.
from the accessible object. Serv- • A set of interfaces for providers to express other
ers can expose the children of properties and functionality specific to the con-
The ecosystem of Windows an accessible object with prop- trol type.
automation technologies, now erties of the accessible object
or with the IEnumVARIANT To improve on Microsoft Active Accessibility, UI Au-
called Windows Automation API, COM interface. Clients, how- tomation aims to address the following goals:
includes classic Microsoft ever, must be able to deal with
both approaches for any server. • Enable efficient out-of-process clients, while
Active Accessibility and This ambiguity means extra work continuing to allow in-process access.
Windows implementations of the for client implementers, and the • Expose more information about the UI in a way
complexity can contribute un- that allows clients to be out-of-process.
UI Automation specification. foreseen problems of server im- • Coexist with and leverage Microsoft Active Ac-
plementations. cessibility without inheriting its limitations.
• Provide an alternative to IAccessible that is
Just as important is the inability to extend Micro- simple to implement.
soft Active Accessibility properties or functions
without breaking or changing the IAccessible COM The implementation of the UI Automation specifica-
interface specification. The result is that you cannot tion in Windows features COM-based interfaces and
expose new control behavior or properties through managed interfaces.
the object model. The object model tends to be both
static and stagnant.
UI Automation Elements

UI Automation UI Automation exposes every piece of the UI to cli-

Specification
UI Automation offers
much better performance for
The UI Automation specification
out-of-process client practices provides more flexible program-
(400% or faster for some scenarios matic access to UI elements on
the desktop, enabling assistive
than Microsoft Active Accessibility technology products such as
running out-of-process), screen readers to provide infor-
mation about the UI to end us-
while adding richness and ers and to manipulate the UI by
flexibility to support the means other than standard input.
latest user interface designs. The specification can be support-
ed across platforms other than
Microsoft Windows.
The implementation of that specification in Win-
dows is also called UI Automation. UI Automation
ent applications as an Automation Element. Provid-
ers supply property values for each element; invoking
a method on an element invokes the corresponding
method on the provider. Elements are exposed as a
tree structure, with the desktop as the root element.
Automation elements expose common properties of
the UI elements they represent. One of these proper-
ties is the control type, which describes its basic ap-
pearance and functionality (for example, a “button”
or a “check box”).
UI Automation Tree
The Automation Tree represents the entire UI: the
root element is the current desktop and child elements
are application windows. Each of these child elements

Figure 5: This UI Automation tree represents all elements on the Run dialog.

18 Windows Automation API 3.0 Overview www.code-magazine.com

can contain elements representing menus, buttons,
toolbars, and so on. These elements in turn can con-
tain elements like list items, as shown in Figure 5.
UI Automation providers for a particular control
support navigation among the child elements of
that control. However, providers are not concerned
with navigation between these control subtrees.
This is managed by the UI Automation core, using
information from the default window providers.
To help clients process UI information more ef-
fectively, the framework supports alternative views
of the automation tree: (1) Raw View, (2) Control
View, and (3) Content View. As Table 1 shows, the
type of filtering determines the views, and the client
defines the scope of a view.
UI Automation Properties
The UI Automation specification defines two kinds
of properties: (1) automation element properties
and (2) control pattern properties. Automation
element properties apply to most controls, provid-
ing fundamental information about the element
such as its name. Examples are listed in Table 2.
Control pattern properties apply to control pat-
terns, described next.
Unlike with Microsoft Active Accessibility, every UI
Automation property is identified by a GUID and a
programmatic name, which makes new properties
easier to introduce.
Automation Tree Description
Raw View The full tree of automation element objects for which the desktop is the root.
Control View A subset of the raw view that closely maps to the UI structure as the user
perceives it.
Content View A subset of the control view that contains content most relevant to the user,
like the values in a drop-down combo box.
Table 1: Types of Automation Tree views.
UI Automation Control Patterns
A control pattern describes the attributes and func-
tionality of an automation element. For example, a
simple “clickable” control like a button or hyperlink
should support the Invoke control pattern to repre-
sent the “click” action.
Each control pattern is a canoni-
cal representation of possible UI
features and functions, as shown From a UI Automation
in Table 3. There are 22 control client’s perspective,
patterns defined to date, some
of which are new in Windows 7, there is no difference between
and the Windows Automation UI Automation providers and
API can support custom control
patterns. Unlike with Microsoft Microsoft Active Accessibility
Active Accessibility role or state servers that implement
properties, one automation ele-
ment can support multiple UI IAccessibleEx correctly.
Automation control patterns.
UI Automation Control Types UI Automation
Community Promise
A control type is another automation element prop- Specifications
erty that specifies a well-known control that the ele-
ment represents. Currently, UI Automation defines The entire accessibility
38 control types, including; Button, Check Box, framework specification is
Combo Box, Data Grid, Document, Hyperlink, Im- available under Microsoft Open
age, ToolTip, Tree, and Window. Specification Promise
(http://www.microsoft.com/
Before you can assign a control type to an element, interop/osp/default.mspx).
the element needs to meet certain conditions, includ- Windows Automation API 3.0
is a Windows implementation
of the specification.
Find more information at the
MSDN Accessibility Developer
Center http://msdn.microsoft.
com/accessibility/

Property Description
AutomationId A string containing the UI Automation identifier (ID) for the automation
element. The AutomationId property of an element is expected to be the
same in any instance of the application, regardless of the local language.
BoundingRectangle The coordinates of the rectangle that completely encloses the element. The
returned rectangle is expressed in physical screen coordinates.
Name A string for the text representation of the element. This string should always
be consistent with the label text on screen. For example, the Name property
must be “Browse…” for the button labeled “Browse…”.
ControlType A ControlType of the automation element, which defines characteristics of
the UI element by well known UI control primitives such as button or check
box.
FrameworkId A string for the name of the underlying UI framework. FrameworkId enables
client applications to apply special cases to a particular UI framework.
Examples of property values are "Win32", "WinForm", and "DirectUI".
Table 2: Examples of UI Automation Element properties.

www.code-magazine.com Windows Automation API 3.0 Overview 19


Supported Platform
UI Automation is supported on
the following platforms to date:
• Windows XP
• Windows Vista
• Windows Server 2003
• Windows Server 2008
• Windows 7
Table 3: Examples of UI Automation Control Patterns.
Control Description
Dock Used for controls that can be docked in a container, like toolbars or tool
palettes.
ExpandCollapse Used for controls that can be expanded or collapsed, like the File menu.
Grid Used for controls that support grid functionality such as sizing and moving
to a specified cell, without header information. The “large icon view” in
Windows® Explorer is an example of a control that follows the Grid control
pattern.
GridItem Used for controls within grids. For example, each cell in Explorer’s “details
view” could follow the GridItem pattern.
Invoke Used for controls that can be invoked, such as a button.
ItemContainer Used for controls that hosts a number of children that may be virtualized.
MultipleView Used for controls that can switch between multiple representations of the
same set of information, data, or children. For example, a list view control
where data is available in thumbnail, tile, icon, list, or detail views.
RangeValue Used for controls that have a range of values. For example, a spinner control
containing years might have a range of 1900 to 2010, while another spinner
control presenting months would have a range of 1 to 12.
Scroll Used for controls that can scroll.
Selection Used for selection container controls. For example, list boxes and combo
boxes.
Table Used for controls those have a grid as well as header information. For
example, Microsoft Excel worksheets.
Text Used for edit controls and documents that expose textual information.
Toggle Used for controls where the state can be toggled. For example, check boxes
and checkable menu items.
Transform Used for controls that can be resized, moved, and rotated. Typical uses for
the Transform control pattern are in designers, forms, graphical editors, and
drawing applications.
VirtualizedItem Used for a virtualized object in a container that supports the ItemContainer
pattern.
Window Used for controls that provide fundamental window-based functionality
within a traditional graphical user interface.

ing a particular automation tree structure, property IAccessibleEx Interface

values, control patterns, and events. However, you
are not limited to these. You can extend a control
with custom patterns and properties as well as with
the pre-defined ones.
The total number of pre-defined control types is sig-
nificantly lower than Microsoft Active Accessibil-
ity accRole definitions, because you can combine UI
Automation control patterns to express a larger set
of features while Microsoft Active Accessibility roles
cannot. You can also customize the description of
control type by LocalizedControlType property while
keeping the baseline type as defined.
UI Automation Events
The IAccessibleEx interface helps existing applica-
tions or UI libraries extend their Microsoft Active
Accessibility object model to support UI Automa-
tion without rewriting everything from scratch. With
IAccessibleEx, you can implement only the differ-
ences between Microsoft Active Accessibility and UI
Automation object models.
Because the MSAA-to-UIA Proxy translates the object
models of IAccessibleEx-enabled Microsoft Active
Accessibility servers as UI Automation object mod-
els, UI Automation clients don’t have to do any extra
work. The IAccessibleEx interface can enable classic
in-process Microsoft Active Accessibility clients to in-
teract directly with UI Automation providers, too.

UI Automation events notify applications of changes

to and actions taken with automation elements. The
four different types of UI Automation events, as list-
ed in Table 4, do not necessarily mean that the visual
state of the UI has changed. The UI Automation event
model is independent of the WinEvent framework in
Windows, although the Windows Automation API
can make UI Automation events interoperable with
the Microsoft Active Accessibility framework.
Which to Support:
Microsoft Active Accessibility,
UI Automation, or IAccessibleEx?
While IAccessibleEx can be a cost effective way of
supporting UI Automation, a couple of technical con-
siderations should be made prior to the decision.

20 Windows Automation API 3.0 Overview www.code-magazine.com

 Event Description
Property change Raised when a property on an UI Automation element or control pattern
changes. For example, if a client needs to monitor an application's check
box control, it can register to listen for a property change event on the
ToggleState property. When the check box control is checked or unchecked,
the provider raises the event and the client can act as necessary.
Element action Raised when a change in the UI results from end user or programmatic
activity. For example, clients can listen for the Invoked event when a user
clicks a button when the InvokePattern is invoked programmatically.
Structure change Raised when the structure of the UI Automation tree changes. The structure
changes when new UI items become visible, hidden, or removed on the
desktop.
General event Raised when actions of global interest to the client occur, such as when the
focus shifts from one element to another or when a window closes.
Table 4: Types of UI Automation Events.

If you are developing a new application or control, Clients’ Golden Question:

I recommend UI Automation as it provides the best
flexibility. Microsoft Active Accessibility may look
simpler in the short term, but there are a lot of seri-
ous limitations to overcome, like its Windows 95-in-
spired object model and the inability to support new
UI behaviors or merge roles.
These shortcomings surface quickly when you try to
introduce new controls.
The UI Automation object model is based on canoni-
cal functionalities of UI features. Control developers
choose from abstracted features (control patterns)
that best suit their controls’ behaviors. This is supple-
mented with control types that offer role information
to users.
Microsoft Active Accessibility,
UI Automation, or Something Else?
Microsoft Active Accessibility is a
“chatty” architecture and is slow
for clients that run out of process. The UI Automation object model
To mitigate this, many acces- is based on canonical
sibility tool programs choose to
hook into and run in the target
functionalities of UI features.
application process. While assis- Control developers choose
tive technologies widely employ
this in-process code practice, the
from abstracted features
risk and complexity are extremely (control patterns) that best suit
high. Thus, you need an out-of-
process solution with better per-
their controls’ behaviors.
formance and reliability.

When to Use IAccessibleEx UI Automation offers much better performance for

Consider the following requirements before you use
the IAccessibleEx interface for your application.
Rule #1: The baseline Microsoft Active Accessi-
bility Server’s accessible object hierarchy must be
clean.
IAccessibleEx cannot fix problems with existing ac-
cessible object hierarchies.
Rule #2: Your IAccessibleEx implementation must
be compliant with both Microsoft Active Accessibil-
ity and UI Automation specifications.
Tools are available to validate compliance with both
specifications.
If either of these baseline requirements is not met,
you should consider implementing UI Automation
natively. You can keep legacy Microsoft Active Ac-
cessibility server implementations for backward com-
patibility if it is necessary. From a UI Automation
client’s perspective, there is no difference between
UI Automation providers and Microsoft Active Ac-
cessibility servers that implement IAccessibleEx cor-
rectly.
out-of-process client practices (400% or faster for
some scenarios than Microsoft Active Accessibility
running out-of-process), while adding richness and
flexibility to support the latest user interface designs.
Conclusions and Resources
Windows Automation API 3.0 in Windows 7 fea-
tures the best of Microsoft Active Accessibility and
the UI Automation specification, while providing a
cleaner migration path from one to the other. The
object model is easier to use and more flexible, the
automation elements reflect the evolution of modern
user interfaces, and developers can define custom UI
Automation control patterns, properties, and events.
UI Automation Specifications are offered as cross-
platform accessibility API under the Microsoft Open
Specification Promise. For more information about
the specification and about how Microsoft imple-
ments it, visit the MSDN Accessibility Developer
Center (http://msdn.microsoft.com/accessibility/).
Masahiko Kaneko

www.code-magazine.com Windows Automation API 3.0 Overview 21

ONLINE QUICK ID 0811052

What’s New in Windows 7

Nazia Zaman
naziaz@microsoft.com,
Program Manager, User
Interface Platform
Nazia is a PM on the User
Interface Platform Team in the
Windows Experience Division,
and is very passionate about
Accessibility.
She graduated from Stanford
University with a Masters in
Computer Science. She is
interested in Human Computer
Interaction, usability, and
kernel programming. Check out
her blog at: http://blogs.msdn.
com/dreamingreality/
For Windows 7, UI Automation has a new COM
client API for use from unmanaged and managed
clients alike. Unmanaged applications can now
use UI Automation without changing languages or
loading the CLR, while simultaneously benefiting
Unmanaged applications
can now use UI Automation
without changing languages
or loading the CLR,
while simultaneously benefiting
from all the latest features.
Automation API
Windows® 7 offers end–to-end accessibility with better
performance, seamless interoperability, and improved framework
design. The Windows Automation API 3.0 combines the advantages of
existing accessibility implementations in Microsoft® Active Accessibility®
and a modern API design in UI Automation to provide easier migration
from and integration with precursor technologies and industry standards.
The major improvements in this accessibility frame- Once you have the Automation object, you can
work are: discover the entire user interface (UI). The UI is
modeled as a tree of automation
• End-to-end unmanaged elements (IUIAutomationEle-
solution, including a native Fast Facts ment objects), each element
client API, faster and more representing single pieces of
robust UI Automation im-
The Windows 7 UI: a button, a window, the
plementations, extensible Automation API is now desktop, and so on. The IUI-
provider implementations, available unmanaged AutomationElement interface
and Win32 proxy for from the client has methods relevant to all con-
performance and avail- to provider sides, boasting trols, such as checking proper-
ability. a 400%+ improvement in ties or setting focus. Here, you
• Harmonization with in- performance. get the element with focus and
dustry standards such as its name:
W3C ARIA and Section
508 specifications. IUIAutomationElement *pElement;
• New properties and control patterns. pAutomation->GetFocusedElement(&pElement);
• Custom properties, events, and patterns. BSTR name;
pElement->get_CurrentName(&name);
std::wcout << L"Focused Name:" << name << L"\n";
An Unmanaged Client API
Furthermore, elements can expose functionality
specific to a particular control through control pat-
terns. Control patterns are collections of associated
properties, events, and methods, and more than one
can apply to an element. Here, I use the Invoke pat-
tern to press a button:
from all the latest features. This
new API is similar to the previ- IUIAutomationInvokePattern * pInvoke;
ous managed API, but is friend- pElement->GetCurrentPatternAs(UIA_InvokePatternId,
lier to C++ developers. __uuidof(IUIAutomationInvokePattern),
(void **)&pInvoke);
The heart of the new COM API pInvoke->Invoke();
is the IUIAutomation interface,
which enables clients to get au- To navigate the tree of automation elements, you
tomation elements, register event can use tree walkers. Here, I create an IUIAutoma-
handlers, create objects, and ac- tionTreeWalker object to navigate to the first child
cess other helper methods. To of an element in the Control View of the tree:
begin using UI Automation,
simply include the Automation IUIAutomationTreeWalker * pWalk;
header (UIAutomation.h), and pAutomation->get_ControlViewWalker(&pWalk);
then CoCreate the Automation object: IUIAutomationElement * pFirst;
pWalk->GetFirstChildElement(pElement, &pFirst);
IUIAutomation * pAutomation;
CoCreateInstance(__uuidof(CUIAutomation), NULL, To address performance issues when communi-
CLSCTX_INPROC_SERVER, cating across processes, clients can fetch multiple
__uuidof(IUIAutomation), properties and patterns at a time. Here, I create a
(void **)&pAutomation); cache request (IUIAutomationCacheRequest ob-

22 What’s New in Windows 7 Automation API www.code-magazine.com

ject) from the Automation object, identify the prop-
erty (Automation ID property) to prefetch, cache it
for an element, and then get the cached ID:
IUIAutomationCacheRequest * pCR;
pAutomation->CreateCacheRequest(&pCR));
pCR->AddProperty(UIA_AutomationIdPropertyId);
IUIAutomationElement * pCachedElement;
pElement->BuildUpdatedCache(&pCachedElement);
BSTR autoID;
pCachedElement->get_CachedAutomationId(&autoID);
std::wcout << L"Cached ID:" << autoID << L"\n";
With UI Automation property conditions, looking
for specific values for a property is easy; you can
combine conditions with AND, OR, and NOT op-
erators to search for UI scenarios. Here, I search for
a check box:
IUIAutomationCondition * pCheckBoxProp;
VARIANT varCheckBox;
varCheckBox.vt = VT_I4;
varCheckBox.lVal = UIA_CheckBoxControlTypeId;
pAutomation->CreatePropertyCondition(
UIA_ControlTypePropertyId,
varCheckBox, &pCheckBoxProp);
IUIAutomationElement * pFound;
pElement->FindFirst(TreeScope_Descendants,
pCheckBoxProp, &pFound);
In addition to these features, UI Automation pro-
vides custom proxy registration, events, and the
Text Pattern for manipulating documents.
Proxy Factory
Clients can customize how UI Automation sees
HWND-based controls by registering customized
providers with a proxy factory implementation.
Custom proxies affect only the client’s own view.
The provider answers CreateProvider calls with
the HWND to be proxied, as well as the idOb-
ject and idChild associated with the WinEvent, if
needed.
Here, I verify the expected values of OBJID_CLIENT
and CHILDID_SELF and create a custom proxy:
IFACEMETHODIMP CustomFactory::CreateProvider(
UIA_HWND hwnd, LONG idObject, LONG idChild,
IRawElementProviderSimple **ppRetVal)
{ // Register the event with default cache request.
*ppRetVal = NULL;
if(idObject == OBJID_CLIENT &&
idChild == CHILDID_SELF)
{
// Create the custom proxy.
*ppRetVal = new CustomProxy((HWND)hwnd);
}
return S_OK;
} lates between Microsoft Active Accessibility and
www.code-magazine.com
Proxy factory registration consists of creating proxy
factory entries and inserting them into a proxy fac-
tory mapping. The mappings are processed in order
as clients register proxies. In this example, I create
the proxy for all HWNDs with a class name con-
taining the word “MYCUSTOMBUTTON”:
// Instantiate the proxy factory.
IUIAutomationProxyFactory * pCF =
new CustomFactory();
// Create an entry with the factory.
IUIAutomationProxyFactoryEntry * pCEnt;
pAutomation->CreateProxyFactoryEntry(pCF, &pCEnt);
pCEnt ->put_ClassName(L"MYCUSTOMBUTTON");
pCEnt ->put_AllowSubstringMatch(TRUE);
Matthew Karr
// Get the mapping.
IUIAutomationProxyFactoryMapping * pMapping; Matthew is a Software
pAutomation->get_ProxyFactoryMapping(&pMapping); Development Engineer on the
User Interface Platform Team,
// Insert the entry at the start of the mapping. Accessibility, Application,
pMapping->InsertEntry(0, pCEnt); and Automation group in the
Windows Experience Division.
Optimizations in the Event Model Matthew Karr has been a
developer at Microsoft for 5
UI Automation clients, like screen readers, track years, working on Automation
events raised in the UI by UI Automation providers and Accessibility. He’s helped
and notify users of the events. To improve efficiency ship UI Automation V1 and the
in Windows 7, providers can raise an event selec- Magnification API with Vista,
tively, notifying only those clients that subscribe to and is currently working on the
that event. COM UI Automation API for
Windows 7.
Common UI Automation events that providers
should support include the following: In his free time he enjoys
juggling, snowboarding,
• Changes in a property or control pattern for a science fiction, and video
UI Automation element. games.
• End user or programmatic activity that affects
the UI.
• Changes to the structure of the UI Automa-
tion tree.
• Shift in focus from one element to another.
To listen to an event, create a COM object imple-
menting the event handler interface, and then call
the corresponding method on the IUIAutomation
object to subcribe the handler to the event. Here, I
subscribe a handler to the Focus Changed event:
class FocusChangedHandler:
public IUIAutomationFocusChangedEventHandler;
FocusChangedHandler * focusHandler =
new FocusChangedHandler();
pAutomation->AddFocusChangedEventHandler(
NULL, focusHandler);
Interoperability with Microsoft Active
Accessibility-based Controls
To improve interoperability, UI Automation trans-
What’s New in Windows 7 Automation API 23

Figure 1: COM diagram shows the role of IAccessibleEx in
extending legacy implementations.
UI Automation implementations. UI Automation
clients can use the new UI Automation services to
interact with earlier Microsoft Active Accessibil-
ity implementations, and Microsoft Active Acces-
sibility clients can interact with UI Automation
providers.
In Windows 7, Microsoft Active Accessibility imple-
mentations can add UI Automation properties and
control patterns by supporting the
IRawElementProviderSimple
interface to expose patterns and
Clients using the properties and the IAccessibleEx
UI Automation client API interface to handle ChildIds.
Figure 1 shows the relationship
can use its services to between the IAccessible, IAcces-
search existing sibleEx, IRawElementProvider-
Simple, and IRangeValuePro-
IAccessible implementations. vider interfaces.
Code that writes to the UI
automation provider interfaces With the IAccessibleEx interface,
developers can extend Microsoft
will still be visible to existing Active Accessibility implementa-
IAccessible clients. tions by adding required UI Auto-
mation object model information.
The new MSAA-to-UIA proxy
featured in the Windows Auto-
mation API provides a variation of a UI Automation
“provider” to Microsoft Active Accessibility imple-
mentations. UI Automation clients can interact with
all variations of UI Automation implementations:
native UI Automation, IAccessible (Microsoft Active
Accessibility), and IAccessible + IAccessibleEx.
New Win32 Support
via OLEACC Plus IAccessibleEx
In Windows 7, OLEACC proxies expose information
about common controls that Microsoft Active Ac-
cessibility cannot express. The MSAA-to-UIA proxy
recognizes IAccessibleEx and forwards this addi-
tional information to UI Automation. For example,
the OLEACC slider proxy adds the IRangeValue
pattern, exposing minimum and maximum values
that Microsoft Active Accessibility cannot expose.
Extending the OLEACC proxies with IAccessibleEx
has the dual benefit of leveraging existing code and
keeping the OLEACC proxies up to date.
24 What’s New in Windows 7 Automation API
Some frameworks such as HTML, Windows Pre-
sentation Foundation (WPF), and Silverlight™ have
a metadata system that can associate accessibility-
related properties with an element. Consequently,
developers can easily fix common bugs such as
an incorrect accessibility name. However, Win32
does not have a similar feature, and the very basic
property system for HWNDs does not apply to sub-
items.
However, with Direct Annotation, developers can
mark up properties on Win32 controls with accessi-
bility property/value information, enabling develop-
ers to fix bugs without needing a full UI Automation
or Microsoft Active Accessibility implementation.
Here, I set the AutomationId on a simple control
with the hwnd:
IAccPropServices *pAccPropServices;
CoCreateInstance(CLSID_AccPropServices, NULL,
CLSCTX_SERVER, IID_IAccPropServices,
(void**)&pAccPropServices);
pAccPropServices->SetHwndPropStr(hwnd,
OBJID_CLIENT, CHILDID_SELF,
AutomationId_Property_GUID, L"Foo ID");
Custom Control Patterns, Properties,
and Events
With Windows 7, you can extend the platform with
custom control patterns, properties, and events. Be-
cause of this support, developers of UI Automation
clients and providers can introduce new accessibil-
ity specifications independent of future operating
system releases.
Developers register and use custom patterns,
properties, and events on both the client side and
the provider side. If a provider registers for a prop-
erty that the client hasn’t, the client can’t retrieve
it, and if a client registers for a property that the
provider hasn’t, the provider can’t respond to a
request for it. Neither of these cases causes errors;
one side merely remains unaware of the other’s
capabilities.
Once the new property, pattern, or event is regis-
tered, using it is just like using a built-in pattern,
property, or event.
Custom Properties
Registration of properties is identical for both cli-
ent and provider. In the following sample, I have a
native UI Automation control and I want to add a
custom string property, LongName:
// The PropertyId for the LongName property.
PROPERTYID longNamePropertyId;
// This is the predefined property GUID,
// the name of the property, and the type.
UIAutomationPropertyInfo longNamePropertyInfo = {
GUID_LongNameProp,
www.code-magazine.com
 L"LongName",
UIAutomationType_String };
// This yields the property ID for new property.
pAutomationRegistrar->RegisterProperty(
&longNamePropertyInfo, &longNamePropertyId);
Retrieving the property is similar to retrieving a
normal property; you use the property ID that the
RegisterProperty method initialized:
VARIANT longNameValue;
pElement->GetCurrentPropertyValue(
longNamePropertyId, &longNameValue);
std::wcout << longNameValue.bstrVal << L"\n";
Again, on the provider side, you call the same regis-
tration method except you add an entry in the con-
trol’s implementation of IRawElementProviderSim
ple::GetPropertyValue.
In this code sample, the UI Automation implemen-
tation keeps a reference to the control in _pOur-
Control, and the control supports a GetLongName
method:
GetPropertyValue(PROPERTYID propertyId,
VARIANT * pRet)
{ Automation framework forwards the property and
pRet->vt = VT_EMPTY;
if(propertyId == longNamePropertyId)
{ Microsoft Word and other
// Get the long name from the control.
pRet->bstrVal = _pControl.GetLongName();
pRet->vt = VT_BSTR;
} tern interfaces. For more details please refer to the
// Deal with other properties...
return S_OK
} New Properties and Control Patterns
Custom Events
Events follow a nearly identical model to properties:
you register the event ID, and then you can use the
custom EventID exactly as you use a normal Even-
tID. However, custom events cannot have argu-
ments; they are merely notifications that the event
occurred.
Here, I register the event. Both listening and raising
would then be identical to other events:
// The EventId for the ThingHappened event
EVENTID thingHappenedEventId;
// Event Information
// This is the event GUID, the name of the event.
UIAutomationEventInfo thingEventInfo = {
GUID_ThingHappenedEvent, L"ThingHappened" };
// This gives you the Event ID for the new event.
pAutomationRegistrar->RegisterEvent(
&thingEventInfo, &thingHappenedEventId);
www.code-magazine.com
Custom Control Patterns
Creating a custom control pattern requires the fol-
lowing:
• Arrays of events, proper-
ties, and methods associated
with the pattern. However, with Direct Annotation,
• IIDs of the pattern’s corre-
sponding provider interface
developers can mark up
and client interface. properties on Win32
• Code to create a client inter-
face object.
controls with accessibility
• Code to perform marshal- property/value information...
ling for the pattern’s proper-
ties and methods.
On the client side, the code that registers a pattern UI Automation as an
must supply a factory for creating instances of a Cli- Automated Test Tool
ent Wrapper that forwards property requests and
method calls to an IUIAutomationPatternInstance UI Automation is specifically
provided by UI Automation. The UI Automation designed to support automated
framework then takes care of remoting and marshal- testing, as well as accessibility.
ling the call. UI Automation separates two
models of UI interaction: one
On the provider side, the code that registers a pattern exposes information about
must supply a “pattern handler” object that performs the UI and the other collects
the reverse function of the Client Wrapper. The UI information needed by test
tools.
method requests to the pattern handler object, which
in turn calls the appropriate method on the target UI Automation providers
object’s provider interface. are applications, such as
The UI Automation framework takes care of all third-party applications,
communication between the client and provider, or controls based on the
both of which register corresponding control pat- Microsoft Windows operating
system. UI Automation clients
Windows 7 SDK. include automated test scripts
and assistive technology
applications. Both a provider
and client are required for UI
Automation to be useful as an
Some other useful properties for automation ele- automated test tool.
ments have been added to UI Automation for Win-
dows 7:
ControllerFor is an array of elements manipulated
by the automation element. Without this property it
is hard to determine the impact of an element’s op-
eration.
DescribedBy is an array of ele-
ments that provide more infor-
mation about the automation To improve interoperability,
element. Instead of using the
object model to discover infor-
UI Automation translates
mation about the element, cli- between Microsoft
ents can quickly access that in-
formation in the DescribedBy
Active Accessibility and
property. UI Automation implementations.
FlowsTo is an array of elements
that suggest the reading order af-
ter the current automation element. FlowsTo is used
when automation elements are not exposed or struc-
tured in the reading order users perceive.
What’s New in Windows 7 Automation API 25
 IsDataValidForForm identifies whether data is valid
in a form.
ProviderDescription identifies source information
for the automation element’s UI Automation pro-
vider, including proxy information.
Control Patterns for Virtualized Child Objects
When a control has too many children to load at
once, a common solution is to treat the excess chil-
dren as virtualized controls. This creates problems
for the UI Automation tree map
because there are only a handful
of real controls and the virtual-
Windows 7 offers ized controls simply don’t exist
in the UI Automation tree.
further extensibility through
the registration of To manage this, UI Automation
offers two control patterns. The
custom control patterns, ItemContainer pattern lets a user
properties, and events. search a container of virtualized
controls for specific properties.
This gives the client a reference
to a virtualized control, but the
user can’t do anything with it. The VirtualizedItem
pattern enables the client to force the item to exist,
either by realizing it internally, or by having it scroll
on screen.
In this client-side code example, I search for a spe-
cifically named item in a virtualized list:
// Get the ItemContainer pattern.
IUIAutomationItemContainerPattern * pContainer;
pElement->GetCurrentPatternAs(
UIA_ItemContainerPatternId,
__uuidof(IUIAutomationItemContainerPattern),
(void**)&pContainer));
// Search the container for the property.
VARIANT varNameStr;
varNameStr.vt = VT_BSTR;
varNameStr.bstrVal = SysAllocString(name);
IUIAutomationElement * pFound;
pContainer->FindItemByProperty(NULL,
UIA_NamePropertyId, varNameStr, &pFound);
// Realize the virtual element.
IUIAutomationVirtualizedItemPattern * pVirt;
pFoundElement->GetCurrentPatternAs(
UIA_VirtualizedItemPatternId,
__uuidof(IUIAutomationVirtualizedItemPattern),
(void**)&pVirt));
pVirtualizedItem->Realize();
New UI Automation Properties for
Accessible Rich Internet Applications
(ARIA)
Every day, Web sites are increasing their utility with
dynamic content and advanced UI controls by us-
ing technologies like Asynchronous JavaScript and
XML (AJAX), HTML, and JavaScript. However, as-
sistive technologies are frequently unable to inter-
act with these complex controls or expose dynamic
content to users. Accessible Rich Internet Applica-
tions (ARIA) is a W3C technical specification for
developing Web content and applications so that
they are accessible to people with disabilities.

W3C ARIA Role MSAA Role UIA Control Type UIA AriaRole Property
button ROLE_SYSTEM_PUSHBUTTON button button
checkbox ROLE_SYSTEM_CHECKBUTTON Checkbox checkbox
combobox ROLE_SYSTEM_COMBOBOX Combobox combobox
grid ROLE_SYSTEM_TABLE DataGrid grid
gridcell ROLE_SYSTEM_CELL DataItem gridcell
group ROLE_SYSTEM_GROUPING Grouping group
img ROLE_SYSTEM_GRAPHIC Image img
link ROLE_SYSTEM_LINK HyperLink link
list ROLE_SYSTEM_LIST List list
menu ROLE_SYSTEM_MENUPOPUP Menu menu
presentation ROLE_SYSTEM_PANE Pane presentation
progressbar ROLE_SYSTEM_PROGRESSBAR ProgressBar progressbar
radio ROLE_SYSTEM_RADIOBUTTON RadioButton radio
slider ROLE_SYSTEM_SLIDER Slider slider
tooltip ROLE_SYSTEM_TOOLTIP Tooltip tooltip
tree ROLE_SYSTEM_OUTLINE Tree tree
treegrid ROLE_SYSTEM_TABLE DataGrid treegrid
Table 1: W3C ARIA Roles can be mapped to Microsoft Active Accessibility roles and UI Automation control types and AriaRole properties.

26 What’s New in Windows 7 Automation API www.code-magazine.com

 W3C ARIA States Microsoft Active Accessibility UI Automation Control UI Automation Aria-
and Properties Patterns and Properties Properties Property
checked STATE_SYSTEM_CHECKED Toggle Pattern, checked checked
controls n/a ControllerFor n/a
describedby n/a DescribedBy n/a
disabled STATE_SYSTEM_UNAVAILABLE IsEnabled False disabled
flowto n/a FlowsTo n/a
invalid n/a IsDataInvalidForForm invalid
labelledby n/a LabeledBy n/a
live n/a n/a live
multiselectable STATE_SYSTEM_EXTSELECTABLE CanSelectMultiple multiselectable
readonly STATE_SYSTEM_READONLY IsReadOnly readonly
required STATE_REQUIRED IsRequiredForForm required
secret STATE_SYSTEM_PROTECTED IsPassword secret
valuemax n/a Maximum Property in valuemax
RangeValue Pattern
valuemin n/a Minimum Property in valuemin
RangeValue Pattern
valuenow IAccessible::get_accValue Value Property in RangeValue valuenow
Pattern
Table 2: W3C ARIA States and Properties can be mapped to Microsoft Active Accessibilty properties and UI Automation control patterns and AriaProperties properties.

To support the ARIA specification, the UI Automa-
tion specification enables developers to associate
UI Automation AriaRole and AriaProperties at-
tributes with W3C ARIA Roles, States, or Proper-
ties. This helps user applications such as Internet
Explorer support the ARIA object model in the
context of UI Automation while keeping a baseline
accessibility object model.
Some parts of the ARIA specification can be
mapped to the desktop-oriented Microsoft Active
Accessibility object model; however, much of the
specification can only be applied to rich Internet
applications. Table 1 lists some examples of map-
pings from W3C ARIA Roles to Microsoft Active
Accessibility Roles and UI Automation Control
Types.
For example, the ARIA Role checkbox is sup-
ported in Microsoft Active Accessibility by the
role ROLE_SYSTEM_CHECKBUTTON and
in UI Automation by the combination of con-
trol type Checkbox and AriaRole checkbox. The
ARIA state checked is supported in Microsoft Ac-
tive Accessibility by the state STATE_SYSTEM_
CHECKED and in UI Automation by the control
pattern Toggle Pattern and the AriaProperties
property checked.
ARIA States and Properties are supported by the
UI Automation AriaProperties property with the
following exceptions: ARIA properties that take
object references (like the describedby property),
and ARIA properties already supported by the ac-
cessibility object model. Table 2 lists examples of
mappings from W3C ARIA States and Properties
to various properties and functions of Microsoft
Active Accessibility and UI Automation.
UI Automation also offers a simple text object
model with the TextPattern pattern, which supports
embedding objects in a document object. This en-
ables user agents and client applications to treat
Web content either as an HTML document or as a
traditional desktop UI depending on the end-user
scenarios.
With these features, UI Automation enables both
the support of and extension of the W3C ARIA
specification without a dependency on a specific
application or browser.
Conclusion
With application user interfaces growing more and
more complex, getting accessibility right is a challenge
for developers. Programmatic access to the UI is criti-
cal in the development of assistive technologies like
screen readers and magnifiers. To address this, the
Windows 7 Automation API aims to provide a com-
plete end-to-end, flexible, extensible, and consistent
framework with improved design and performance.
For further details, please refer to the Windows 7
SDK or the MSDN Accessibility Developer Center
(http://msdn.microsoft.com/accessibility/).
Nazia Zaman
Matthew Karr

www.code-magazine.com What’s New in Windows 7 Automation API 27

ONLINE QUICK ID 0811062
Mark Rideout
markri@microsoft.com
(425)-703-1688
Mark Rideout is a Program
Manager on the Silverlight
Team at Microsoft. Mark is
responsible for features around
text and text input in addition to
input and accessibility features.
Mark has worked on multiple
Microsoft UI frameworks
such as Windows Forms and
has been working with the
.NET Framework since 1.0.
Mark started at Microsoft in
1997, and prior to that, Mark
worked as a consultant where
he designed and developed
business applications.
Mark enjoys everything Lego,
pinball, and Disneyland. He
enjoys teaching his 2-year-old
daughter about everything he
likes.
28 Creating Accessibility-aware Silverlight 2 Content
Creating Accessibility-aware
Silverlight 2 Content
If you haven't heard, accessibility is one of the most important
aspects of a Web site experience. By using the accessibility features in
Silverlight™ 2, you can provide the best experience for all users. Building a
rich Internet experience can be a daunting task when you have to balance a
cool visual design with usability.
sability goes beyond the initial determination elements into two types: simple visuals that usually
U that a user interface (UI) "makes sense" as the
UI has to be accessible to everyone. Silverlight
2 enables building a cool and accessible experience,
derive directly from the FrameworkElement class,
or functional controls that derive from the Control
class. A TextBlock, Image, or MediaElement are ex-
whether you are building an entire Silverlight appli- amples of simple visuals while a Button, TextBox, or
cation or just a few Silverlight controls. This article ListBox are examples of functional controls.
provides you with guidelines for taking advantage
of these features and building an accessible Silver- Along with the visual tree of elements, your Silver-
light application. light 2 application has a corresponding accessibil-
ity tree. UI Automation presents three accessibility
If you attempted to add accessibility to your Sil- views of the visual tree: a control view, a content
verlight 1 content, you probably found that there view, and a raw view. Generally, all elements that
wasn't much support except for derive from Control show up
something analogous to "alt" in the control view while ele-
text for screen readers. Silver- Fast Facts ments that derive directly from
light 2 on the other hand pro- FrameworkElement show up
Accessibility is an
vides necessary features that in the content view. TextBlock
you can build on. ever-increasing requirement can appear in either the con-
as you develop applications. tent view when used inside a
Silverlight 2 provides
In general, when you build your control or data template or the
Web application with accessibil- the tools you need to make control otherwise. If you cre-
ity in mind, there are a general your application accessible ate custom Silverlight controls,
set of areas to think about: but you need to understand you can identify whether it’s a
control, content, or both (or
how best to do it.
• Screen reader support. none).
• Keyboard navigation.
• High contrast presentation. There are different ways to think of elements be-
ing visible or invisible. Silverlight 2 reports elements
whose Visibility property is equal to Visibility.Col-
Ensuring Your Silverlight 2 Content lapsed as hidden, but reports elements that are in
Supports Screen Readers the tree and "hidden" from view as being visible el-
ements. This distinction is important because you
Silverlight 2 provides information to screen readers might style your app to have different UI "modes"
about your application via the new UI Automation by just z- ordering the elements. If you have rea-
framework. At the application level, you work with sons for not setting the Visibility property, then you
UI Automation using the AutomationProperties should at least disable controls that are "hidden"
class and its corresponding properties. When creat- from view to ensure that screen readers don't at-
ing custom controls or extending controls, you work tempt to interact with them.
with AutomationPeer and the UI Automation pro-
vider interfaces. This article looks at some impor-
tant UI Automation concepts you should be aware XAML Tag Your Elements
of as you build your Silverlight 2 application.
As you develop your Silverlight application, you
sometimes need to provide additional accessibility
Silverlight Accessibility Tree information to ensure your content is readable. For
example, screen readers won't know anything about
Your Silverlight 2 application is comprised of many your Image element except that it is an image. As
visual elements from Borders to Control-based ele- with HTML, you need to provide "alternative" text-
ments. For the most part, you can split the visual based information. With Silverlight, you provide al-
www.code-magazine.com
ternative accessibility information by using the Auto- tems, Microsoft® Active Accessibility®-based screen
mationProperties static class. readers can read Silverlight 2 content via the UI
Automation UIA-to-MSAA Bridge. Because the ac-
There are cases when you won't have to add addi- cessibility technologies are somewhat different, not
tional markup via AutomationProperties. For ex- all data transfers into Microsoft Active Accessibility
ample, controls that have text in them (like Buttons) properties and some concepts are
already provide default information. In other cases, simplified.
you do need to provide additional data.
For more information on the Along with the visual tree
Using AutomationProperties is very easy both in UIA-to-MSAA bridge, see the
your XAML markup and in code. Using the Image Windows Automation API 3.0 of elements,
element as an example, you set the AutomationProp- Overview article by Masahiko your Silverlight 2 application
erties.Name attribute with XAML markup like this: Kaneko in this issue. In addition,
you can search MSDN for Active has a corresponding
<Image x:Name="image" Source="/dogandcats.png" Accessibility Bridge to UI Auto- accessibility tree.
AutomationProperties.Name="Image of a dog and mation for specific details on the
cat on a couch"/> limitations.

In code, you use the appropriate attached property

set/get method:
AutomationProperties.SetLabeledBy(this.nameInput, this.nameLabel);
For best practices on providing good alternative
text, see the MSDN technical article "Creating Text
Equivalents for Images" (http://msdn.microsoft.
com/en-us/library/ms971334.aspx).
Sometimes you might have a TextBox, Image, or other
control labeled with text. For example, a TextBox for
the user's name might have text to the left specifying
"Name:". You can use the TextBlock element to pro-
vide that static text label, and the AutomationProper-
ties.LabeledBy property to tell that the TextBlock is
labeling the TextBox. Because Silverlight 2 does not
support element Binding syntax (using the Element-
Name syntax) to set the LabeledBy property in XAML,
you have to resort to some code, like this XAML:
<StackPanel Orientation="Horizontal">
<TextBlock Text="Name:" x:Name="nameLabel"/>
<TextBox Width="100" x:Name="nameInput"/>
</StackPanel>
Then you set the LabeledBy property like so:
AutomationProperties.SetLabeledBy(this.nameInput, this.nameLabel);
Ensure Keyboard Interactivity AutomationPeer
is Based upon
Creating an accessible Silverlight application means
creating a keyboard-accessible application. Try us-
the WPF Design
ing your application with only a keyboard to iden- The design for the
tify where functionality is unavailable to keyboard- AutomationPeer class and
only users. Sometimes you'll find that you require derived classes comes from
mouse actions to use certain features. Other times the Windows Presentation
you'll find that you cannot tab to an element. Foundation set of classes.
This provides you with an
By thinking about keyboard accessibility during ap- opportunity to write your
plication design and development, you will avoid AutomationPeer accessibility
these types of issues. For more information on key- code once and use it both in
board best practices, see Guidelines for Keyboard WPF and Silverlight.
User Interface Design in MSDN (http://msdn.mi-
crosoft.com/en-us/library/ms971323.aspx). Sharing designs also helps
you as you learn about adding
accessibility since there are
IsTabStop and Tab Order already materials out about
WPF and accessibility.
With the new keyboard and focus model in Silverlight
2, you can create a keyboard-friendly experience. You
specify what controls can receive focus via the IsTab-
Stop property. Most Silverlight controls, such as But-
ton, already set the IsTabStop property to true. After
you design the UI, take the time to set the TabIndex
property to control the tab order. One thing to note is
that only Control-based elements can receive focus.

To properly support screen readers, you need to en-

sure that all your non-text controls that show up in
the accessibility tree provide at least the Automa-
tionProperties.Name value. I recommend that you
spend the time to provide other values such as Help-
Text. There are some great accessibility verification
tools recently released by Microsoft that work great
with UI Automation. Check it out by searching for
Microsoft Accessibility Labs on MSDN.
UI Automation and Microsoft Active Accessibility
Screen Readers—Bridging the Gap
As I've mentioned earlier, Silverlight 2 relies on the
operating system's UI Automation support. This does
not imply that only UI Automation-based screen
readers are supported. On supported operating sys-
Image and TextBlock
(Non-Control-based Elements)
If you don't think about keyboard accessibility, you
might add interactivity to an Image, Path, or Media-
Element with mouse input, but visual elements that
don’t derive from the Control class cannot receive
focus. When you enable mouse actions on these
types of elements, you may also want to enable the
functionality with a keyboard command or with a
control in the UI.
Alternatively, if you want to enable focus on an Im-
age, you can create a custom UserControl element
that contains the Image and add your keyboard logic
there. Do note that you will also need to provide a fo-
cus visual indication, as described later in this article.

www.code-magazine.com Creating Accessibility-aware Silverlight 2 Content 29

 Custom Keyboard Actions
The trick in providing custom keyboard actions
with a hotkey combination is to make sure your
combination doesn't conflict
with the browser keyboard ac-
With Silverlight, tions because the browser will
always win.
you provide alternative
accessibility information A good keyboard combina-
tion that I've used is combining
by using the AutomationProperties a number key with Shift and
static class and attached Control keys. For example, to
provide keyboard hotkeys for a
properties. particular view in my UI, I listen
(hook up event handlers) for the
Shift, Control, and number keys
associated with the view. I add a KeyDown event
handler to the root visual of my application since
keyboard events bubble when not handled:
if (e.Key == Key.D1 &&
((Keyboard.Modifiers &
(ModifierKeys.Control | ModifierKeys.Shift)) ==
(ModifierKeys.Control | ModifierKeys.Shift)))
{
// Perform hotkey action here
} zooming. When users zoom, the Silverlight browser
Regardless of what custom keyboard actions you add
you need to let users know that they exist. You can
do this by setting the AutomationProperties.Accel-
eratorKey property. In addition, you should add a
tooltip or UI that describes the hotkey combination.
With Great Power Comes
Great Responsibility
As you develop a great UI with complex gradients
and cool graphics, you need to ensure you have a
good experience with high contrast settings. I won't
go so far as to say you shouldn’t make your UI cool;
just put in the extra effort to make an accessible ex-
perience. Here are a few things to think about:
• Design with accessibility in mind.
• Support different text sizes or browser zoom.
• Honor high contrast settings.
Design with Accessibility in Mind
Using XAML and styling your application is great,
but you can create some ugly UI that doesn't pres-
ent well to users who need higher contrast for read-
ability.
One important thing you can do during your de-
sign is to convey information with visual cues other
than color. For example, if you highlight particular
list items with a red font, you could also bold them,
so the user has a non-color cue to distinguish high-
lighted items. You can optionally add some sound
as the user moves over list items to note that the list
item is "active".
30 Creating Accessibility-aware Silverlight 2 Content
To help you understand some of the ways people
might use your application, think about what your
application would look like if you printed it on a
grayscale printer. Think about what it would be like
to use it with the keyboard only. Think about whether
your application is usable with the sound turned off.
Finally, think about whether you could easily explain
how to use your app over the phone. All of these will
help your design be more usable for more people in
more situations.
Support Changing Font Sizes
Silverlight 2 doesn't automatically change size
based upon the computer's DPI settings nor does
the content automatically change size based upon
the browser's zoom or text size settings. There are
three ways that you can scale your content to en-
sure your application can provide a large font ex-
perience: based upon the Web browser's zoom set-
ting, based upon a user-selectable UI, and based
upon a user-selectable font size UI. The last option
only focuses on scaling the text size vs. scaling the
whole UI.
Modern Web browsers have support for whole page
plug-in receives a resize event. Silverlight provides a
public Resized event that, with a little code in your
application’s Loaded event, you can use to dynami-
cally resize your Silverlight content:
App.Current.Host.Content.Resized += new EventHandler(Content_Resized);
In the XAML, just put a ScaleTransform that you
will use when the Silverlight plug-in resizes:
<UserControl.RenderTransform>
<TransformGroup>
<ScaleTransform x:Name="zoomTransform"
ScaleX="1" ScaleY="1"/>
</TransformGroup>
</UserControl.RenderTransform>
Lastly, apply the scale as a factor of your applica-
tion’s size:
void Content_Resized(object sender, EventArgs e)
{
System.Windows.Interop.Content content =
App.Current.Host.Content;
double scalePlayerX = (content.ActualWidth /
this.Width);
double scalePlayerY = (content.ActualHeight /
this.Height);
this.zoomTransform.ScaleX = scalePlayerX;
this.zoomTransform.ScaleY = scalePlayerY;
}
A second approach is based upon the same code that
you use for scaling based upon the Web browser's
zoom except you provide your own UI for scaling.
www.code-magazine.com
Migration Headache?

For immediate relief,

visit
www.VFPConversion.com
today!

VFP Conversion is a migration services brand of EPS

Software Corp., providing expert upgrading of VFP
applications to the latest technologies available from
Microsoft. To learn more about how VFP Conversion can
assist your enterprise, call toll free: 1 (866) 529-3682.
info@VFPConversion.com www.VFPConversion.com

Supporting high contrast is an important part to
making your application accessible. Silverlight 2
provides the SystemParameters.HighContrast value
to indicate when you should render in a high con-
trast mode. A high contrast mode can be as simple
as omitting non-important images, gradients, or pat-
Creating an accessible
Silverlight application means
creating a keyboard-accessible
application.
Best Accessible
Silverlight™
2 Experience
Comes from a UI
Automation-based
Screen Reader
It should make sense that you
will get the best screen reader
experience with Silverlight
when the screen reader
uses UI Automation. The
technology offers much more
information than predecessors.
UI Automation-based screen
readers can only read the
rich UI Automation concepts
and additional information
provided.
The last approach is to provide an option to increase
the font size. Silverlight 2 does not recognize the
browser's text size settings, so you need to provide the
user with a custom UI (as simple as a set of buttons
for selecting text size). Adjusting the font size can be a
great approach if you have a lot of text and the overall
UI doesn't need scaling. Since adjusting the font size
causes the text to take up more space, you need to use
layout elements such as Grid and StackPanel instead
of hard coding widths/heights. Silverlight elements
inherit the FontSize property throughout, so setting
it at the root of your application causes the value to
propagate.
Bring Contrast to Your UI
terns behind text to more com-
plex styling that makes your ap-
plication design a bit simpler. In
addition, a high contrast mode
should remove or reduce transi-
tion animations, including the
use of flashing.
You can take advantage of Silver-
light's styling system to provide
additional styles for use in high
contrast scenarios. Silverlight 2
does not ship with high contrast styles for built-in
controls, so you will need to create this as you style
your application.
Expression Blend provides support for creating cus-
tom styles for your controls. I recommend creating
two styles that you can switch between based upon
the SystemParameters.HighContrast value. There
is one catch: you can only apply a Silverlight con-
trol style once, and it has to be before or during the
control's constructor. The more holistic approach
is defining custom control styles during application
startup, but you can apply a custom style in your
control's constructor as well:
public class MyButton : Button
{ Listing 1.
public MyButton()
{ To simplify technology and focus on the purpose
if (SystemParameters.HighContrast)
{ creating provider-side UI Automation information.
this.Style = (Style)Application.
Current.Resources["btnContrast"];
}
}
} Change is Good, But Notify Others
}
For an example of a Silverlight control "theme"
framework, see Nikhil Kothari's blog for details
(http://www.nikhilk.net/Silverlight-Themes.aspx).
Custom Controls/UI
So far, I've addressed application-level accessibility,
but what’s good for the application developer is good
for the control developer. There are a few more con-
cepts that you need to understand first though.
Derive from Existing Controls
The Silverlight team has spent a lot of time making
their controls accessible; you should try to derive
from the closest Silverlight control. In addition,
you need to extend the control’s AutomationPeer,
which encapsulates accessibility functionality. If
you can't find an AutomationPeer that matches
your needs, then you should at least derive from
FrameworkElementAutomationPeer.
Implement AutomationPeer and
Associated Provider Interface
The AutomationPeer class exposes UI Automation
by rolling up most of the non-pattern related UI Au-
tomation properties exposed by the Automation-
ElementIdentifiers class.
Silverlight primarily chooses to focus on Auto-
mationPeer concepts where it can. The best way
to describe this is to say that the AutomationPeer
class provides the necessary default UI Automation
property values, and you identify a control’s acces-
sibility support via functions rather than a role.
UI Automation identifies control functionality by a
set of interfaces called control patterns. For exam-
ple, to identify that you can "invoke" a control, you
implement the IInvokeProvider interface in your
control's AutomationPeer. To add a control pattern,
you create a new AutomationPeer class and over-
ride its GetPattern method to return the interface
corresponding to the pattern requested. Now that
you have a custom AutomationPeer class, you need
to make your custom control return an instance of
it. Override the OnCreateAutomationPeer meth-
od on your control and return a new instance of
your custom AutomationPeer. Listing 2 is an ex-
ample of adding the IToggleProvider to a custom
AutomationPeer created for a light switch control.
The important parts of the custom control are in
of Silverlight, Silverlight supports only the APIs for
Silverlight does not provide the client-side UI Au-
tomation APIs.
Because accessibility properties change often, screen
readers rely on change notification to be efficient.
Change notification covers such things as knowing
when focus changes, knowing when a control be-

32 Creating Accessibility-aware Silverlight 2 Content www.code-magazine.com

comes disabled, and knowing when control pattern for identifying and notifying others about changes.
properties change. See Listing 1 and Listing 2 for an example of rais-
ing property changes.

Provide Focus Indication

… when you design, do it with
accessibility in mind.
Silverlight provides two methods to raise a change
notification: You use the AutomationPeer.
RaisePropertyChangedEvent method to signal
property changes, and use the AutomationPeer.
RaiseAutomationEvent method to signal other
types of automation changes.
Luckily, Silverlight already tells screen readers about
most changes for you. Silverlight provides notifica-
tions to UI Automation clients for the Automation-
Peer IsEnabled, IsOffscreen, Name, and ItemStatus
values. In addition, Silverlight provides structure
change notifications that tell you when items are
added and removed from the visual tree. By using
or deriving from a Silverlight control, you also take
advantage of control-specific notifications, such as
when a CheckBox is checked.
Visually, your control can do most anything. When
people talk about accessibility support, one im-
portant visual is focus. In Silverlight 2, the control
template and styling model is very rich, but what
doesn't exist is a common definition for visually
identifying focus.
If your control derives from a common Silverlight
control, you should use the FocusVisualElement
identification defined for the control and just re-
style it as necessary. Using Silverlight's Visual State
Manager (VSM), you can provide a custom control
template that defines a new focus element.
If your control is totally unique, you should create
your custom focus visual and add code in the Got-
Focus and LostFocus event handlers to show and
hide the focus indicator. You should also create a
control template and provide a named focus ele-
ment, similar to what Silverlight does.
High Contrast

If you create your own control with its own Auto- Lastly, instead of putting the onus on the applica-
mationPeer functionality, then you are responsible tion developers using your custom control, you

Listing 1: Example control raising UI Automation change notifications.

public class LightSwitch : Control public void Toggle()
{ {
// Not a complete Silverlight custom control but enough changeState(!this._stateOn);
// to demonstrate providing accessibility support via }
// AutomationPeer and UI Automation
protected override AutomationPeer OnCreateAutomationPeer()
private bool _stateOn; {
public LightSwitch(){} return new LightSwitchAutomationPeer(this);
}
public bool State
{ public void changeState(bool newState)
get {
{
return this._stateOn; // notify accessibility that the state changed
} LightSwitchAutomationPeer ap =
FrameworkElementAutomationPeer.FromElement(this) as
set LightSwitchAutomationPeer;
{
changeState(value); if (ap != null)
} {
} ap.RaiseStateChangedEvent(this._stateOn, newState);
}
protected override void
OnMouseLeftButtonDown(MouseButtonEventArgs e) // flip light switch state
{ this._stateOn = !this._stateOn;
e.Handled = true;
base.OnMouseLeftButtonDown(e); // update visuals
this.Toggle(); this.updateVisuals(this._stateOn);
} }
}

www.code-magazine.com Creating Accessibility-aware Silverlight 2 Content 33

 Listing 2: Custom AutomationPeer with Togglefor example control.
public class LightSwitchAutomationPeer : protected override AutomationControlType
FrameworkElementAutomationPeer, IToggleProvider GetAutomationControlTypeCore()
{ {
public LightSwitchAutomationPeer(LightSwitch light) // Button is the closest match
: base(light) return AutomationControlType.Button;
{ }
}
protected override string GetClassNameCore()
// Helper function for raising notification {
internal void RaiseStateChangedEvent(bool oldValue, return "LightSwitch";
bool newValue) }
{
if (oldValue != newValue && #region IToggleProvider Members
AutomationPeer.ListenerExists(
AutomationEvents.PropertyChanged)) public void Toggle()
{ {
base.RaisePropertyChangedEvent( if (!base.IsEnabled())
TogglePatternIdentifiers.ToggleStateProperty, {
oldValue == true ? ToggleState.On : ToggleState.Off, // Don't toggle if the control is not enabled
newValue == true ? ToggleState.On : ToggleState.Off); throw new ElementNotEnabledException();
} }
} ((LightSwitch)this.Owner).Toggle();
}
// Let UI Automation know that the pattern is supported
public override object GetPattern( public ToggleState ToggleState
PatternInterface patternInterface) {
{ get {
if (patternInterface == PatternInterface.Toggle) ((LightSwitch)this.Owner).State == true ?
{ ToggleState.On : ToggleState.Off;
return this; }
} }
return base.GetPattern(patternInterface);
} #endregion
}

should add the logic necessary to customize the erating system does not support UI Automation,
controls visual style based upon the high contrast and the Mozilla plug-in technology does not sup-
option mentioned earlier in this article. port interaction with the OS X accessibility frame-
work.

OS Limitations
For Silverlight 2, the operating system and, to some
extent, the browser dictate the level of accessibility
that Silverlight can provide.
Silverlight is a Web browser plug-in based upon ei-
ther Mozilla plug-in or Microsoft ActiveX technol-
ogy, depending upon the browser. The Web browser
is responsible for providing operating system inter-
action and notifications with the plug-ins contained
on a Web page. In some cases, a plug-in can work
directly with the operating system and bypass the
browser.
Conclusion:
Silverlight’s Accessible Future
With Silverlight 2, you can control high contrast
styles, keyboard navigation, and screen reader sup-
port. You can even take advantage of UI Automa-
tion wherever it’s implemented. Silverlight’s new
features can help you build Web applications and
controls that support accessibility, so all users can
gain enjoyment and value from your work. And re-
member, when you design, do it with accessibility
in mind.

Silverlight can provide accessibility information Mark Rideout

when used inside of browsers running on operat-
ing systems that support the UI Automation speci-
fication. Currently, Microsoft Windows XP and
later support UI Automation; Novell has made
a commitment to supporting the UI Automation
interfaces on Linux. Currently, Apple’s OS X op-

34 Creating Accessibility-aware Silverlight 2 Content www.code-magazine.com

ONLINE QUICK ID 0811072

Making Custom Controls

Accessible
While custom controls are introduced every day, not all of
them are easily accessible. This article provides a quick summary
of Microsoft® technologies that help make Win32-based custom controls
programmatically accessible. Techniques range from implementing UI
Automation, to creating or overriding properties with Dynamic Annotation,
LeAnne Fossmeyer to using the new IAccessibleEx interface to close the gap between UI
LeAnne is a Content Publishing Automation and Microsoft Active Accessibility®.
Lead in the Windows
Experience organization at
Microsoft. he Windows® operating system offers several Quick Tips: “Do I Really Need that
T technologies to help make controls accessible:
Microsoft Active Accessibility, UI frameworks,
and UI Automation. As Table 1 shows, the most
Control?”
Before investing time in designing a new control
effective technology depends on how developers in- or customizing a standard control, you should ask
troduce their new controls. yourself: “Do I really need that control?”

For controls and frameworks If you’re lucky, you may find a

written from scratch, UI Auto- Fast Facts standard control that does the
mation is recommended today. job. Many modern UI libraries
While Microsoft Active Acces- To meet the support accessibility and cus-
sibility is handy for relatively accessibility requirements, tomization.
simple controls, the technology a small change in standard
doesn’t support the complexity controls can cause If you customize a standard
of modern user interfaces (UI). more code changes than control, try not to overload
the change itself. it with features. Unnecessary
For controls based on common complexity can result in ex-
Never underestimate the
controls (such as COMCTL32. pensive accessibility support
DLL or USER32.DLL), Dynam- implication of a small cost, not to mention reduced
ic Annotation enables modest modification. usability. Never underesti-
customization. However, it may mate the implication of a small
not be sufficient for significant modification. If you have to in-
customization or for sub-classed controls. For con- troduce major customizations, consider creating a
trols based on solid Microsoft Active Accessibility control from scratch
implementations, developers can use the IAcces-
sibleEx Interface specification to enhance accessi- If you have to introduce a new control, try to follow
bility with new UI Automation interfaces. standard practices to support UI accessibility. For
example, never assume that all users have access to
This article introduces the basics of using UI Au- pointing devices, and try to use system or desktop
tomation, Dynamic Annotation, and the IAcces- theme colors to avoid accessibility problems with
sibleEx interface for developing controls. custom colors. While the focus of this article is on

Type of Type of Customization Suggested Approach

Custom Control
New control Write new UI code from scratch Implement UI Automation or Microsoft Active Accessibility for
relatively simple controls such as a button.
COMCTRL/USER controls Make small changes that Use Dynamic Annotation to modify (or add) Microsoft Active
with minor modification have no functional impact Accessibility/UI Automation properties.
(background color)
Microsoft Active Accessibility- Add new functionality not Implement the IAccessibleEx interface and address only the gap
based controls compatible with Microsoft from Microsoft Active Accessibility to UI Automation.
Active Accessibility design
Table 1: Suggested solutions by type of custom controls.

36 Making Custom Controls Accessible www.code-magazine.com

programmatic access to the user interface, these fun-
damental accessibility criteria are very important.
Many accessibility practices are available in the
form of industry standards (such as Section 508
of the Federal Rehabilitation Act). Some useful re-
sources are introduced at the Microsoft Accessibil-
ity Developer Center (http://msdn.microsoft.com/
en-us/accessibility/default.aspx).
Part I: Implementing UI Automation
This article talks about implementing UI Automa-
tion for new controls in a few simplified steps. De-
signing a UI Automation object model for a new
user interface is straightforward if you follow a few
basic principles:
1. Define user scenarios clearly. Actual user sce-
narios are helpful. During the design process,
you shouldn’t rely on a single operation device
such as mice, custom color, or screen size.
2. Define the UI functionality as simply as pos-
sible. Is it clickable (such as a button or hy-
perlink)? Does it allow for selection (such as
a list box)?
3. Isolate fundamental UI elements by function
or feature. Structure a complex UI in a way
consistent with user scenarios. A keyboard
navigation model is a useful guide.
4. Match a UI feature from the previous step with
existing controls. The UI Automation Control
Type specification is a useful guideline. If you
have trouble finding the correct control type,
see the Control Pattern specifications and lo-
cate a control type that has the control pat-
terns you want.
Step 1: The Foundation
Note: This sample project is available for download
at MSDN Online (http://msdn.microsoft.com/en-
us/library/ms771315(VS.85).aspx). The Windows
SDK also features two additional UI Automation
provider samples.
The foundation of UI Automation provider objects
is the IRawElementProviderSimple interface im-
plemented by the applications or by the UI Frame-
work that supports UI Automation. Objects that
contain a more complex tree structure should also
implement IRawElementProviderFragment, and
the root for this structure implements IRawEle-
mentProviderFragmentRoot. Objects whose per-
formance is very important can use IRawElement-
ProviderAdviseEvents to know which events they
should raise. Finally, you should use the IRawEle-
mentProviderHwndOverride interface when the
provider needs to reorder the automation tree.
Most providers do not implement this interface.
In this project, you implement only IRawElement-
ProviderSimple, which contains the following
methods:
www.code-magazine.com
// IRawElementProviderSimple MEET MEMBERS OF
STDMETHODIMP GetPatternProvider( MICROSOFT'S
PATTERNID patternId, IUnknown** pRetVal); ACCESSIBILITY TEAM
STDMETHODIMP GetPropertyValue(
PROPERTYID propertyId, VARIANT* pRetVal);
STDMETHODIMP get_HostRawElementProvider(
IRawElementProviderSimple** pRetVal);
STDMETHODIMP get_ProviderOptions(
ProviderOptions* pRetVal);
Before investing time
designing a new control or
Michael Bernstein
customizing a standard control, (Lead Software Design
you should ask yourself: Engineer, User Interface
Platform Team)
“Do I really need that control?”
My team builds great
Accessibility and Speech user
experiences in Windows, as
well as the developer libraries
For the UI Automation Framework to recognize and tools that enable those
your UI Automation provider implementation, you experiences. This is a very
must respond to the WM_GETOBJECT message. rewarding area for me: I can
The UiaReturnRawElementProvider function open a new world of computer
helps produce the key to establish a connection to experiences to those who have
the UI Automation Core: never been able to access them
before
case WM_GETOBJECT:
{
if (_pProxy == NULL)
{
// Create an automation element object
_pProxy =
CCheckbox3UIAProxy::Create(this);
}
return UiaReturnRawElementProvider(
_hwnd, wParam, lParam, _pProxy);
}
break;
Now you are ready to implement the support for
UI Automation properties and pattern interfaces.
Step 2: Implementing the Properties
and Control Patterns David Tseng
(Software Design Engineer in
UI Automation providers add support for prop- Test, User Interface Platform
erties with the GetPropertyValue method. For Team)
control patterns, providers support the corre-
I'm helping to ensure that the
sponding patternId for the GetPatternProvider
user has a great experience no
method. matter how he or she interacts
with a computer—whether it
You can register the properties and control patterns be through speech recognition,
with GUIDs (such as the Toggle_Pattern_GUID) screen reading, or screen
that you map to the corresponding IDs with the magnification. This happens by
UiaLookupId function. For existing control pat- making sure the developer is
terns and properties, you can find corresponding empowered with high quality
IDs in UIAutomationClient.h (such as UIA_Toggle- mechanisms to build the best
PatternId). experience possible.
Making Custom Controls Accessible 37

Vidhya Sriram
(Software Development
Engineer in Test,
User Interface Platform Team)
My team not only works on
making programmatic access
to GUI applications easier
but also enables application
developers to expose richer
information about their
application to assistive
technology products. I love
working with this team for
the commitment and passion
shown towards making
computers easier to use for
physically challenged users
and thereby bringing a positive
change in their lives.
In this example, you get the property value of a IUnknown **pRetVal)
check box: {
// Clear out param
HRESULT CCheckbox3UIAProxy::GetPropertyValue( *pRetVal = NULL;
PROPERTYID propertyId,
IUnknown **pRetVal) if (patternId == UIA_TogglePatternId)
{ {
// Clear out param *pRetVal =
pRetVal->vt = VT_EMPTY; static_cast<IRawElementProviderSimple*>(this);
AddRef();
// Specify ControlType.CheckBox for the }
// ControlType property return hr;
if (propertyId == UIA_ControlTypePropertyId) }
{
pRetVal->vt = VT_I4;
pRetVal->lVal = UIA_CheckBoxControlTypeId; Part II: Using Dynamic Annotation
}
return S_OK; Now, I will talk about using Dynamic Annotation
} for Win32® controls with relatively minor modifica-
tions. The technique was originally introduced with
Microsoft Active Accessibility version 2.0. Its pur-
pose is to enable a custom control to expose acces-
Many accessibility practices sibility information without having to fully imple-
ment the IAccessible interface.
are available in the form of
industry standards With Windows 7 Automation API, you can now
use Dynamic Annotation to override either Micro-
(such as Section 508 of soft Active Accessibility or UI Automation proper-
the Federal Rehabilitation Act). ties of the proxy objects that OLEACC creates for
standard Windows controls (many of those sup-
ported by COMCTL32.dll or USER32.dll). If your
custom control is a slight modification of an exist-
To support controls with a binary status such as the ing control or requires support for only a few ac-
check box (checked or cleared), the Toggle Pattern cessibility properties, Dynamic Annotation is a
is best, and the implementation is straightforward good candidate for implementing your accessibility
as shown in Listing 1. solution.

Here, you get the pattern provider for the toggle
pattern:
HRESULT CCheckbox3UIAProxy::GetPatternProvider(
PATTERNID patternId,
Dynamic Annotation provides three different mech-
anisms for handling annotations: Direct Annota-
tion, Value Map Annotation, and Server Annota-
tion. In this article I discuss only Direct Annotation
and Server Annotation.

Listing 1: The Toggle Pattern supports the binary status of a check box control.
// IToggleProvider to support the Toggle control pattern. *pRetVal = ToggleState_On;
HRESULT CCheckbox3UIAProxy::Toggle() }
{ break;
HRESULT hr = CheckAlive();
if (SUCCEEDED(hr)) case MIXED:
{ {
_pControl->MoveToNextState(); *pRetVal = ToggleState_Indeterminate;
} }
return hr; break;
}
case UNCHEcKED:
HRESULT CCheckbox3UIAProxy::get_ToggleState(ToggleState* pRetVal) {
{ *pRetVal = ToggleState_Off;
HRESULT hr = CheckAlive(); }
if (SUCCEEDED(hr)) break;
{ }
switch(_pControl->CheckboxCheckedState()) }
{ return hr;
case CHECKED: }
{

38 Making Custom Controls Accessible www.code-magazine.com

A sample Dynamic Annotation project is available Other Considerations:
for download at MSDN Online. WinEvents
If you customize a
Applications and the UI Frame-
How It Works work use WinEvents to no- standard control, try not to
tify accessibility applications of overload it with features.
Dynamic Annotation allows you to modify some of changes in the user interface.
the Microsoft Active Accessibility properties, but When one of the following hap- Unnecessary complexity
not every Microsoft Active Accessibility property is pens, call NotifyWinEvent: can result in expensive
relevant. The supported properties are:
• An object is created, de- accessibility support cost,
• Name (PROPID_ACC_NAME) stroyed, shown, hidden, re- not to mention reduced usability.
• Description (PROPID_ACC_DESCRIPTION) ordered, or invoked.
• Role (PROPID_ACC_ROLE) • A selection is changed.
• State (PROPID_ACC_STATE) • A Name, Value, State, De-
• Help (PROPID_ACC_HELP) scription, Location, Parent, Help, Default-
• KeyboardShortcut (PROPID_ACC_KEY- Action, or Accelerator accessibility property
BOARDSHORTCUT) changes in an object.
• DefaultAction (PROPID_ACC_DEFAULTAC-
TION) These calls are needed only for objects for which
you implement accessibility. The system provides
While annotating many of these Microsoft Active accessibility for common objects and will call the
Accessibility properties is straightforward, be care- appropriate events when needed. Not all annota-
ful when you annotate the Microsoft Active Ac- tions require an additional WinEvent.
cessibility Role property. Annotation of Role won’t
affect the behavior of the control or proxy object.
For example, do not annotate a button as a slider Part III: Implementing
because the baseline Win32 button control may IAccessibleEx
not offer all of the functionality of a slider con- … you can now use Dynamic
trol. If you already have a solid Mi- Annotation to override
crosoft Active Accessibility im-
To annotate a control, create an AccPropServices plementation of a control, you Microsoft Active Accessibility or
COM object, call the appropriate SetHwndProp can use the IAccessibleEx in- UI Automation properties.
methods to annotate the control, and then release terface to add more accessibil-
the AccPropServices object. Do not forget to call ity functionality. The advantage
CoInitializeEx to support all COM practices in of this interface is that you can
your WinMain routine (Listing 2). reuse existing accessible object
implementations, properties, object tree structures,
All annotations made in a Microsoft Active Acces- and WinEvents. As long as the baseline Micro-
sibility property are reflected in the UI Automa- soft Active Accessibility implementation is clean,
tion translation as well as in the Microsoft Active IAccessibleEx is an attractive solution to enhance
Accessibility-to-UI Automation Proxy. If you want the custom control for UI Automation.
to override or add a UI Automation property to the
control, you can specify a UI Automation Identi- To extend your Microsoft Active Accessibility im-
fier GUID instead of the Microsoft Active Acces- plementations with the IAccessibleEx interface, do
sibility PropID. the following:

Listing 2: Annotation supports extending UI Automation properties for existing controls.

HRESULT CFancyTextControl::SetAccessibleProperties() hr = pAccPropServices->SetHwndProp(_hwnd,
{ OBJID_CLIENT,
// COM has been assumed to be initialized... CHILDID_SELF,
IAccPropServices* pAccPropServices = NULL; PROPID_ACC_ROLE,
var);
HRESULT hr = CoCreateInstance(CLSID_AccPropServices,
NULL, CLSCTX_SERVER, IID_IAccPropServices, pAccPropServices->Release();
(void**)&pAccPropServices); }
return hr;
if (SUCCEEDED(hr)) }
{
// Annotating the Role of this object be STATICTEXT
VARIANT var;
var.vt = VT_I4;
var.lVal = ROLE_SYSTEM_STATICTEXT;

www.code-magazine.com Making Custom Controls Accessible 39

 Listing 3: These methods help UI Automation map to a corresponding IAccessibleEx instance.
HRESULT CListboxAccessibleObject::GetObjectForChild( if (pAccEx != NULL)
long idChild, IAccessibleEx **pRetVal) {
{ pAccEx->AddRef();
VARIANT vChild; }
vChild.vt = VT_I4; *pRetVal = pAccEx;
vChild.lVal = idChild; return S_OK;
HRESULT hr = ValidateChildId(vChild); }
if (FAILED(hr))
{ HRESULT CListItemAccessibleObject::GetIAccessiblePair(
return E_INVALIDARG; IAccessible **ppAcc, long *pidChild)
} {
// List item accessible objects are stored as an array of if (!ppAcc || !pidChild)
// pointers; for the purpose of this example, I assumed {
// the list contents will not change. Accessible objects return E_INVALIDARG;
// are created only when needed. }
if (itemProviders[idChild - 1] == NULL) CListboxAccessibleObject* pParent =
{ m_control->GetAccessibleObject();
// Create an object that supports UI Automation and
// IAccessibleEx for the item. HRESULT hr = QueryInterface(
itemProviders[idChild - 1] = __uuidof(IAccessible), (void**)ppAcc);
new CListItemAccessibleObject(idChild, if (FAILED(hr))
g_pListboxControl); {
if (itemProviders[idChild - 1] == NULL) *pidChild = 0;
{ return E_NOINTERFACE;
return E_OUTOFMEMORY; }
} *pidChild = m_childID;
} return S_OK;
IAccessibleEx* pAccEx = static_cast<IAccessibleEx*> }
(itemProviders[idChild - 1]);

1. Implement IAccessibleEx. Listing 3 shows the Conclusions and Resources

implementation of the GetObjectForChild
and GetIAccessiblePair methods so UI
Automation can map an IAccessible and
ChildId pair to a corresponding IAccessibleEx
instance.
5. Expose IAccessibleEx with IServiceProvider::
QueryService.
6. Implement IRawElementProviderSimple to
provide property values and pattern providers
for your control.
7. Implement control patterns and properties
that describe your control.
8. Implement WinEvents for your control.
While there are many ways to make custom con-
trols accessible, choosing the right solution can be
tricky. It is easy to underestimate the cost of cus-
tomizing controls. Be aware of the hidden costs of a
complex UI design, and make sure you have a plan
for making your wonderful new controls accessible
to everyone.
The examples introduced in this article are avail-
able for download from the MSDN article, “Mak-
ing Custom Controls Accessible.” (http://msdn.mi-
crosoft.com/en-us/accessibility/cc307845.aspx) A
more thorough discussion of this topic is available
on the MSDN Accessibility Development Center
(http://msdn.microsoft.com/en-us/accessibility/).

Designing a UI Automation
LeAnne Fossmeyer
object model for a new user Michael Bernstein
interface is straightforward if you David Tseng
Vidhya Sriram
follow a few basic principles.

A sample IAccessibleEx project is available for

download at MSDN Online. The sample project
has baseline accessibility support for a custom List
Box control and the UI Automation RangeValue
control pattern.

40 Making Custom Controls Accessible www.code-magazine.com

ONLINE QUICK ID 0811082

Karl Bridge
Karl Bridge is a Programming
Writer, specializing in developer
content, for the Windows
Experience (WEX) organization
at Microsoft.
As the name implies, WEX is
focused on the Windows user
experience for consumers,
IT professionals, and
developers. Karl has written
extensively for MSDN and the
Windows SDK with a focus on
making the Windows developer
community a happy and
integral part of the Windows
experience. Prior contributions
include the developer
documentation for managed UI
Automation and the Windows
Vista Sidebar platform.
Causing confusion,
consternation, and occasional
despair amongst Karl's family,
friends, and co-workers are
his mid-Atlantic sensibilities—
a confluence of English
eccentricity tempered with
Canadian reserve and further
jumbled by numerous strolls
through Europe, the Middle
East, North Africa, and Asia.
Fortunately, Karl has a spell-
checker and an editor to buffer
readers from the inevitable
mélange of mid-Atlantic
spelling and grammar.
Microsoft Accessibility
Testing Tools vs. the Ten-ton
Gorilla of Accessibility
Guidelines Compliance
Close your eyes, ignore your mouse, navigate with your keyboard,
and rely on your ears alone. Now try to use an application you’ve
built or tested. Can you? The few informal tests described in this
article can expose a plethora of usability and accessibility shortcomings,
oversights, and other issues in your application. But how do you test, assess,
and rectify them?
o ensure the most consistent user experience by exposing control properties, methods, and
T across the Microsoft® Windows® product line,
Microsoft has developed a number of tools to
assist developers and testers to verify the accessi-
events such that you can use a tool to manipulate
and interact with the UI manually or through au-
tomation.
bility implementations in their
applications. By incorporating the relevant
Fast Facts components of the Windows
In this article, I describe each According to a study that Automation API and follow-
of these tools and explain what ing common accessible de-
Microsoft commissioned
they are designed to test and sign practices, developers and
the scenarios in which each
and Forrester Research testers can make applications
tool, alone or in combination, conducted: running on Windows signifi-
may be useful. I also describe cantly more useful for people
• 40% (51.6 million) of
how you can use these tools with vision, hearing, or motion
computer users with mild
throughout the development disabilities.
cycle and how to incorporate difficulties or impairments
them into automated and non- are likely to benefit from In the following sections I
automated test frameworks. the use of accessible briefly describe the frameworks
technology. that constitute the Windows
• 17% (22.6 million) of Automation API.
An Overview of the computer users with
Windows Automation severe difficulties or
Microsoft Active
Frameworks impairments are likely to
Accessibility
benefit from the use of
The tools I discuss in this ar- accessible technology. Microsoft Active Accessibility
ticle are designed to test and is a set of COM interfaces and
report on a set of standardized APIs that provide the means to
accessibility issues typical of an application user expose and collect information about UI elements
interface (UI). The tests rely on programmatic ac- and controls. This allows assistive technologies,
cess of the UI implementing the relevant API com- test tools, and automation frameworks program-
ponents defined in the framework known as the matic access to and manipulation of the UI.
Windows Automation API.

Despite the name, the Windows Automation UI Automation

API encompasses both accessibility and automa-
tion frameworks, including Microsoft Active Ac-
cessibility®, UI Automation, the IAccessibleEx
interface, as well as other related and evolv-
ing technologies. Each of these frameworks en-
ables programmatic access to the UI at run time
UI Automation is similar to Microsoft Active Ac-
cessibility in that it provides a means for exposing
and collecting information about UI elements and
controls to support user interface accessibility and
software test automation. However, UI Automation

42 Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance www.code-magazine.com
is a newer technology that provides a much richer
object model than Microsoft Active Accessibility,
and it is compatible with both Win32® and .NET.
Note: UI Automation exposes each object in the UI
as an AutomationElement object to client applica-
tions. An AutomationElement object exposes com-
mon properties of the UI element it represents, such
as the control type and relevant control patterns that
provide properties specific to the control type.
The IAccessibleEX Interface
The IAccessibleEX interface is a new interface
that provides the ability for developers to add par-
tial support for UI Automation to controls with
existing IAccessible COM interface implementa-
tions. This is transparent to testers and the tools
discussed in this article.
Evaluating the Accessibility
of Your Application
There are two primary approaches to evaluating
the accessibility of an application: a manual audit
based on a set of commonly accepted heuristics for
an accessible UI (also known as static testing) and
a partial audit to a fully automated test platform
that evaluates the implementation of an accessibil-
ity framework within your application’s code (dy-
namic testing). Both methods can provide results
based only on a set of guidelines, either from the
heuristics of the audit or those built into the design
of the accessibility framework.
Accessibility Auditing
Typically, auditing your application for accessi-
bility issues is a manual process that involves at-
tempting to access and use the full functionality
of the application exclusively through keyboard
input and other assistive technologies (such as
screen readers). As an auditor, you assess each is-
sue discovered during the audit and log a bug
for each issue as necessary. You then assign a
priority and severity to each bug and, where
possible and appropriate, provide recommenda-
tions for fixing or working around the issues that
the audit uncovers. These recommendations are
based on a combination of experience and acces-
sibility guidelines from industry or advisory groups.
Since there is an inherent risk of testing the func-
tionality of the tool rather than the application,
auditing should be conducted by seasoned testers
familiar with accessibility requirements and the
broad range of assistive technologies.
Accessibility Testing
For the purposes of this article, accessibility test-
ing is associated with automated, repeatable code
www.code-magazine.com
execution based on a set of predefined test cases.
Depending on the sophistication of the tool, I will
supply generic recommendations for specific issues
uncovered during testing.
In addition to compiling reports for basic unit
tests, some of the tools discussed also offer support
for regression and integration testing. This abil-
ity to find and track an accessibility bug through
to resolution in a process you can automate makes
these tools indispensible additions to a tester’s
toolkit.
Where to Get
the Tools
Despite the name, The tools discussed in this
the Windows Automation API article are available from
encompasses both accessibility various locations,
which I’ve listed below.
and automation frameworks.
UI Automation Verify
Available from CodePlex,
the Microsoft open-source
And in this Corner, We Have… The Tools project-hosting Web site:
http://www.codeplex.com/
The following Microsoft tools enable developers to UIAutomationVerify
test accessibility implementations and functionality
in applications using Microsoft Active Accessibility UI Spy
or UI Automation. UI Spy typically ships with the
Windows SDK. However, since
it did not ship in the Windows
UI Automation Verify SDK for Windows Server 2008
and .NET Framework 3.5,
UI Automation Verify (UIA Verify) is a test auto- released in February of 2008,
mation framework that facilitates automated testing you can find it in the earlier
for specific Microsoft UI Automation implementa- Windows SDK for Vista Update:
tions. This framework provides the basis for the UI
http://www.microsoft.
Automation Test Library command-line tool and
com/downloads/details.
Visual UI Automation Verify (Visual UIA Verify), a
aspx?familyid=4377F86D-
Windows-based graphical user interface (GUI) for
C913-4B5C-B87E-EF72E-
the test framework. Most of the UIA Verify frame-
5B4E065&
work functionality is provided through a DLL (UI-
displaylang=en
ATestLibrary.dll) that enables the testing of specific
UI Automation functionality and that logs the test
results.
Note: The Visual UIA Verify GUI does not sup-
port event monitoring or testing and validating
text-based content using the UI Automation Text-
Pattern and TextPatternRange control pattern in-
terfaces. The UI Automation Test Library does not
share these limitations.
To use the UI Automation Test Library, a driver (or
client) application obtains an AutomationElement
object from a control that requires verification. The
driver, in turn, supplies the AutomationElement
object to the UI Automation Test Library, which ex-
ecutes the tests required to validate the UI Automa-
tion implementation. Figure 1 illustrates the typi-
cal workflow when using the UI Automation Verify
framework to test an application.
The Visual UIA Verify framework is a Windows
driver for the UI Automation Test Library. This tool
Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance 43
 is designed for manual testing. It provides an inter-
face to UI Automation Test Library functionality
and eliminates the code overhead of a command-
line tool. Figure 2 illustrates the five functional ar-
eas of the Visual UIA Verify framework.

UI Spy

UI Spy, the precursor to the UIA Verify framework,

is a Windows application designed for UI Automa-
tion testing of specific accessibility issues.

While the lack of built-in automation support limits

the effectiveness of UI Spy, it can still be an invalu-
able tool for viewing an application's UI hierarchi-
cal structure, property values of individual controls,
and event tracking. You can also use UI Spy to in-
Figure1: Here you can see the UI Automation Verify framework workflow between driver, teract with controls through UI Automation control
application, and UIATestLibrary. patterns. This extensive functionality provides the

Figure 2: The five functional areas of the Visual UI Automation Verify framework.

44 Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance www.code-magazine.com
 If you don't need the entire
SDK, you can get the .NET
Framework Tools with UI Spy
by downloading the Windows
SDK for Vista Update. This will
install side by side with the new
Windows SDK for Windows
Server 2008. For example: C:\
Program Files\Microsoft SDKs\
Windows\v6.0\Bin (with UISpy)
or C:\Program Files\Microsoft
SDKs\Windows\v6.1\Bin
UI Accessibility Checker
AccChecker is available from
CodePlex, the Microsoft open-
source project hosting Web
site: UI Accessibility Checker:
http://www.codeplex.com/
AccCheck
Inspect Objects, Accessible
Explorer, and Accessible Event
Watcher
These tools are available as
individual downloads from the
Microsoft Active Accessibility
2.0 Software Development
Kit (SDK) Tools Web site:
http://www.microsoft.
com/downloads/details.
aspx?familyid=3755582A-
A707-460A-BF21-
1373316E13F0&
displaylang=en#filelist

Figure 3: The control view of UI Spy querying displays the UI Automation properties of an element,
running with limited user access.

means for developers and testers to verify that an
application UI is programmatically accessible to as-
sistive technology devices. Figure 3 shows UI Spy
querying and displaying the UI Automation proper-
ties of an element.
content using the TextPattern and TextPattern-
Range control pattern interfaces of UI Automation.
UI Accessibility Checker

UI Spy is similar to Visual UIA Verify in that it does UI Accessibility Checker (AccChecker) verifies the
not support the testing and validation of text-based design and implementation of Microsoft Active

www.code-magazine.com Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance 45
 This ability to find and
track an accessibility bug
through to resolution in a process
you can automate makes these
tools indispensible additions to a
tester’s toolkit.

Accessibility in a control or application UI, regard-

less of the underlying UI framework.

The AccChecker architecture incorporates three

levels, or stages, of functionality:

1. A Windows GUI application designed to sup-

port manual testing, message logging, and
suppression generation.
2. An API designed for use in automated testing
frameworks.
3. A console application that supports unman-
aged test automations typically unable to use
the AccChecker-managed API.

All three levels of AccChecker provide similar

functionality through verification routines that in-
Figure 4: The AccChecker UI runs with these default verifications. clude Microsoft Active Accessibility programmatic
access, programmatic event generation, control
layout and keyboard navigation validation, and a
basic screen-reader transcription service. Figure 4
illustrates the initial view of the AccChecker UI
with default verifications.

Inspect Objects

Inspect Objects (Inspect) is a Microsoft Active Ac-

cessibility-based tool designed for inspecting and
testing IAccessible COM interface properties as
well as navigating UI elements and controls.

The Inspect tool tracks elements by using key-

board focus, mouse cursor movement, or its navi-
gation tools. This built-in navigation is based on a
snapshot view of the Microsoft Active Accessibility
element tree hierarchy.

The Inspect tool is very useful for quickly discov-

ering UI elements that lack unique and meaning-
ful Microsoft Active Accessibility Name property
values. The Name property is required for assistive
technology devices, such as screen readers, to iden-
tify controls such as buttons, text boxes, list boxes,
images, and links. Figure 5 shows the Inspect tool
querying the Microsoft Active Accessibility prop-
erties of an element in the Notepad menu.

Accessible Explorer

Accessible Explorer (AccExplorer) is a Microsoft

Figure 5: The Inspect tool UI queries Notepad. Active Accessibility-based tool that offers func-

46 Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance www.code-magazine.com
tionality that is very similar to the Inspect tool.
However, it differs in two important features:

• A somewhat limited ability to track and dis-

play a live view of the Microsoft Active Ac-
cessibility element tree.
• Microsoft Active Accessibility verification of
specific UI elements. The AccExplorer tool
provides a passed, failed, and warning system
on selectable Microsoft Active Accessibility
properties based on the HRESULT returned
by the automated query.

Figure 6 illustrates how to use the AccExplorer

tool to verify specific Microsoft Active Accessibil-
ity properties for the Notepad content pane.

Accessible Event Watcher

Accessible Event Watcher (AccEvent) is a com-

mand-line tool that provides developers and tes-
ters with a means of validating Microsoft Active
Accessibility-based WinEvents raised by UI ele-
ments. The AccEvent tool reports events when an
element is invoked or selected, when it changes
state, and when there is a change in focus.

You can track in-process and out-of-process events

based on the options available with the Microsoft
Active Accessibility SetWinEventHook function.

Even though the AccEvent tool may report some

UI Automation-specific WinEvents, you should
use a dedicated UI Automation client for testing
UI Automation events due to the differing de-
signs of the event object models in the two frame
works.

Figure 7 illustrates how the AccEvent tool catches Figure 6: The AccExplorer tool can test specific Microsoft Active Accessibility properties in the
the toolbar clock events. Notepad content pane.

Conclusion
No tool alone can definitively test and guaran-
tee compliance with all accessibility guidelines.
However, the tools described in this article make
achieving that goal significantly easier through
their comprehensive coverage of the many accessi-
bility compliance issues that software vendors face
today.

Karl Bridge

Figure 7: AccEvent reports time changes for the toolbar clock.

www.code-magazine.com Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance 47
ONLINE QUICK ID 0811092

Internet Explorer 8 New

JP Gonzalez-Castellan
JP is the Program Manager
responsible for improving
the Accessibility experience
in Internet Explorer. He was
born in Argentina and got his
Computer Science degree
at Rose-Hulman Institute of
Technology in Indiana. He has
been involved at Microsoft with
Accessibility since 2004.
example is the wheelchair access ramps. After the
Accessibility Features
Windows® Internet Explorer® 8 has a lot of cool new features
that make Web page browsing more accessible. I’m going to
cover Caret Browsing, Zoom Version 2, High DPI, Accessibility Rich
Internet Applications (ARIA) support and User Interface Automation (UI
Automation) support. Caret Browsing is particularly helpful for low-mobility
users. Zoom 2 and High DPI support targets low-vision users, and the new
ARIA and UI Automation support targets screen-reader users. Low-mobility
users prefer to use the keyboard or devices that interact with a virtual
keyboard. Some low-vision users require specialized assistive software to
interact with computers while others can do well with features and tools
shipped with the operating system.
hen you make something accessible you are You can hold the SHIFT key down and press the
W
not only impacting the accessibility com-
munity but the entire user base. My favorite
arrow keys to select text.

As I mentioned before, many users prefer the key-

American Disability Act was passed, public gather- board to the mouse because they find it faster for
ing places, like airports, added wheelchair ramps. certain tasks. Users can select a word or phrase,
Airports soon noticed that mothers with baby stroll- as you see in Figure 2, bring up the Accelerators
ers and passengers with rolling through the context-menu key,
suitcases were using the ramps select Translate with Windows
too, since it was easier than Fast Facts Live, and see the selection’s
picking up the strollers and According to a meaning in Spanish, without
suitcases over the ledge. Im- ever taking their hands off the
Harris Poll survey,
prove the accessibility of your keyboard.
software and everybody wins. Americans with disabilities
When you provide robust key- spend twice as much time
board navigation to benefit us- on the Internet as those Zoom 2
ers who can’t use the mouse, without disabilities.
you are also enabling users who Roughly 750,000 million Page zoom lets you enlarge or
want to perform tasks faster, people the world over reduce the view of a Web page to
since there are now fewer tasks improve readability. The feature
have some disability.
they need to use the mouse for. is particularly useful on very large
and very small displays, allowing
for scaling of content while maintaining the intended
Caret Browsing page layout. The second iteration of the zoom feature
set (the first shipped in Internet Explorer 7) focuses
…users can select a word Caret Browsing is a new Acces- on providing a higher-quality, more predictable, and
or phrase, bring up the sibility feature that allows users persistent zooming experience. Primary features in
to navigate a Web page using a this release include the elimination of horizontal
Accelerators through the moveable cursor on the screen scroll bars for the majority of mainstream scenar-
context menu key, select with the keyboard. Users can ios and the introduction of persistent zoom states.
Translate with Windows Live, select and copy text as well as I personally find this feature useful when using my
tables and images using only the TV to browse the Web. I sit far away from the TV;
and see the selection’s meaning keyboard. using a wireless keyboard, I use the Zoom to make
in Spanish, without ever taking the pages more readable. The extensive keyboard
Moving the cursor within the shortcut’s list allows me to use the mouse as little as
their hands off the keyboard. text of a Web page is similar to possible.
moving the cursor within the
text of a Microsoft Word docu- Zoom 2 works by adjusting the internal state of the
ment, as shown in Figure 1. Pressing F7 toggles dots-per-inch (DPI) calculations within Internet
Caret Browsing on or off. You can enable Caret Explorer 8. To understand this better, consider how
Browsing for one tab or for all tabs and windows. different display settings affect a page's layout.

48 Internet Explorer 8 New Accessibility Feature www.code-magazine.com

 Resources and
References
You can learn much more
about ARIA and how to use it
in your Web content from the
following W3C resources:
Introduction to ARIA http://
www.w3.org/TR/wai-aria-
primer/
ARIA Roles, States and
Figure 1: Caret is visible on the screen for the first time. Properties http://www.w3.org/
TR/wai-aria/
ARIA Best Practices
http://www.w3.org/TR/wai-
aria-practices/
To learn more about Windows
Accessibility APIs go to
http://msdn.microsoft.com
and search for Microsoft
Active Accessibility and UI
Automation.

Figure 2: Pressing the context menu key brings up the context menu; so you can access Web Accelerators using only the
keyboard.

Screen Resolution: Changing from a 1200 ×
1600-pixel screen resolution to an 800 × 600-pixel
resolution means changing the spacing between
pixels. In other words, pixels are effectively bigger.
A screen element of 100 × 100 pixels is now twice
as big on the 800 × 600-pixel screen as it was on the
1200 × 1600-pixel screen.
System DPI: Changing from 96 DPI to 192 DPI,
but keeping all other factors the same, implies more

www.code-magazine.com Internet Explorer 8 New Accessibility Feature 49

 pixels in an inch. The spacing between pixels has
not changed. However, more pixels are required to
form a logical inch.
Zoom Factor: Changing the zoom factor from
1 to 2 means that everything should be twice as
big. Figure 3 shows how to calculate the Zoom
Factor.
html#percentage-units). The Internet Explorer page
zoom feature cannot change any of the above set-
tings except for the zoom factor. However, it can
lay out or render a page at a DPI setting that is
different from the configured system DPI, thus
achieving zoom results.

HTML Layout Measurements: This value is al- You can see the importance
ways 96 DPI, as explained in the CSS 2.1 speci- of carefully selecting
fication (http://www.w3.org/TR/CSS21/syndata.
the anchor for positioned elements
and using relative dimensions
instead of absolute or
device-dependent dimensions
(such as pixels).

The sample code in Listing 1 divides the screen into

quadrants of color, using absolute positioning, to
show the difference between the Internet Explorer
8 adaptive page zoom and the Internet Explorer 7
Figure 3: Zoom factor = System DPI/Layout DPI. optical page zoom

Listing 1: Sample Web page to understand better the new Zoom. The boxes in Figure 4 are absolutely positioned,
and they are anchored to the corners of the view-
<html> port. Each is sized using relative dimensions (%).
<head> The box in the middle is anchored to the center
<title> Zoom 2 Sample </title> of the body. In Figure 5, you can see how Inter-
net Explorer 7 displays a 175% zoom while Figure
</head> 6 shows how Internet Explorer 8 handles a 175%
zoom.
<style>
You can see the importance of carefully selecting
div { the anchor for positioned screen elements and
width: 50%; height: 50%; position: absolute; using relative dimensions instead of absolute or
} device-dependent dimensions (such as pixels). If
you had positioned the center DIV using pixels,
#center { the DIV would have moved from the center when
width: 20%; height: 20%; left: 40%; zoomed.
top: 40%; border: 5px black solid;
}
High DPI
</style>
You can use the Windows DPI Scaling (https://window-
<body style="border:0; padding:0;"> shelp.microsoft.com/Windows/en-US/help/454f5078-2b51-
4cda-b4c0-6391e870c41d1033.mspx) feature to scale
<div id="red" style="left:0; top:0; width:50%; up Windows fonts and user interface (UI) elements
height:50%; background-color:red;"> Red </div> (such as buttons, icons, input fields, and so on) by
a given percentage. This is different from the scaling
that occurs when you lower the display resolution be-
<div id="blue" style="left:50%; top:0; width:50%;
cause in the DPI Scaling case, Windows is providing
height:50%; background-color:blue;"> Blue </div>
fonts and UI elements that are drawn with more pix-
els, resulting in a larger, higher fidelity, and sharper
<div id="yellow" style="left:0; top:50%; width:50%;
Windows experience.
height:50%; background-color:yellow;"> Yellow </div>
Like Windows Vista, the Internet Explorer 8 user
<div id="green" style="left:50%; top:50%; width:50%;
interface is entirely DPI Scaling-aware, and you
height:50%; background-color:green;"> Green </div>
will notice that all UI elements and fonts are scaled
accordingly. Additionally, larger and higher fidelity
</body> icons are used. Notice the difference in the Internet
Explorer chrome when Windows DPI Scaling is set
</html> to 96 DPI (Figure 7) and 120 DPI (Figure 8).

50 Internet Explorer 8 New Accessibility Feature www.code-magazine.com

Figure 4: Notice that at 100% the square is centered in the viewport.

By default, Internet Explorer 8 will zoom the con-

tent of the Web to match your Windows DPI Scal-
ing settings. (Please note that this is a change from
Internet Explorer 7, which did not zoom to match
Windows DPI Scaling settings.) For example, if
you set your DPI Scaling to 120 DPI, Internet Ex-
plorer 8 will zoom the content of Web pages by
125%.

ARIA syntax is
a great mechanism to
unlock your dynamic,
rich Web applications
for everyone.

For more information, especially for developers

who want to take advantage of High DPI in their
Web pages and WebOCs, please see: Making the
Web Bigger (http://go.microsoft.com/fwlink/?LinkID=12 Figure 5: 175% zoom in IE 7 magnifies the text and divs in the Web page and forces the
5391&clcid=0x409) horizontal and vertical scrollbars to show up.

www.code-magazine.com Internet Explorer 8 New Accessibility Feature 51

 Figure 6: 175% zoom in IE 8 magnifies the Web page’s text but not the divs. So the layout of the page is maintained and hence no horizontal and vertical scrollbars
are needed.
Figure 7: Internet Explorer Chrome
at 96 DPI. Figure 8: Internet Explorer Chrome at 120 DPI.
ARIA and UI Automation Support
In the latest release of Internet Explorer 8, Micro-
soft has made a big investment in Accessible Rich
Internet Applications (ARIA). The W3C defines
ARIA as a syntax for making dynamic Web con-
tent and custom UI accessible. Windows Internet
Explorer 8 recognizes the ARIA role, state, and
property information from controls and exposes it
to assistive technologies. Assistive technologies can
then use Microsoft Active Accessibility® and/or UI
Automation Accessibility APIs, which are support-
ed for the first time in Internet Explorer, to retrieve
the information. Instead of building separate simpli-
52 Internet Explorer 8 New Accessibility Feature
fied Web pages for accessibility, you can use ARIA
to mark up your rich Web applications with roles,
states, and properties. For example, to match the
behavior you create through script, you can define
a DIV element as a button, check box, or another
ARIA role.
ARIA syntax is a great mechanism to unlock your dy-
namic, rich Web applications for everyone. Today Web
pages with dynamic content and custom UI controls
(such as TreeView controls) have to reuse existing
HTML controls. For example, you can make custom
TreeView controls accessible by defining each item as
an HTML list element. This approach can add com-
plexity to the code, make it more difficult to imple-
ment, and prevent all users from getting the same rich
behavior.
Code Sample: Create an Interactive
TreeView Control Using ARIA
The code sample in Listing 2 uses ARIA to create
an interactive TreeView control using HTML and
www.code-magazine.com
 Listing 2: Creating an interactive TreeView control.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" }
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> }
}
<html> </script>
<!-- saved from url=(0014)about:internet <-runs script on your local machine </head>
without gold bar prompt-->
<head> <body style="margin: 0px">
<title>TreeView ARIA Example</title> <!-- Create an unordered list, which will represent the top level tree
<style type="text/css"> element in ARIA -->
li <ul id="Tree" role="tree">
{
list-style-type: none; <!--
} Create a list item that will represent the parent tree item and
</style> set its ARIA state to
<script type="text/javascript"> collapsed by setting 'aria-expanded="false"'
// Function to expand and collapse tree items by referencing the -->
ariaExpanded attribute. <li role="treeitem" id="A1" aria-expanded="false"
function Toggle(root, itemId) title="Transportation">
{ <a href="javascript:Toggle('A1','sub1')">+</a>
var el = document.getElementById(itemId) <a href="http://en.wikipedia.org/wiki/
var treeitem = document.getElementById(root) Transportation">Transportation</a>
if (el) <!-- Create a group of list items -->
{ <ul style="display:none" id="sub1" role="group">
if (treeitem.ariaExpanded == "true") <a href="http://en.wikipedia.org/wiki/Car" role="treeitem"
{ title="Car"><li> Car </li> </a>
treeitem.ariaExpanded = "false"; <a href="http://en.wikipedia.org/wiki/Boat" role="treeitem"
el.style.display = "none"; title="Boat"><li> Boat </li> </a>
treeitem.childNodes[0].innerText="+ "; <a href="http://en.wikipedia.org/wiki/Bike" role="treeitem"
} title="Bike"><li> Bike </li> </a>
else </ul>
{ </li>
treeitem.ariaExpanded = "true"; </ul>
el.style.display = "block"; </body>
treeitem.childNodes[0].innerText="- "; </html>

JavaScript. Users of Internet Explorer 8 using an as- Conclusion

sistive technology (such as a screen reader) can in-
teract with this control and experience the same rich
behaviors users without vision impairments do. If you
don’t have access to a screen reader you can use the
Inspect32 testing tool available at http://www.micro-
soft.com/downloads/. You can experience the benefits
of ARIA first hand by hovering with the mouse or
tabbing with the keyboard through the TreeView. The
role displayed in Inspect 32 is “treeview item” instead
of “outline item”. Similarly a screen reader will speak
out “treeview item” instead of “outline item”.
With Internet Explorer 8 and its support for ARIA
and UI Automation, developers have a number of
useful tools for developing accessible, rich Web
pages. With Caret Browsing, the new Zoom func-
tionality, and High DPI support, Internet Explor-
er 8 helps make content more accessible to your
users.
JP Gonzalez-Castellan

Users of Internet Explorer 8

using an assistive technology
(such as a screen reader)
can interact with a custom the
TreeView control and experience
the same rich behaviors users
without vision impairments do.

www.code-magazine.com Internet Explorer 8 New Accessibility Feature 53

Are You Ready To Start?
EPS
Can Help!
EPS Software provides:

Application
Development
• .NET 3.0 (WPF, WCF)
• Windows & Smart Client
• ASP.NET & AJAX Web
• Tablet PC & Mobile PC
• Windows Mobile

Prototyping Services
Situation Assessment
Project Management
Mentoring
Training

Contact us at:
info@eps-software.com
866-529-3682

www.eps-software.com
ONLINE QUICK ID 0811102

A Pragmatic Approach to
WPF Accessibility
As is often the case, applications are not typically designed
with accessibility in mind. Usually, applications are designed to satisfy
business requirements. If those business requirements do not include
accessibility, more likely than not, the application as a whole will be
inaccessible to important segments of users. There are, however, steps you
Alvin Bruney can take to mitigate this common lack of foresight in requirements analysis.
Alvin Bruney is a Technology
Specialist working for Royal
o start off a project right, address accessibility in WPF is due to the fact that the new UI Automa-
Bank of Canada in the .NET
Centre of Excellence program.
He is a Microsoft Press author
and a long-time ASP.NET MVP.
T at the design level, even when overlooked in
the business requirements. Treat accessibility
as you would usability: neither is explicitly a busi-
tion handles accessibility support behind the scenes.
You can take advantage of this framework with new
projects and use the framework’s bridge and proxy
ness requirement, but both are required parts of a technology to extend existing projects’ feature sets.
successful user interface. Think about how users
can interact with your user interfaces and plan for UI Automation is more robust and exposes more
alternate accessibility with screen readers, magni- programmatic information to assistive technolo-
fiers, visual cues, keyboard usage, and so on. gies, allowing for an improved experience. You
can read more about UI Auto-
Another way to ensure acces- Fast Facts mation elsewhere in this mag-
sibility is to use the right devel- azine as well as at the MSDN®
opment tools. The good news
According to the U.S. Accessibility Developer Cen-
is that all versions of Visual President's Committee on ter. For now, I want to focus
Studio® address accessibility; Employment of People with on testing accessibility.
the bad news is that this sup- Disabilities, the majority of
port doesn’t guarantee perfect accommodations cost less
coverage, and you should in- than US$500, and many Testing WPF Applications
clude test and validation tools
have little or no cost at
in your development arsenal. When you build WPF applica-
Once you’ve got your design
all. However, the average tions, you can test accessibil-
in place and you’ve built what recruiting cost equals ity support using tools such as
you believe to be an accessible more than 13 percent of an Jaws, UI Spy, Windows-Eyes,
application, control, Web page, employee's annual salary. and ZoomText to name a few.
or service, you can use these There’s even an accessibil-
tools to test your deliverables ity checker, shown in Figure 1,
for compliance with accessibility requirements. on the Visual Studio tools menu for applications
based on Web site project templates. This tool ana-
In this article, I’ll talk about lyzes Web pages against Web Content Accessibility
these issues with respect to Win- Guidelines (WCAG) and Section 508 of the U.S.
Treat accessibility dows® Presentation Foundation Rehabilitation Act.
as you would usability: (WPF). I’ll also touch on some
areas where current offerings A strategy that I recommend is to use a combi-
neither is explicitly a from adaptive technology ven- nation of tools so that you can achieve great-
business requirement, dors present challenges to both er accessibility testing coverage. For instance,
users and developers. the Accessibility Checker in Visual Studio tar-
but both are gets Web pages; UI Spy targets UI Automa-
required parts of a tion compliance; AccChecker targets Microsoft®
successful user interface. WPF Has Accessibility Active Accessibility® compliance and general acces-
Built In sibility testing. You can decide which tool is a better
fit based on your needs and the underlying acces-
sibility framework you’re using. You might use UI
When you build a WPF application, you still need Spy and UIA Verify for testing WPF applications
to design for accessibility; however, it’s an easier based on UI Automation and Accessibility Checker
road to travel today because of the new accessible for those based on Microsoft Active Accessibility.
technology framework that you can build on.
Bear in mind that none of these tools help with
One reason for the increased accessibility support designing accessibility into your applications. They

56 A Pragmatic Approach to WPF Accessibility www.code-magazine.com

can only validate that what you did design and build
complies with known, testable requirements.
Challenges with the New Technology
For the most part, supporting accessibility with
the UI Automation framework works as intended.
However, I want to walk through some cases where
potentiality may not align with reality. UI Automa-
tion is new, and some assistive technology vendors
(ATVs) do not yet support it, which presents sev-
eral challenges. Here are some specifics you may
run into, as well as a few workarounds.
JAWS Screen Reader
The screen reader JAWS has a “JAWS Cursor” mode
that allows users to read text and UI information line
by line on the screen. When in this mode, the reader
does not read the screen information for applications
built with WPF native controls. The root cause is the
current lack of support for UI Automation in JAWS.
Fortunately, Jaws Cursor mode is not the primary inter-
face of the screen reader, and the workaround is simply
to avoid this mode where possible. As of this writing,
JAWS plans to fully support UI Automation in 2009.
Win32 Controls
WPF supports legacy Win32 controls with the Hwnd-
Host control, providing an easier migration path from
Win32 to XAML-based applications. However, there’s
Figure 1: The accessibility checker in Visual Studio 2008 analyzes Web pages and displays results.
www.code-magazine.com
a price for supporting this backward compatibility:
bitmaps and images that use the legacy raster technol-
ogy begin to blur and distort under
increased DPI. Users with certain
types of visual impairments may
However, there’s a price for
have difficulty working with such
an application. supporting this backward
Even users without visual impair-
compatibility: bitmaps
ments who work with these ap- and images that use the legacy
plications at non-native DPIs can
suffer eye strain after prolonged
raster technology begin to blur and
exposure. To put this in perspec- distort under increased DPI.
tive, consider a desk clerk work-
ing eight hours a day for several
months with a software application where some of
the images are blurred. This situation can easily lead
to a health issue, which can quickly turn into a li-
ability issue.
To make working with your applications easier on ev-
eryone’s eyes, you should create vector-based images
and bitmaps using tools such as Microsoft Blend, or
convert existing raster-based images to vector-based
images. Vector-based images can respond automati-
cally to changes in DPI settings while retaining vi-
sual clarity. There are already a number of products
that offer this conversion for free or for a fee.
ZoomText
Users with visibility impairments use programs like
ZoomText to read content on screen. ZoomText, created
by Ai Squared, is a screen reader-magnifier for Microsoft
A Pragmatic Approach to WPF Accessibility 57

Expanding Law
On October 2, 2007, National
Federation of the Blind (NFB)
v. Target Corporation was
certified as a class action
lawsuit. The plaintiff, NFB,
alleges that the Target.com
Web site violates the rights
of the disabled because
some parts of the site are
inaccessible to the visually
impaired.
The case raises eyebrows as
the first accessibility-based
class action lawsuit to target a
public facing Web site owned
by a non-federal entity. These
lawsuits have tended to target
government institutions.
If NFB prevails, the mandate
for and scope of accessibility
requirements may apply to
all Web pages. But, as I have
shown you in this article,
there are proactive steps
that IT organizations can
take to comply with these
requirements.
58 A Pragmatic Approach to WPF Accessibility
Figure 2: Parts of the user interface are cut off on this magnified screen.
Windows. In some instances, the part of the magnified
screen on a WPF native application may completely dis-
appear or become unreadable as in Figure 2.
This happens because the hardware-accelerated WPF
rendering bypasses the software rendering layer. While
you can avoid this in general either by turning off hard-
ware acceleration or by using the Magnification API,
the latter option is not available with the ZoomText
application. It’s important to note that while this is not
strictly a WPF issue, the end result is that WPF-based
applications may be inaccessible to ZoomText.
Unfortunately, the workarounds are not particularly
pleasant. One option is to use another screen magni-
Figure 3: The hardware acceleration slider lets you disable
hardware acceleration in Windows XP.
fier that supports UI Automation. Today, the choices
available are pretty close to nil. The situation is expect-
ed to improve as ATVs start supporting WPF, UI Auto-
mation, and other standard accessibility technologies.
A second option is to disable hardware acceleration
for Microsoft Windows in the dialog shown in Figure
3. Hardware acceleration uses DirectX® to improve
video and animation quality. When you disable hard-
ware acceleration, DirectX is not used.
Hardware Refreshes
Another accessibility issue may arise particularly in
the workplace with hardware refreshes. Periodically,
the computers or workstations that employees use
are upgraded in a hardware refresh cycle, includ-
ing new monitors. While applications built on WPF
technology will automatically scale correctly to fit
the increased screen resolution, legacy-based con-
trols may not scale as you expect.
Some common controls such as the WPF dialog box
are not WPF native. Instead, they are simply wrappers
around legacy Win32 technology. Usually, issues with
these controls manifest themselves in any number of
ways, from controls with chopped or unreadable text
to controls that do not maintain their layout integrity
under different screen conditions, such as the DPI
changes and high contrast mode in Figure 4.
Migration from Legacy Technologies
Even legacy Visual Basic® 6 (VB6) migration can
inadvertently introduce accessibility issues in WPF.
Let’s examine one common scenario. Most VB6
www.code-magazine.com
common controls have an autosize property that al-
lows the control to grow or shrink as the form or
parent container is resized. By default, the autosize
property is set to false.

When these applications are migrated to .NET, the

default setting for the autosize property remains set
to false, and the result is that the contents of the con-
trol cannot grow as screen conditions change. Con-
trast this to a new application developed using native
WPF controls where the autosize property isn’t even
present. Here, the control automatically grows as re-
quired to meet users’ needs.

Figure 4: The effects of legacy technology can manifest as

The visually impaired user
couldn’t care less which
technology is to blame;
the application is inaccessible!
You need to plan for this
progression to avoid introducing
accessibility issues into your
applications.
Another accessibility issue arises when these legacy
VB6 controls are migrated first to .NET, where they
retain their “inaccessible” settings, and then to WPF
inside a WindowsFormsHost control. It’s important to
note that this not a WPF issue at all. In fact, its roots
are buried deep in VB6 legacy settings. However, WPF-
based applications can inherit the accessibility burden.
The visually impaired user couldn’t care less which
technology is to blame; the application is inaccessi-
unreadable text in high contrast mode.
ble! You need to plan for this progression to avoid in-
troducing accessibility issues into your applications.
Conclusion Why Bother
Designing and
In my experience, WPF has made the most progress Building Accessible
on the accessibility front by far. However, you cannot
rely on a platform and tools alone to cover accessibility
Applications?
mandates. The onus is on you to design for accessibility For one, an accessible
so that you can deliver a better product to your clients. application caters to the largest
possible audience, and that’s
As the WPF technology gains traction, assistive technol- always good for business.
ogy vendors will come on board in increasing numbers The services you offer won’t
with support for the new UI Automation framework. exclude the estimated 20
At that point, you can expect WPF-based applications million Americans with some
to have full accessibility support. Until that time, you sort of disability (source:
should plan for accessibility and use the available tools American Census Bureau,
described earlier to help test for accessibility coverage. 2006).
Alvin Bruney

ADVERTISING INDEX
Advertisers Index
Active Software Professionals VFP Conversion
www.aspalliance.com 6 www.VFPConversion.com 31

Applied Innovations WestWind

www.appliedi.net 76 www.west-wind.com 35

Aspose Xiine
www.aspose.com 75 www.xiine.com 69

Code Magazine
www.code-magazine.com 7

Developer Express
www.devexpress.com 2
Advertising Sales:
DevMavens Lake Quincy Media LLC
www.devmavens.com 15 330-677-2440
ssmith@lakequincy.com
msmith@lakequincy.com
EPS Software Corp.
www.eps-software.com 54, 60 Sales Managers
Erna Egger
Microsoft +43 (664) 151 0861
www.microsoft.com/enable/ 5 This listing is provided as a courtesy to our readers and erna@code-magazine.com
advertisers. Tammy Ferguson
Tech Conferences The publisher assumes no responsibility for errors 832-717-4445 ext 26
www.devconnections.com 41 or omissions. tammy@code-magazine.com

www.code-magazine.com A Pragmatic Approach to WPF Accessibility 59

ONLINE QUICK ID 0811112

Brendan McKeon
Brendan is a senior software
development engineer in the
Windows division. He’s worked
at Microsoft for 10 years and
was a key senior contributor to
UI Automation.
Writing a UI Automation
Provider for a Win32-based
Custom Control
Do you have a complex custom control that you want to make
programmatically accessible, but you aren’t sure how? Custom controls,
by their nature, tend to be very diverse: each is typically written for a specific
purpose, making it difficult to generalize implementation details. How do you
know what to implement? You should consider supporting accessibility for any
custom control that performs its own rendering and input management—routing
mouse and keyboard input—within the HWND that it owns.
n this article, I walk through the steps you need to providers expose information about the UI as a tree of

I follow to implement a server-side provider for a

Win32-based custom control
using Microsoft® UI Automa-
tion. The example I use assumes
the custom control has its own
Handle to a Window (HWND)
and Window Procedure. While
the emphasis is on unmanaged
C++ implementations, the tech-
niques are equally applicable to
managed code.
What is a Provider?
IRawElementProviderSimple objects. The IRawEle-
Fast Facts
In general, UI Automation
uses client-side providers to
support built-in controls such
as Win32 buttons,
while server-side providers
expose information about
custom controls to
UI Automation.
mentProviderSimple interface
is the base interface representing
and exposing information about
a single UI element.
The MyCustomControlProvider
class only exposes information
for the overall HWND. Here, I
complete the implementations
of these IRawElementProvid-
erSimple methods. In this ex-
ample, I expose a custom name
and specify a control type:

UI Automation presents the UI on the desktop to a

client application in the form of a tree of IUIAuto-
mationElement objects, each of which can expose
properties describing the UI. The UI itself must sup-
ply these properties using APIs and messages. The
work of calling the UI-specific APIs and returning
the information back to UI Automation is done by
a component called a provider.
IFACEMETHODIMP MyCustomControlProvider ::
get_ProviderOptions(ProviderOptions * pRetVal)
{
*pRetVal = ProviderOptions_ServerSideProvider
| ProviderOptions_UseComThreading;
return S_OK;
}

UI Automation comes with providers for the vari-

ous standard OS controls, such as Win32 button
and list controls, already built in. To make a cus-
tom control accessible through UI Automation, you
need to supply a UI Automation provider for that
control.
Exposing Built-in Properties of a Single
Element
The first step to making your custom control ac-
cessible is to create a class that implements the UI
Automation provider interface, IRawElementPro-
viderSimple, as shown in Listing 1.
While UI Automation clients see the entire user inter-
face (UI) as a tree of IUIAutomationElement objects,
This method tells UI Automation that the class
is a server-side provider rather than a client-side
provider. In general, UI Automation uses client-
side providers to support built-in controls such as
Win32 buttons, while server-side providers expose
information about custom controls to UI Automa-
tion.
The second provider options flag tells UI Automa-
tion to use COM’s threading rules when it makes
calls using the interface: incoming calls must come
in on the same thread on which you handed out
this object, which saves you the trouble of having
to do synchronization ourselves. This assumes that
CoInitialize was called on the main UI thread dur-
ing application startup so that the main UI thread is
Apartment-threaded.

62 Writing a UI Automation Provider for a Win32-based Custom Control www.code-magazine.com

 Listing 1: This example is an initial implementation of a provider class, MyCustomControlProvider.
#include <UIAutomation.h> // Standard implementations of AddRef, Release,
// QueryInterface go here
// New class that exposes a custom control to UI Automation.
// This implements the provider-side interfaces. // IRawElementProviderSimple methods
class MyCustomControlProvider: public IRawElementProviderSimple IFACEMETHODIMP get_ProviderOptions(ProviderOptions * pRetVal);
{ IFACEMETHODIMP GetPatternProvider(PATTERNID idPattern,
HWND _hwnd; IUnknown ** pRetVal );
IFACEMETHODIMP GetPropertyValue(PROPERTYID idProp,
// Reference to control data or object model goes here VARIANT * pRetVal );
IFACEMETHODIMP get_HostRawElementProvider(
public: IRawElementProviderSimple ** pRetVal );
};
MyCustomControlProvider(HWND hwnd): _hwnd(hwnd)
{
}

Listing 2: The WM_GETOBJECT message ensures UI Automation can access an instance of your provider.
LRESULT MyCustomControlWndProc(HWND hwnd, UINT uMsg, lParam, pProvider);
WPARAM wParam, LPARAM lParam) pProvider->Release();
{ return lres;
switch(uMsg) }
{ }
case WM_GETOBJECT: return DefWindowProc(hwnd, uMsg, wParam, lParam);
{ }
MyCustomControlProvider * pProvider =
new MyCustomControlProvider();
LRESULT lres = UiaReturnRawElement(hwnd, wParam,

IFACEMETHODIMP MyCustomControlProvider::
Patterns are interfaces that UI Automation uses
GetPropertyValue(PROPERTYID idProp,
to expose control-specific functionality such as
VARIANT * pRetVal )
selection or command invocation. I’ll revisit {
them later, so I’ll set the return value to NULL pRetVal->vt = VT_EMPTY;
here: if(idProp == UIA_NamePropertyId)
{
IFACEMETHODIMP MyCustomControlProvider:: pRetVal->bstrVal = SysAllocString(L“MyControlName“);
GetPatternProvider(PATTERNID idPattern, pRetVal->vt = VT_BSTR;
IUnknown ** pRetVal ) }
{ else if(idProp == UIA_ControlTypePropertyId)
*pRetVal = NULL; {
pRetVal->lVal = UIA_ButtonControlTypeId;
return S_OK;
pRetVal->vt = VT_I4;
}
}
return S_OK;
The GetPropertyValue method handles element }
properties. UI Automation divides properties into
two categories: those related to a specific area of
functionality—for example, those dealing with se- Finally, get_HostRawElementProvider tells UI Au-
lection, which only apply to controls that manage tomation which HWND this control is associated
selection—and those that broadly apply to any el- with:
ement, including Name, Enabled, and Control
Type. This latter category of property is handled IFACEMETHODIMP get_HostRawElementProvider(
in the GetPropertyValue method. IRawElementProviderSimple ** pRetVal )
{
For now, I’m supplying suitable values for the return UiaHostProviderFromHwnd(_hwnd,
element’s Name and Control Type properties. pRetVal);
These are two of the more important properties, as }
screen readers typically read the name and type of
a control as focus changes, but I can support any Once you’ve implemented the class, you have to
number of properties later: ensure that UI Automation can actually access an

www.code-magazine.com Writing a UI Automation Provider for a Win32-based Custom Control 63

 Listing 3: One technique for handling a provider’s lifetime is to create and update a state flag.
class MyCustomControlProvider: public IRawElementProviderSimple
{ IFACEMETHODIMP MyCustomControlProvider::GetPropertyValue(
HWND _hwnd; PROPERTYID idProp, VARIANT * pRetVal )
{
// Create a pointer to internal control state, set to // Always clear out parameter before anything else
// non-NULL in the constructor. pRetVal->vt = VT_EMPTY;
// OnUIDisposed sets it to NULL to indicate that UI is no
// longer present. Using NULL to indicate this avoids having // Now I check to see if the UI is still present
// a separate flag. if(_pControl == NULL)
// Incoming UI Automation interface methods must check this {
// before use. return UIA_E_ELEMENTNOTAVAILABLE;
// }
IMyInternalControlInterface _pControl;
// UI is still present, so code continues as before
... if(idProp == UIA_NamePropertyId)
{
// Called when corresponding UI is deleted ... // get name from _pControl and return via pRetVal.
void OnUIDisposed(); }
}; ...

void MyCustomControlProvider::OnUIDisposed()
{
_pControl = NULL;
}

instance of it. This is done by handling the WM_ JECT message to see if the HWND supports UI Au-
GETOBJECT message in the window’s Window tomation, and uses the returned provider to obtain
Procedure, as shown in Listing 2. more properties.

In response to WM_GETOBJECT, I create and re-
turn an instance of the provider to UI Automation
via the UiaReturnRawElement API from UIAuto-
mationCore.dll. Standard COM reference counting
rules apply: UiaReturnRawElement calls AddRef()
internally to claim its own reference; so I call Re-
lease() when I’m done with the WndProc’s refer-
ence.
Also, the code here creates a new instance of the
provider each time: this is allowable, as is creating a
single instance the first time and caching that value
for later reuse. Just remember to use the Release()
method on that reference in the WM_DESTROY
handler.
Note: Throughout this article, I refer to the Inspect
Objects tool, which is available in the Active Ac-
cessibility 2.0 SDK Tools download on the Micro-
soft Download Center (http://www.microsoft.com/
downloads/details.aspx?familyid=3755582a-a707-
460a-bf21-1373316e13f0&displaylang=en). This tool
helps you see what UI Automation sees. You can use
it to test and verify your provider implementations.
All this happens only on demand: it’s only when a
client requests an object for this specific HWND
that the WM_GETOBJECT message is sent. Get-
PropertyValue is called only when a client requests
a property.
Connecting a Provider to Real Control
Data
To make this example more realistic, I’m going to
modify the provider code to contain a reference to
the custom control’s own data so that it can return
real data from GetPropertyValue. Depending on
how your control works, this might involve a pub-
lic or internal COM object, window messages, or
direct access to the control’s data.
An important issue to address here is that of pro-
vider lifetime. UI Automation holds onto a refer-
ence to the control on behalf of clients, and clients
are not required to release their references at any
particular point in time. This means that a provider
can outlive its corresponding UI.

If I point the Inspect Objects tool at the custom con-
trol, it should report the Name and Control Type
that I’ve specified above. What’s happening here is
that Inspect Objects asks UI Automation for the ob-
ject at a specified point on the screen. UI Automa-
tion determines the HWND, sends a WM_GETOB-
One way to deal with this situation is to add an
“alive” flag to the provider and set it to a “dead”
state when the UI is no longer present. Further
incoming calls to the IRawElementProvider can
then check this flag and return the defined value
UIA_E_ELEMENTNOTAVAILABLE.

64 Writing a UI Automation Provider for a Win32-based Custom Control www.code-magazine.com

Listing 3 shows the new code that takes provider
lifetime into account.
Now if you point the Inspect Objects tool at the
control, you should see the control’s actual name
instead of the string.
Adding Structure by Adding
IRawElementProviderFragment
So far, I’ve simply exposed a single element. How-
ever, most custom controls have some internal
structure, whether items in a ListBox or a tree of el-
ements in a HTML-like control. The next step is to
expose that structure to UI Automation by adding
the IRawElementProviderFragment interface.
You need to consider what sort of class structure
to use to expose your control’s internal structure.
If you have a two-level homogenous container—
for example, a list that contains only one type of
list item—then it may make sense to have one class
for the overall parent list and another class for the
items. On the other hand, if the control has a more
generalized object tree—as is the case with HTML—
then it may make more sense to have a hierarchy
of provider objects that mirrors the UI’s own class
structure. It could have a base class that implements
common functionality and derived classes that han-
dle element-specific behavior.
Whatever structure you choose, you need to imple-
ment both IRawElementProviderSimple and
IRawElementProviderFragment interfaces for each
class that represents an element in the structure. As
you’ll see later, the root element also needs to imple-
ment IRawElementProviderFragmentRoot.
You only need to access the tree structure when a cli-
ent requests it. Furthermore, you don’t expose a tree
structure all at once; instead, you access or traverse
it element by element as needed.
Because internal structure is very control-specific,
I won’t provide concrete examples in this walk-
through. I’ll just outline what each method does.
Note that, as with IRawElementProviderSimple, all
the methods should check that the corresponding UI
element is “alive” first before they go on to do the
real work.
The Navigate method is perhaps the most obvious
method in this interface. It’s called with a specified
direction (Parent, Next or Previous sibling, or First
or Last child), and returns an instance of IRawEle-
mentProviderFragment that represents the element
in the specified direction. This object must also im-
plement IRawElementProviderSimple or else UI
Automation would be able to navigate to it but not
get any information about it:
IFACEMETHODIMP MyCustomControlProvider::
Navigate(NavigateDirection direction,
IRawElementProviderFragment ** pRetVal );
www.code-magazine.com
The returned IRawElementProviderFragment in-
stance can be a new instance of a provider or a
cached instance, so long as it obeys COM’s refer-
ence counting rules. Note that the object that this is
called on continues to point to the same UI element
that it was previously pointing to. Note on Microsoft®
Active Accessibility
The tree structure that is returned by this meth- Compatibility
od should be consistent: if you can navigate to a
child, then when you navigate back to the par- UI Automation automatically
makes some of the structure,
ent, you should end up at a provider instance
that represents the same UI element. If you cache properties and events of your
providers, this may be the same provider object custom control available to
instance. Microsoft Active Accessibility
clients. The information UI
The root provider of your structure, the one that oc- Automation translates is the
cupies the entire HWND, must return NULL when subset of data that Microsoft
asked for its parent or siblings. You should provide Active Accessibility supports.
just the subtree structure that corresponds to the UI
that your control is providing; UI Automation will WM_GETOBJECT requests
do the work of integrating this into the larger tree- both Microsoft Active
of-peer-HWNDs. Accessibility support and UI
Automation support from
Next, for the GetRuntimeId method, UI Automa- a HWND, with the lParam
tion requires that you return an integer array that indicating which type the
uniquely identifies this specific element within the sender is looking for. When
scope of the owning HWND: you pass the provider to
UiaReturnRawElement, it
IFACEMETHODIMP MyCustomControlProvider:: checks the lParam and wraps
GetRuntimeId (SAFEARRAY ** pRetVal); the provider in a UIA-to-MSAA
adaptor, which it then hands
If you already have a unique integer assigned to back to the client.
each UI element, using that would work perfectly
here as a single element array. Be cautious about This UIA-to-MSAA translation
using a pointer value because you have to take into wrapper also implements
account that pointers on 64-bit systems require two the IAccessibleEx interface,
integers. which allows Microsoft
Active Accessibility clients to
Once you have your array, prepend to it the value query for the UI Automation
UiaAppendRuntimeId and use the API’s SafeArray- properties and patterns that
CreateVector and SafeArrayPutElement functions are outside the scope of the
to convert the C-style array to a SAFEARRAY. Microsoft Active Accessibility
interfaces.
The root element of your control can simply return
NULL here: UI Automation automatically con-
structs Runtime IDs for the host HWND, and from
UI Automation’s point of view, the root element of
the control is essentially the same element as that
HWND.
The get_BoundingRectangle method returns the
bounding rectangle coordinates of the control.
For this, return the location of the UI element
in screen coordinates. If the element is not vis-
ible, return a rectangle consisting of all 0s. You
may need to use MapWindowPoints to convert
from HWND client-relative coordinates to screen
coordinates:
IFACEMETHODIMP MyCustomControlProvider::
get_BoundingRectangle(UiaRect * pRetVal);
As with GetRuntimeId, the root element doesn’t
have to do anything here, since its location is the
same as the host HWND, and UI Automation can
already get that information from the HWND.
Writing a UI Automation Provider for a Win32-based Custom Control 65
 You only use the GetEmbeddedFragmentRoots IFACEMETHODIMP MyCustomControlProvider::
method when your control hosts other HWNDs ElementProviderFromPoint(double x, double y,
within it. Since I’m not covering this issue yet, this IRawElementProviderFragment ** pRetVal);
method returns NULL here.
Note on Client-side IFACEMETHODIMP MyCustomControlProvider::
Providers Again, the root element doesn’t have to do any- GetFocus(
UI Automation also supports thing because its location is the same as the host IRawElementProviderFragment ** pRetVal);
client-side providers, which are HWND:
loaded by the client application These methods should return a provider instance
and often provide the default UI IFACEMETHODIMP MyCustomControlProvider:: representing the focused element or the element
Automation support for many GetEmbeddedFragmentRoots(SAFEARRAY** pRetVal); at the specified point (in screen coordinates), or
of the standard Win32 controls. NULL if the point or focus is on the root element
UI Automation calls the SetFocus method when it itself or if the point or focus is not on the ele-
Server-side and client-side wants you to set your control’s internal focus state ment at all. For both these methods, the provider
providers are very similar to a particular element. UI Automation automati- should return the deepest element possible.
in terms of general class cally focuses the parent HWND first, so you only
structure: they implement the need to update your control’s internal state: Finally, I’ll return to the implementation of
same set of IRawElement* IRawElementProviderSimple::get_HostRawEle-
interfaces and expose IFACEMETHODIMP MyCustomControlProvider:: mentProvider. You should modify this such that
properties and patterns in SetFocus(); it only returns the HWND for the root. For all
the same way. But there are child elements within the control, it should return
differences. The get_FragmentRoot method provides UI Au- NULL.
tomation with a convenient way to get to the root
While server-side providers element of your control without having to call You can take a piecemeal strategy for implement-
live in the same process as Navigate(Parent) repeatedly: ing all of these methods. First, start with imple-
the control itself; client-side menting the Navigate and BoundingRectangle
providers live in the client IFACEMETHODIMP MyCustomControlProvider:: methods. This enables you to get to the root el-
application’s address space. get_FragmentRoot( ement by hovering the cursor over any part of
Server-side controls can IRawElementProviderFragmentRoot ** pRetVal); the control, and then walking through the tree
obtain information from using the Navigation functionality of the Inspect
the underlying control by Next, you need to implement two structure-related Objects tool. Use the highlight rectangle feature
accessing the control’s internal methods only on the root node. These are on the to quickly verify that the locations are being ex-
data if necessary. Client-side IRawElementProviderFragmentRoot and allow UI posed correctly as you navigate. Then you can
controls, however, must rely on Automation to request the focused element or the implement ElementProviderFromPoint. When
APIs or window messages. element at a specific screen location: you hover the mouse over an element within the
control, Inspect Objects should go straight to that
element instead of stopping at the HWND. Final-
Listing 4: This method returns a pattern provider based on the patterns the control implements. ly, implement the GetRuntimeId and GetFocus
methods.
IFACEMETHODIMP MyCustomControlProvider::GetPatternProvider(
PATTERNID idPattern, IUnknown ** pRetVal )
{ Deciding What Part of the Tree Structure to
// Always start by setting output parameter to NULL Expose
*pRetVal = NULL;
When deciding what parts of the tree structure to
// Check that control is still alive expose, remember that UI Automation is only inter-
ested in elements that are important to an end user.
if(_pControl == NULL)
This includes any element that an end user would
{
perceive as being a separate control in its own right.
return UIA_E_ELEMENTNOTAVAILABLE; For example, a push-button and its text are typically
} not considered distinct elements from an end-user’s
point of view, even if they are represented as such
// Now check if it‘s a pattern that this element supports in a control’s internal structures. In such a case, you
if(idPattern == UIA_ExpandCollapsePattern && should expose just a single element representing the
entire button.
_pControl->CanExpand())
{
// Assuming here that this class also implements Exposing Control-specific Functionality
// IExpandCollapseProvider
Using Patterns
*pRetVal = this;
AddRef(); The next step in implementing a provider is to cus-
} tomize the provider with information specific to
the corresponding UI. I’ve already covered prop-
erties (such as Name and Control Type), so it’s
return S_OK;
time to turn to control patterns. A pattern is just an
} interface that contains both properties and meth-

66 Writing a UI Automation Provider for a Win32-based Custom Control www.code-magazine.com

ods related to a common theme. For example, the
SelectionItem pattern interface has properties like
get_IsSelected that return the current selected
state, and methods like Select that change the se-
lection.
Implementing a pattern is a matter of figuring out
which interfaces to implement, and then implement-
ing the interface in terms of the underlying control
data. The mechanics of exposing the interface are
similar to using QueryInterface, but instead using
the method IRawElementProviderSimple::GetPat
ternProvider. This method takes a parameter indi-
cating the pattern that is requested and returns an
IUnknown object that implements the correspond-
ing interface.
While it is often most convenient to implement the
pattern interface on the same provider object, this
is not required. You can implement the interface
on a separate object if desired, though this separate
object will need to adhere to the same warnings re-
garding lifetime that the original provider does.
Listing 4 shows a simple implementation of Get-
PatternProvider which implements an interface on
the same object as the provider.
It isn’t necessary to cast the returned object to the
interface corresponding to the requested pattern:
UI Automation calls QueryInterface on the re-
turned object as a separate step after this to get the
correct interface.
Ensure that manipulating the control using these
interfaces is consistent with manipulating the con-
trol using the mouse or keyboard. For example, if
selecting some element with a mouse or keyboard
causes some other element to change state, that
same change should occur when the element is
selected using the ISelectionItemProvider::Select
method.
[Name and Control Type]
are two of the more important
properties, as screen readers
typically read the name and type of
a control as focus changes…
Notifying UI Automation of UI Events
The last piece of UI Automation functionality that
you need to address is events. When control state
changes, you need to tell UI Automation so that it
can inform any interested clients. For example, if
the focus changes from one element to another, UI
Automation can inform a screen reader, which can
in turn query for the name of the element with fo-
cus.
www.code-magazine.com
There are three main categories of events in UI
Automation:
• Events associated with a property, such as
changing the Name or IsEnabled values.
• Events associated with an action, such as the
pressing of a button.
• Events associated with the creation of deletion
of elements, called structure change events.
The general technique for notifying UI Automa-
tion of an event is the same in all three cases: the
control creates an instance of a provider for the UI
element that corresponds to the event, and then
calls one of the UI Automation “Raise” APIs, pass-
ing in the ID of the event or property as appropri-
ate.
For example, to signal focus change, a control
would use code similar to the following, which as-
sumes that pFocusedItem is a reference to an item
that has just received focus:
// Use UiaClientsAreListening to avoid creating a
// provider if no clients are listening
if(UiaClientsAreListening())
{
IRawElementProviderSimple pProvider = new
MyCustomControlProvider(hwnd, pFocusedItem);
UiaRaiseAutomationEvent(pProvider,
UIA_FocusChangedEventId);
pProvider->Release();
}
If clients are listening, the UiaRaiseAutomation-
Event call will create a reference to the provider that
remains in place until the client is done with the ele-
ment.
Before the Inspect Objects tool can track focus in the
control, I need to implement the HasKeyboardFo-
cus property to return a VT_BOOL variant with
value VARIANT_TRUE when the element has focus.
When that’s done, Inspect Objects should be able to
track focus as you move keyboard focus around with-
in your control using tab or arrow keys.
NOTE: The UiaClientAreListening method returns
TRUE if a client is listening to any event from this
process. You can get more fine-grained information
by implementing the optional IRawElementPro-
viderAdviseEvents interface on the root element of
your control. UI Automation will then call the Advi-
seEventAdded and AdviseEventRemoved methods
on this interface as clients register and unregister for
specific events.
Deciding which Properties, Patterns, and
Events to Implement
So far, I’ve discussed the mechanics of exposing
the structure of a control and of exposing the prop-
Writing a UI Automation Provider for a Win32-based Custom Control 67
 Property Description
Name Identifies the control. This is often the text on the control or the label. The
name shouldn’t contain the type of the control. For example, “OK” rather
than “OK button.”
ControlType Specifies one of the defined UIA_XxxxControlTypeId values from
UIAutomationClient.h, like UIA_ButtonControlTypeId.
While ControlType is effectively an integer enumeration value,
LocalizedControlType is a human-readable string. UI Automation
automatically supplies the LocalizedControlType based on the ControlType
that you supply, so you don’t need to implement the LocalizedControlType
property.
IsEnabled Identifies whether the element is enabled or disabled.
HasKeyboardFocus Identifies whether the element has keyboard focus.
IsKeyboardFocusable Identifies whether the element can have keyboard focus.
AcceleratorKey and Identifies the accelerator key or access key for the control.
AccessKey
Table 1: Your custom control should support these properties for each of its elements.

erties, patterns, and events for elements of the con-

trol. But how do you decide what to expose? Much
of that depends on the purpose and functionality
of the control. However, there are a few properties
that almost all elements should expose, as listed
in Table 1.

UI Automation provides some other properties

automatically, such as ProcessId and NativeWin-
dowHandle. Even though these are defined in the
header file, you don’t need to implement support
for them.

When deciding what parts of the

tree structure to expose,
remember that UI Automation is
only interested in elements that are
important to an end user.

In determining which other properties and patterns

to support, a useful guide is the control type: many
control types have a set of associated properties
and patterns that are expected for controls of that
type. For example, buttons are expected to imple-
ment the Invoke pattern, and combos and lists are
expected to implement the Selection pattern.

You can find further information on these proper-

ties and patterns, as well as the entire Windows
Automation 3.0 API, in the upcoming Windows®
Software Development Kit for Windows 7.

Brendan McKeon

68 Writing a UI Automation Provider for a Win32-based Custom Control www.code-magazine.com

Have You Xiine It?

All You Can Read.

Digitally.
Experience the next generation
in digital reading FREE.
Go to www.xiine.com to read
CoDe Magazine and other
great publications.
ONLINE QUICK ID 0811122
Matthew Karr
Software Development
Engineer
Matthew Karr has been a
developer at Microsoft for 5
years, working on Automation
and Accessibility. He’s helped
ship UI Automation V1 and the
Magnification API with Vista,
and is currently working on the
COM UI Automation API for
Windows 7.
In his free time he enjoys
juggling, snowboarding,
science fiction, and video
games.
70 Creating UI Automation Client Applications
Creating UI Automation
Client Applications
Sometimes an application needs to interact with the user
interface (UI) of a second application. The first application might
be a test application that drives the UI of the target to run through some
automated tests. It might describe the UI out loud, as an aid to users that
are blind. It might be a speech application that allows users to give vocal
commands. In each of these cases, the application needs a way to inspect
and interact with the UI of the system and other running applications.
icrosoft® UI Automation is a tool for doing Automation enables programmatic access through
M just that. It provides an abstracted model
of the UI, and allows a client application
to both investigate and manipulate the UI of all
five components:
• The UI Automation tree facilitates navigation
running applications on the system. In this article, through the structure of the UI. The tree is
I explain how you can use UI Automation for au- built from the collection of window handles
tomated testing, and I walk through code samples (HWNDs).
that demonstrate the basic features of UI Automa- • Automation elements are individual com-
tion. ponents in the UI. These can often be more
granular than a window handle.
• Automation properties provide specific infor-
Using UI Automation for mation about UI elements.
Automated Testing • Control patterns define a particular aspect of a
control’s functionality. They can consist of in-
formation about properties, methods, events,
The goal of many automated test tools and scenar- and structures.
ios is the consistent and repeatable manipulation • Automation events provide event notifications
of the user interface. This manipulation can involve and information.
simple unit testing of specific controls, or complex
recording and playback of test scripts that iterate
through a series of generic actions on a group of Basics of a UI Automation Client
controls. Application
One complication that arises
from automated applications Fast Facts For the Windows® 7 operating
is the difficulty of synchroniz- system, UI Automation has a
ing a test with a dynamic tar- Microsoft UI Automation new COM client API for both
get. As an example, consider was available for unmanaged and managed cli-
a list box control—such as one both managed and ents. Unmanaged applications
contained in the Windows unmanaged applications can now use UI Automation
Task Manager—that displays from the beginning; without requiring a change
a list of currently running ap- in languages or loading the
however, the framework
plications. Since the items in common language runtime
interface was limited
the list box are dynamically (CLR).
updated outside the control for managed API
of the test application, consis- for its client applications. Perhaps the best way to un-
tently repeating the selection Now with new derstand UI Automation in
of a specific item in the list box introduction of COM general, and the COM client
is impossible. Similar problems based unmanaged client API, specifically, is to see them in ac-
arise when attempting to re- the UI Automation clients tion. Listing 1 demonstrates a
peat simple focus changes in a can also be managed complete program that gets the
UI that is outside the control of UI element at the current cur-
or unmanaged.
the test application. sor position, and then prints its
name and control type.
Programmatic access provides the ability to imitate,
through code, any interaction and experience ex- Next I’ll walk through the various pieces that make
posed by traditional mouse and keyboard input. UI up this example.
www.code-magazine.com
 Listing 1: The program gets the UI element at the current cursor position and prints its name and control type.
int _tmain(int argc, _TCHAR* argv[])
{ {
// Initialize COM and create the main Automation object
IUIAutomation *g_pAutomation;
CoInitialize(NULL);
HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL,
CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation),
(void**)&g_pAutomation);
if(FAILED(hr))
return (hr);
// Get the element under the cursor
// Use GetPhysicalCursorPos to interact properly with
// High DPI
POINT pt;
GetPhysicalCursorPos(&pt);
IUIAutomationElement *pAtMouse;
hr = g_pAutomation->ElementFromPoint(pt, &pAtMouse);
if(FAILED(hr))
return hr;
// Get the element‘s name and print it
BSTR name;
hr = pAtMouse->get_CurrentName(&name);
Creating the CUIAutomation Object
The heart of the new COM API is the IUIAuto-
mation interface, which lets clients get automa-
tion elements, register event handlers, create
various helper objects, and access other helper
methods.
To get started using UI Automation in your applica-
tion, do the following:
1. Include Uiautomation.h in your project head-
ers. This header will bring in the other headers
that define the API.
2. Declare a global pointer to an IUIAutomation
interface.
3. Initialize COM.
4. Create an instance of the CUIAutomation class
and retrieve the IUIAutomation interface in
your global pointer.
The following example function creates the object
instance and stores the retrieved interface address
in the global pointer g_pAutomation:
HRESULT InitializeUIAutomation()
{
CoInitialize(NULL);
HRESULT hr =
CoCreateInstance(__uuidof(CUIAutomation),
NULL, CLSCTX_INPROC_SERVER,
__uuidof(IUIAutomation),
(void**)&g_pAutomation);
return (hr);
} tain information about UI elements, usually controls.
www.code-magazine.com
if(SUCCEEDED(hr))
wprintf(L“Element‘s Name: %s \n“, name);
SysFreeString(name);
}
// Get the element‘s Control Type (in the current languange)
// and print it
BSTR controlType;
hr = pAtMouse->get_CurrentLocalizedControlType(&controlType);
if(SUCCEEDED(hr))
{
wprintf(L“Element‘s Control Type: %s \n“, controlType);
SysFreeString(controlType);
}
// Clean up our COM pointers
pAtMouse->Release();
g_pAutomation->Release();
CoUninitialize();
return 0;
}
Directly Obtaining UI Automation Elements UI Automation and
Threading
Once you have the UI Automation object, you can
discover the entire UI. The UI is modeled as a tree Because of the way UI
of automation elements (IUIAutomationElement Automation uses Windows
interface objects), where each element represents messages, conflicts can occur
a single piece of UI. The IUIAutomationElement when a client application
interface has methods relevant to all controls, such attempts to interact with its
as checking properties or setting focus. own UI on the UI thread. These
conflicts can lead to very slow
The root element of the UI Automation tree is the performance or even cause the
desktop. You can obtain this element by calling either application to stop responding.
the IUIAutomation::GetRootElement or IUIAutom
ation::GetRootElementBuildCache method. Each If you intend your client
method retrieves an IUIAutomationElement inter- application to interact with all
face pointer, from which you can search or navigate elements on the desktop—
the rest of tree, as described later in this article. including its own UI—you
should make all UI Automation
If you have screen coordinates—such as the cur- calls on a separate thread. This
sor position in this example—you can retrieve an recommendation also applies
IUIAutomationElement interface by calling the when your application locates
IUIAutomation::ElementFromPoint method. elements and uses control
patterns.
To retrieve an IUIAutomationElement interface
It is safe to make UI
from a window handle (HWND), call the IUIAuto
Automation calls within a UI
mation::ElementFromHandle method.
Automation event handler
because the event handler is
You can retrieve an IUIAutomationElement inter-
face that represents the focused control by calling
never called on a UI thread.
However, when subscribing to
the IUIAutomation::GetFocusedElement method.
events that may originate from
your client application’s UI, you
must make the call on a non-UI
UI Automation Properties for Clients
thread. Remove event handlers
Properties of IUIAutomationElement objects con- on the same thread.
Creating UI Automation Client Applications 71
 Listing 2: This method uses the RangeValue pattern to set a control to its maximum value.
HRESULT TurnItUp(IUIAutomationElement *pElement)
{ double max;
IUIAutomationRangeValuePattern * pRangeVal; hr = pRangeVal->get_CurrentMaximum(&max);
HRESULT hr = if(SUCCEEDED(hr))
pElement->GetCurrentPatternAs(UIA_RangeValuePatternId, {
__uuidof(IUIAutomationRangeValuePattern), hr = pRangeVal->SetValue(max);
(void **)&pRangeVal); }
if(FAILED(hr) || pRangeVal == NULL) pRangeVal->Release();
{ return hr;
return hr; }
}
The properties of an element are generic to all ele-
ments and not specific to a control type. Control pat-
terns, discussed later, expose control-specific proper-
ties.
UI Automation properties are read-only. To set prop-
erties of a control, you must use the methods of the
appropriate control pattern; for example, use the
IScrollProvider::Scroll method to change the posi-
tion values of a scrolling window.
To improve performance, you
can cache property values of
controls and control patterns
Programmatic access when you retrieve elements.
provides the ability to imitate,
Some generic properties, and
through code, any interaction and all control pattern properties,
experience exposed by traditional are available as properties on
the IUIAutomationElement
mouse and keyboard input. interface or control pattern
interface, and you can retrieve
them with accessor methods.
When using the generic accessors, you must spec-
ify properties with the property identifiers defined
in UIAutomationClient.h. You use them to specify
properties when retrieving property values, con-
structing property conditions, and subscribing to
property-changed events.
If a UI Automation provider does not implement a
property, UI Automation is able to supply a default
value for that property. For example, if the pro-
vider does not support the property identified by
UIA_IsDockPatternAvailablePropertyId, UI Auto-
mation returns FALSE.
Control Patterns
Control patterns complement properties. Control
patterns are collections of associated properties,
events, and methods that describe an element.
More than one pattern can apply to a single ele-
ment.
A control pattern represents a collected group
of capabilities. The pattern might be something
72 Creating UI Automation Client Applications
simple like the Invoke pattern, which lets clients
invoke a control. In contrast, the more compli-
cated Value pattern supports getting and setting
a control’s value, and then checking whether the
value is read-only.
You can obtain a control pattern by calling the
IUIAutomationElement::GetCurrentPattern or
IUIAutomationElement::GetCurrentPatternAs
method, or their cached versions. Once you get
a control pattern interface, use it as you would
the element itself, either by directly calling control
pattern methods or by accessing control pattern
properties.
The TurnItUp method in Listing 2 uses the Ran-
geValue pattern to set a control (such as a volume
slider) to its maximum value.
In addition to checking for errors when getting
the current pattern, this example also checks for
NULL because NULL is a valid return value when
a control does not support the pattern requested.
Control Types
While a control type is mechanically a simple enu-
meration property, on a conceptual level it is much
more important. An element’s control type is a
broad classification of the control, such as button,
window, or list box. Each control type has certain
expected control patterns. For example, a button
control should support either the Invoke pattern or
the Toggle pattern, and a hyperlink should support
the Invoke Pattern and possibly the Value pattern.
Some controls have conditional support for several
control patterns. For example, the menu item con-
trol has conditional support for the Invoke, Expand
Collapse, Toggle, and SelectionItem control pat-
terns, depending on the menu item control’s func-
tion within the menu control.
Searching and Navigation
There are two major ways of getting to elements.
The first is to use a Find method, which lets the UI
www.code-magazine.com
 Listing 3: The FindFirst method here finds a particular application window.
IUIAutomationElement* GetTopLevelWindowByName(LPWSTR windowName)
{ if (pRoot)
if (!windowName)
{ IUIAutomationCondition* pCondition;
return NULL;
} NamePropertyId,
IUIAutomationElement* pRoot;
IUIAutomationElement* pFound;
VARIANT varProp;
varProp.vt = VT_BSTR;
varProp.bstrVal = SysAllocString(windowName);
// Get the desktop element
HRESULT hr = g_pAutomation->GetRootElement(&pRoot);
Automation core minimize cross-process calls and
improve searching time. The second is to use a tree
walker to walk a UI tree.
Using Find Methods
The Find methods require three things: a parent ele-
ment, a scope to search on, and (most importantly)
a condition to search for.
You create conditions by calling various APIs on
the CUIAutomation object. You can create simple
Property conditions, or combine them with And,
Or, and Not operators to produce more complex
conditions.
Once you create a condition, you need to specify
the element to start searching from and the search
scope. The element you want to start from is the
element on which you call the FindFirst or Fin-
dAll method. The search scope is typically one of
three choices: the element itself, its children, or its
descendants.
If a UI Automation
provider does not implement
a property, UI Automation
is able to supply a default value
for that property.
You usually want to limit search scope as much as
possible to improve performance and to reduce the
chance of finding an element you weren’t looking
for.
Listing 3 is a simple example of searching for a
named application window using FindFirst meth-
ods. In the example, the search condition (a par-
www.code-magazine.com
// Get a top-level element by name, such as „Program Manager“
{
g_pAutomation->CreatePropertyCondition(UIA_
varProp, &pCondition);
pRoot->FindFirst(TreeScope_Children, pCondition, &pFound);
pRoot->Release();
pCondition->Release();
}
VariantClear(&varProp);
return pFound;
}
ticular name) is quite simple. However, conditions
can be considerably more complex.
Using Tree Walkers
Another method for navigating through the tree is the
use of tree walkers. Tree walkers allow use of direct-
navigation methods such as moving between parent,
child, and sibling to walk through a filtered view of
the tree. There are several built-in filtered views:
• Raw or unfiltered, which shows all elements.
• Control view (the default), which filters out
elements that either are redundant or are just
used for layout.
• Content view, which filters controls even more
selectively than Control view does.
Users can also create their own custom views with
Conditions.
The following simple example shows how to walk
to the first child of the control referenced by pEle-
ment:
// Get the control view walker
IUIAutomationTreeWalker * pWalk;
g_pAutomation->get_ControlViewWalker(&pWalk);
// Go to the element‘s first child
IUIAutomationElement * pFirst;
pWalk->GetFirstChildElement(pElement, &pFirst);
UI Automation Events for Clients
UI Automation lets clients subscribe to events of
interest. This capability improves performance by
eliminating the need to continually poll all UI ele-
ments in the system to see if any information, struc-
ture, or state has changed.
Efficiency is also improved by the ability to listen
for events only within a defined scope. For example,
Creating UI Automation Client Applications 73
 a client can listen for focus change events on all UI
Automation elements in the tree, or on just one ele-
ment and its descendants.
Client applications subscribe to events of a particu-
lar kind by using methods such as AddAutomation-
EventHandler or AddFocusChangedEventHandler
to register an event handler These methods take a
callback interface, which should be implemented in
an object created by the client. A method such as
IUIAutomationEventHandler::HandleEvent will
then be called on the appropriate thread whenever
the event occurs.
On shutdown, or when UI Automation events are
no longer of interest to the application, UI Auto-
mation clients should remove the events. They do
so either by calling a removal method for specific
events such as RemoveAutomationEventHandler
or RemoveFocusChangedEventHandler, or by
calling the RemoveAllEventHandlers method to
remove all events.
UI Automation and Screen Scaling
Starting with Windows Vista®, users can change
the screen resolution (in dots per inch or dpi) so
that most UI elements appear larger. Previously,
the scaling had to be implemented by applications.
Now, the Desktop Window Manager performs de-
fault scaling for all applications that do not handle
their own scaling. UI Automation client applica-
tions must take this feature into account.
The UI Automation API does not use logical coor-
dinates. Methods and properties either return phys-
ical coordinates or take them as parameters.
By default, UI Automation applications that run at
a screen resolution other than 96 dpi will get incor-
rect results from these methods and properties. For
example, because the cursor position is in logical co-
ordinates, the client cannot simply pass these coor-
dinates to the IUIAutomation::ElementFromPoint
method to get the element that is under the cursor.
In addition, the application will not be able to cor-
rectly place windows outside its client area.
The solution to this situation has two parts:
• Make your client application aware of screen
resolution by calling the Win32 function Set-
ProcessDPIAware at startup. This function
makes the entire process aware of screen reso-
lution, so that no windows belonging to the
process are scaled.
• Call GetPhysicalCursorPos to get the current
cursor coordinates.
User Account Control and User Rights
Finally, you need to consider User Account Con-
trol and user rights to ensure your client can access
the information it needs. Before a user launches
74 Creating UI Automation Client Applications
The properties of an element
are generic to all elements and not
specific to a control type.
Control patterns,
discussed later, expose
control-specific properties.
an administrative application the operating system
prompts the user for consent to run the application
with elevated user rights. Similarly, when a user at-
tempts to perform a task that requires administra-
tive user rights, Windows presents a dialog box that
asks the user for consent to continue. This dialog
box is protected from cross-process communica-
tion, so that malicious software cannot simulate
user input.
Only signed client applications are trusted to com-
municate with applications running at a higher us-
er-rights level. To gain access to the protected sys-
tem UI, applications must be built with a manifest
file that includes a special attribute. This UIAccess
attribute is included in the <requestedExecution-
Level> tag where the value of the level attribute is
an example only, as follows:
<trustInfo xmlns=
„urn:0073chemas-microsoft-com:asm.v3“>
<security>
<requestedPrivileges>
<requestedExecutionLevel
level=“highestAvailable“
UIAccess=“true“ />
</requestedPrivileges>
</security>
</trustInfo>
The UIAccess attribute value is “false” by default;
that is, if the attribute is omitted, or if there is no
manifest for the assembly, the application will not
be able to gain access to protected UI.
With access to the system and target application’s
UI, your client applications can navigate the UI,
retrieve properties and control patterns for UI ele-
ments, and listen to relevant UI events.
Matthew Karr
www.code-magazine.com