BEST PRAC TICES

Client Choice topic

August 10, 2006

Death By A Thousand Cuts Kills Web Experience
When A Large Number Of Flaws Add Up To A Major Problem
by Moira Dorsey
with Harley Manning and Caroline L. Carney

EXECUT I V E S U M MA RY
When asked about the most important attributes of their favorite sites, online consumers say that ease
of finding content and function and legibility are more important than superior content and function.
But high-quality navigation and legibility are a rarity among the last 211 sites that Forrester evaluated.
And when sites have flaws that hinder a user’s ability to find, read and trust content, it adds up to a major
problem — even when each flaw is minor on its own. To avoid delivering site experiences that fall victim
to the cumulative effects of many minor problems, site managers should evaluate the most important
users’ paths, focus on fixing problems with well-known solutions first, and put a business-centric design
process in place to avoid building flawed sites in the first place.

TARGET AUDIENCE
Customer experience professional

RESEARCH CATALYST
Clients selected this topic for Client Choice research.

WEB USERS VALUE EASE OF FINDING AND READING CONTENT OVER CONTENT QUALITY
Forrester recently asked members of our North American Consumer Technology Adoption Study
(NACTAS) panel to tell us what they like best about the site they visit most. Interestingly, superior
content and function were not at the top of our panelists’ list (see Figure 1). Instead, they rated ease of
finding content and function as the most important attribute of their favorite sites. Text legibility came in
a close second, while trust in a site’s security and reliability ended up in third place.

But Sites Throw Up A Series Of Barriers To Finding, Reading, And Trusting Content
Online consumers value clear navigation, strong text legibility, and high reliability — but that’s not what
they usually get. Many sites that we evaluated failed tests of how well they help users find, read, and trust
content (see Figure 2 and see Figure 3). And when sites serve up a series of flaws in a single scenario, it
adds up to a major headache for users (see Figure 4). A recent review of Merrill Lynch’s site for personal
investors illustrates this cumulative effect.1

Headquarters
Forrester Research, Inc., 400 Technology Square, Cambridge, MA 02139 USA
Tel: +1 617/613-6000 • Fax: +1 617/613-5000 • www.forrester.com
Best Practices | Death By A Thousand Cuts Kills Web Experience 2

· Unclear menus slow and confuse site visitors. Starting at the home page, Merrill Lynch forces
users to puzzle over ambiguous menu categories like “Bullseye” and “Total Merrill.” The home
page does expose subcategory choices for “Total Merrill,” shedding light on what users will
find there. However, it provides no help for prospects who want to know what they’ll find in
other, equally unclear categories like “Beyond Banking.” This means site visitors must pause to
consider their uncertain options and then make multiple attempts before they find the right
path to the information they need.

· Small text strains users’ eyes throughout. Much of the essential content on the site appears in
very small text — 8 point Arial. Often, the type sits on colored backgrounds that reduce contrast,
making it even harder to read — especially on the retirement menu, where some of the most
important content appears in a salmon-colored box. This means that investors must squint at
page after page of text as they conduct the research needed to make complex buying decisions.
While the eyestrain that this causes may seem minor at first, it causes fatigue over time, turning
the buying process into an endurance test.

· Hidden links to privacy and security policies raise doubts. When prospects click “Have us
contact you,” they’re asked to fill out a form that requires their name, address, home phone
number, and email address. Would-be customers with concerns about privacy and security
won’t see a reassuring link to the site’s “Privacy Statement,” which is buried in a footer that
appears well below the “fold” of the page. Some prospects will request a contact anyway, some
will go look for the policy, and some will simply abandon the form without clicking submit.

August 10, 2006 © 2006, Forrester Research, Inc. Reproduction Prohibited

Best Practices | Death By A Thousand Cuts Kills Web Experience 3

Figure 1 Online Consumers Prioritize Ease Of Finding Content And Function Over Superior Quality

“Thinking about your most frequently visited Web site, please indicate what you like about the site?”

It’s easy to find the content and 55%

function I want

It’s easy to read 49%

I trust the site to be secure and 39%

perform reliably

The site is fast and responsive 38%

It’s a fun site 30%

Compared with other sites, it has 30%

better content
I like how the site looks 26%
(appearance) Intuitive navigation and
Compared with other sites, it has easy-to-read text provide
20%
better function more of a competitive
The site personalizes information edge than better content
16%
I see when I visit and function.
Other (please specify) 4%

Base: Consumers who go online once a month or more

(multiple responses accepted)

Source: Forrester’s NACTAS Q2 2006 Automotive, Customer Experience, And Government Online Survey
40127 Source: Forrester Research, Inc.

August 10, 2006 © 2006, Forrester Research, Inc. Reproduction Prohibited

Best Practices | Death By A Thousand Cuts Kills Web Experience
Figure 2 Failure Rates For Forrester’s Tests Related To Finding Content
Tests related to ease of
finding content
A) Are category and subcategory
names clear and mutually exclusive?
B) Do menu categories immediately
expose or describe their
subcategories?
C) Are items classified logically?
D) Are hyperlinks clear and informative?
E) Is the task flow efficient?
F) Are keyword-based searches
comprehensive and precise?
40127
August 10, 2006
4
Failure What Forrester’s Web Site Review looks for
rate
59% • Clearly labeled category and subcategory names at all
levels of the site.
• Labels set users’ expectations about what they’ll find.
• Labels don’t overlap in meaning with other categories
or subcategories.
• Best practice: Category and subcategory names include
trigger words related to user goals.
67% • The home page displays links to — or descriptions of —
content and function within major categories.
• Subcategories are not hidden within roll-over menus.
• Best practice: Second-level pages immediately display
contextual subcategories.
38% • Items are listed in categories and subcategories where
users would logically expect to find them.
• Items are not wildly cross-listed in an effort to compensate
for poor information architecture.
• Best practice: Alternative, logical categorization schemes
enable users to sort or narrow content.
37% • Wording on navigation elements other than menu
categories is clear and sets user expectations.
• Best Practice: Wording on hyperlinks and buttons is highly
descriptive and free of redundant words.
70% • Users don’t have to take extra steps to accomplish
their goals.
• Users don’t have to navigate to lower levels of the site
when content could have appeared at a higher level.
• Users don’t have to repeat steps, such as entering
information they’ve already provided.
• Best practice: Allow users to control the steps in the
process.
58% • Keyword search engine retrieves essential information
and puts it in order of relevance.
• Search results page displays user’s search term.
• Best practice: Search engine compensates for common
misspellings, accommodates synonyms, and enables users
to sort or narrow search results by meaningful criteria.
Source: Forrester Research, Inc.
© 2006, Forrester Research, Inc. Reproduction Prohibited
Best Practices | Death By A Thousand Cuts Kills Web Experience
Figure 3 Failure Rates For Forrester’s Tests Related To Reading And Trusting Content
Tests related to ease of
reading content
G) Is text legible?
H) Does text formatting and layout
support easy scanning?
I) Do pages use space effectively?
Tests related to building trust
J) Does the site present privacy and
security policies in context?
K) Does the site perform well?
40127
August 10, 2006
5
Failure
rate What Forrester’s Web Site Review looks for
79% • Text used for essential content appears to be at least
9 point for Verdana, at least 10 point for Arial, and at least
11 point for Times New Roman.
• Text color for essential content has strong contrast with
the background.
• Best practice: Text legibility for all content — even content
that is not essential for user goals — is large and well-
contrasted with the backgound.
39% • Site facilitates scanning by appling formatting practices
such as font choice, style, and color, and layout practices
such as text spacing, indentation, justification, and bullets.
• Best practice: Formatting and layout focus the user’s
attention on the most relevant content.
62% • Layouts display and prioritize as much content and
function as possible — avoiding large graphics or blocks
of white space that don’t add value to the display but do
displace valuable content and function.
• Best practice: All page layouts make good use of space.
82% • Clearly labeled links to privacy and security policies next
to fields that ask for personal information.
• Both privacy and security safeguards are described in the
policies.
• Best practice: Links include a summary of each policy with
the links.
42% • The experience is free of any major system errors (those
that prevent users from accomplishing a goal) such as the
site being down or not accepting a user’s order.
• The experience has no more than one or two minor errors
(those that don’t prevent users from accomplishing a goal)
such as missing graphics.
Source: Forrester Research, Inc.
© 2006, Forrester Research, Inc. Reproduction Prohibited
Best Practices | Death By A Thousand Cuts Kills Web Experience 6

Figure 4 Multiple Flaws In A Single Scenario Take A Cumulative Toll

Home page Category pages Search Product page Shopping cart Checkout

A) Categories C) Items F) Search fails to G) Tiny text J) No obvious K) Page not

unclear misclassified find relevant H) Text hard to security policy found
B) Subcategories D) Hyperlinks result scan
hidden unclear I) Wasted space
E) Extra clicks

“I have no idea “Where is the “No match “Why is this so “How will you “Transaction
where to start.” information I found?!” hard to read?” protect my failed — I give
need?” identity?” up!”

40127 Source: Forrester Research, Inc.

R E C O M M E N D AT I O N S

PREVENT FLAWS THAT ADD UP TO A MAJOR IMPACT

To avoid the cumulative, negative impact that a series of flaws can have on users’ Web interactions,
site managers should:

· Look for scenarios with multiple flaws. Expert reviews are a quick, cost-effective way to
identify well-known usability flaws. They enable site managers to identify major and minor
flaws by testing user scenarios against a set of research-based criteria like those in Forrester’s
Web Site Review methodology.2 Site managers should start by testing user scenarios that
are most important to driving business results. To help zero in on scenarios with the greatest
concentration of flaws, site managers should leverage data from other evaluation tools, like
Web analytics software.3 For example, TeaLeaf Technology can track users’ actions click by
click within a page — enabling analysts to look for common patterns of activity among users
who abandoned a given page.
· Fix the problems with well-known solutions first. With expert evaluation results in hand,
site managers should sort the list of identified flaws into three categories: those that have
known solutions, those that require application of a known methodology to find a solution,
and those that require innovation.4 By prioritizing fixes to items in the “known solutions”
category, site managers can make quick improvements — with high probability of success.
And many of the problems that fall into the “known solutions” category, such as text legibility
and formatting, remain unresolved by a majority of sites. As a result, just fixing these basic
problems differentiates against the competition.

August 10, 2006 © 2006, Forrester Research, Inc. Reproduction Prohibited

Best Practices | Death By A Thousand Cuts Kills Web Experience 7

· Apply standards within a consistent process to avoid flawed designs. Improving Web
experiences requires a business-centric design process that starts with setting business goals,
incorporates the appropriate design skills and methodologies, and ends with measuring
business results.5 One piece of this successful framework: a set of standards that designs
must comply with. For example, site owners can establish detailed specs for clear menus and
legible text, which helps ensure that confusion and eyestrain aren’t built into designs — and
don’t creep into redesigns.6 To help keep its firm’s design principles and process top of mind,
Fidelity’s online user interface design group displays them on large posters in their “design
labs” where teams meet throughout the course of a project.

ENDNOTES
1
Forrester recently examined how well full-service brokerage sites enable prospects to find information
on retirement planning offerings. Our reviews revealed flaws on every site that we evaluated, but those
uncovered on the Merrill Lynch site illustrate the effect of multiple problems on a single scenario. See the
May 16, 2006, Best Practices “Make The Most Of Your Financial Services Brand On The Web.”
2
Web Site Reviews help firms improve business results because they uncover flaws that get in the way of
users accomplishing the goals most important to driving business goals like shifting interactions to lower-
cost channels or increasing revenue with higher conversion rates. See the May 6, 2005, Best Practices
“Executive Q&A: Web Site Reviews.”
3
Evaluation tools such as focus groups, Web analytics, surveys, and A/B tests vary in terms of the
information they provide, what they cost, and how well they provide information on user scenarios that
span multiple channels. Getting a holistic view of whether users can accomplish their goals and the quality
of the experience requires using the data from these tools in conjunction with each other. See the June 2,
2006, Best Practices “Companies Need A Portfolio Of Tools To Evaluate Customer Experience.”
4
Design flaws come in three flavors: solved, known methodologies for solving, and those that require
innovation to discover the solution. See the February 17, 2006, Best Practices “Beating The Three Web
Design Challenges.”
5
In addition to having a business-centric design process that aligns business goals with customer goals,
having a design-centric culture that understands the business value of customer experience is another
critical factor in enabling firms to improve the customer experience that they deliver. See the March 31,
2006, Best Practices “Culture And Process Drive Better Customer Experiences.”
6
For example, the 25 heuristics in Forrester’s Web Site Review methodology are practices that can not
only be used to evaluate usability throughout the design process, but they can also be used as a set of
fundamental principles that designs should comply with. See the March 11, 2005, Best Practices “Best And
Worst Of Site Design, 2005.”

Forrester Research (Nasdaq: FORR) is an independent technology and market research company that provides pragmatic and forward-thinking advice about
technology’s impact on business and consumers. For 22 years, Forrester has been a thought leader and trusted advisor, helping global clients lead in their markets
through its research, consulting, events, and peer-to-peer executive programs. For more information, visit www.forrester.com.
© 2006, Forrester Research, Inc. All rights reserved. Forrester, Forrester Wave, Forrester’s Ultimate Consumer Panel, WholeView 2, Technographics, and Total
Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Forrester clients may make one
attributed copy or slide of each figure contained herein. Additional reproduction is strictly prohibited. For additional reproduction rights and usage information,
go to www.forrester.com. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. To purchase reprints
of this document, please email resourcecenter@forrester.com. 40127