Cover

Be sure to:

Update META data

Update logo and logo link on A Master

Update Date and Product on B Master

Web Performance Monitoring 101

Contents
Preface...................................................................................................3

Complex Spreadsheets Demand Rigorous Peer Review....................... 3

The Business of Web Performance........................................................5

So, What is WPM, Exactly?....................................................................7

The Importance of the User Experience.................................................8

The Global Performance Gap................................................................9

Web Performance and Security............................................................ 11

Implementation Considerations............................................................ 11
Benchmarking.................................................................................................. 12
3rd Party Content............................................................................................. 14
Looking at Internal Performance...................................................................... 14

Conclusion............................................................................................15

About AlertSite......................................................................................16

AlertSite 2
by SMARTBEAR
Preface
Web performance issues start like classic murder mysteries — you see
things are afoul, but you can’t figure out why.

It can be frustrating for an organization, which often spends more time

on isolating the cause of a performance issue than on actually repairing
the problem. It’s equally frustrating for end users, who are most likely
customers trying to conduct business with the organization.

Web Performance Monitoring (WPM) is about more than simply under-

standing whether a site and its applications are functioning normally. It’s
about having the ability to process, correlate, and normalize the data
needed to isolate the problem so you can optimize the user experience.

Because identifying performance issues is the primary objective, WPM

solutions are deployed by developers as often as IT operations. In
fact, the importance of having full visibility and control into applica-
tion performance impacts nearly all major IT and business goals, and
WPM is appearing as a part of nearly all major IT initiatives, from cloud
migrations to mobile strategies — a testament to its overall impact on an
organization’s revenue.

Complex Spreadsheets Demand Rigorous Peer Review

In a widely quoted study of e-commerce sites, the Aberdeen Group
found that a one-second delay in page response time results in a 7%
reduction in online customer conversion, 11% fewer page views and a
16% decrease in customer satisfaction. In monetary terms, that means
if your site earns somewhere around $100,000 each day, you could
be losing nearly $7 million annually due to a seemingly modest lag in
performance. 1

1 The Performance of Web Applications: Customers are Won or Lost in One Second,

Aberdeen Group, November 30, 2008

AlertSite 3
by SMARTBEAR
While you might be quick to dis- ...a one-second delay in page
miss this data on the grounds that response time results in a 7%
users are being too persnickety, reduction in online customer
recent science shows that they conversion, 11% fewer page
can’t help it. In 2012, researchers views and a 16% decrease in
at Glasgow’s Caledonian Univer- customer satisfaction.
sity strapped EEG caps on users to
monitor their brain waves and aimed EOG devices at them to track eye
and facial movements while they shopped online.

The participants were separated into two groups. One had a constantly
throttled 5 Mbps connection; the other a paltry 2 Mpbs connection.
Users with the slower connection had to concentrate 50% more to
complete the same tasks and showed significantly greater agitation and
stress.

In media and e-commerce companies, the impact of performance on

revenue can be huge — product pages slow to render, and video and
audio files that fail to play are a surefire way to lose visitors, and in turn
revenue. In the SaaS industry, the impact can be even greater. World-
wide SaaS revenue is forecast to reach $14.5 billion in 2012 – up 17.9%
from 2011 – with long-term projections topping $22 billion by 2015. 2

In the highly competitive SaaS world, the importance of lowering your

Customer Acquisition Cost is second only to customer retention. Profit-
able SaaS companies don’t begin to make money on a new subscriber
until 18 months into the relationship on average. Driving down that
number and keeping customers longer is therefore key — and both are
directly dependent on Web performance.

Internal testing of client SaaS applications accelerated by site optimiza-

tion company Strangeloop shows that a 50% increase in response time
2 “Gartner Says Worldwide Software-as-a-Service Revenue to Reach $14.5 Billion in 2012,”

Press Release, Gartner.com, March 27, 2012

AlertSite 4
by SMARTBEAR
boosts conversions rates by 5% and page views by 26% while decreas-
ing bounce rates by up to 14%.

SaaS customers have very little invested in their chosen solution be-
cause the costs associated with switching providers are so low. Increas-
ingly, the battle for a competitive advantage among SaaS providers is
being fought and won over better performance.

The Business of Web Performance

As of this writing in October of 2013, Mark Zuckerberg continues to

delay the launch of online video ads on Facebook. Never mind that
the holdup is costing Facebook $2 million a day; Zuckerberg is worried
about performance.

Issues like load times and ...Mark Zuckerberg continues to

streaming interruptions are delay the launch of online video ads
all the more challenging on Facebook. Never mind that the
when your user base of 1.1 holdup is costing Facebook $2 million
billion people is access- a day; Zuckerberg is worried about
ing the site via all manner performance.
of devices and connection
speeds. Facebook engineers have toiled over how to make the ads fast
enough so as not to alienate users and, ironically, cause a social media
disaster — especially during launch. 3

The delay comes despite the fact that some potential Facebook adver-
tisers created their video ads in anticipation of a summer 2013 rollout.
They have had to make alternative marketing plans for time-sensitive
products. Nevertheless, Zuckerberg would rather disappoint the adver-
tisers than his all-important user base.

3 Facebook Moves Cautiously on Video Ads, Wall Street Journal, August 8, 2013

AlertSite 5
by SMARTBEAR
Zuckerberg is likely familiar with some of the more spectacular web per-
formance disasters, as many were front-page news around the world. In
April 2011, the Olympic Games in London made 6.6 million tickets avail-
able online — all at once. Late on the last day to purchase their website
was slowed to a crawl under heavy load conditions, and the deadline
to purchase had to be extended by several days to prevent the games
from being stuck with unsold tickets.

Another famous Web performance SNAFU occurred when Dr Pepper

promised everyone in the US a free can of their soda if Guns n’ Roses
released the “Chinese Democracy” album any time in 2008. Guns n’
Roses released the album in 2008. There are 300 million people in the
United States. You can imagine how that played out.

Dr. Pepper’s website was flooded with customers, who had 24 hours to
fill out a voucher for a free soda on Nov. 23, the day “Chinese Democra-
cy” was released. When Dr. Pepper’s site failed to respond to the traffic,
the company extended the offer for another day.

Guns N’ Roses lawyer Alan Gutman summed it up in an L.A. Times

blog post, “What happened on November 23 was a complete fiasco. In
what could only be characterized as reckless indifference or complete
stupidity, Dr. Pepper was completely unprepared for the traffic to its
site. Most visitors were greeted with error messages. Some people who
got through to Dr. Pepper’s servers were told to call a toll free number,
few of whom got through. Many walked away angry at Dr. Pepper and
soured on release of ‘Chinese Democracy’.” 4

4 Pop & Hiss, The LA Times Music Blog, November 26, 2008

AlertSite 6
by SMARTBEAR
So, What is WPM, Exactly?

The importance of Web performance monitoring really can’t be under-

stated — but what does I actually entail? WPM is not a single technol-
ogy, but rather a range of technologies used to monitor the successful
delivery of applications to end-users — with most offerings providing
basic services consisting of:

¿¿ Basic Availability Monitoring: Websites have advanced into

complex systems, however, knowing when your site is up or
down and monitoring and managing the performance of basic IP
services like HTTP and HTTPS, DNS resolution, email protocols
and TCP connect remains crucial — WPM does this.

¿¿ Speed and Performance Monitoring: Whether a result of internal

issues or 3rd party content every second of a slowdown or
outage hurts revenue and damages the user experience —
WPM measures and benchmarks response times, page load
times and user experience metrics, and the factors contributing
to the results, to allow you to identify the source of performance
bottlenecks and address them before your users are affected.

¿¿ Transaction Monitoring: Every website has at least one

business-critical web transaction that directly affects revenue
— whether it’s an account log-in or shopping cart checkout or
something as simple as an information request form. WPM
solutions enable you to keep a watchful eye on the performance
your organization’s most important transactions.

¿¿ Mobile Monitoring: The world has become increasingly mobile-

centric — WPM solutions provide insight into the performance
of mobile websites and ensures they render correctly across
today’s litany of mobile devices.

¿¿ Web Services Monitoring: At this level WPM is focused on the

performance of single and multi-step calls to Web services built
on SOAP and REST. WPM solutions will identify things like

AlertSite 7
by SMARTBEAR
 network connectivity errors, socket-based errors, http protocol
errors, SSL errors, and authentication errors. At the same time
End Point Availability, Remote Method Availability, Remote
Method Response Time, and Remote Method Returned Data
are monitored to ensure the accuracy of the values being
returned over Web Services.

¿¿ Alerting: WPM not only monitors your websites and applications,

but also alerts you in the event of a slowdown or outage so
that fixes are made ASAP. Advanced solutions allow you to
set performance thresholds – so you are alerted whenever
performance slips below what you define as acceptable —
enabling you to address performance concerns before they
manifest on the site.

The Importance of the User Experience

The basics of WPM have been laid out above, but how do these metrics
add up to what your users are actually experiencing? After all, who
cares that your site is available and backend systems are performing
correctly if users abandon a task because they feel that the application
is too slow? And so what if a Web page loads in less than 2 seconds
if it delivers the wrong content to the visitor? The user’s experience is
quickly becoming the only metric that matters. So, the question for WPM
today has evolved into “How do we monitor and measure user experi-
ence so that it can be improved?”

Because application design makes extensive use of Web Services, the

ability to monitor the processing that takes place within the browser has
become critical. While a number alone may indicate a problem, it can’t
tell the story of what the user is actually experiencing. We need to see
and capture the performance of each step in the navigation process.
This begins when the page’s first pixels become visible in the browser,

AlertSite 8
by SMARTBEAR
continues when all content above Because application design
the fold has been displayed and makes extensive use of Web Ser-
extends through to the end of vices, the ability to monitor the
a user’s transaction — whether processing that takes place within
across multiple pages or a single the browser has become critical.
page visit. WPM should capture
the playback of the entire experience from the user’s point of view —
and provide the ability to measure and monitor the factors contributing
to the performance at each stage. In the end, you need to be able to
effectively optimize your site and application performance in a manner
that directly improves your users’ perception of performance — WPM
needs to provide the toolset to do so.

If you go one step further, and merge Google Analytics data with your
WPM metrics, you can produce incredible insight into how changes in
performance are affecting visitor behavior. The WPM story suddenly
becomes a story of business impact on your brand and your bottom line.
And the answer to where to invest your time and dollars on optimization
is even clearer.

The Global Performance Gap

As noted above, Web performance is a matter of individual perception,

and that perception depends entirely on where you sit.

For all the performance challenges experienced in the U.S., they are
likely exacerbated overseas.

The global average connection speed in the first quarter of 2013 was
3.1 Mbps. That’s less than half of the 8.6 Mbps enjoyed in the U.S. and
it includes developed countries like Italy, Spain and Portugal. In fact,
the U.S. is faster on average than every country except for Sweden, the

AlertSite 9
by SMARTBEAR
Netherlands, Switzerland and South Korea. 5

This is “on average” because there is wide disparity within the U.S. If
you’re online in Virginia or Vermont, for example, your connection speed
is likely twice that of someone in Arkansas or Alaska. But connection
speeds are getting faster, right?

Yes, but not at the same rate everywhere. In parts of Europe and Asia,
the gap is widening. And while connection speeds are getting faster,
Web pages are getting bigger. The average homepage today contains
98 page objects, a 13% increase from last year. 6

It’s crucial for organizations conducting international business to mea-

sure and monitor site performance from within the countries and regions
their customers are located — a site with acceptable performance in the
US may require considerable optimization to meet performance expec-
tations internationally.

AlertSite offers over 80 global monitoring stations

5 State of the Internet Report: Q1 2013, Akamai

6 State of the Internet Report: Q1 2013, Akamai

AlertSite 10
by SMARTBEAR
Web Performance and Security

We’ve talked about performance monitoring’s impact on page-views,

conversions, customer satisfaction, retention and revenue in general —
but what about when the game is all or nothing?

Just recently the Syrian Electronic Army hacked the DNS settings for
both the New York Times and Twitter. The hacks were virtually identi-
cal — phishing expeditions led to hacked email accounts where server
passwords were exposed — but the results were very different.

Both hacks happened on a Tuesday. Twitter quickly restored service,

but by Thursday afternoon, Times users were still having trouble ac-
cessing the site. According to InformationWeek, the difference was that
“Twitter actively monitors its DNS settings and thus learned of the hack
very quickly.” 7

Comprehensive Web performance monitoring often plays an impor-

tant role in identifying performance anomalies resulting from a security
breach.

Implementation Considerations

The keys to successfully implementing a strong Web performance moni-

toring solution revolve around detecting relevant events early — within a
comprehensive picture.

At its most basic, your implementation needs to ensure your site is avail-
able, responding well and that business critical Web transactions are
available and performing acceptably — and that you are made aware of
any aberration before your visitors experience it.

As we discussed above, performance threshold notifications can give

you a “head’s up” before your site falls below what you consider to be
acceptable performance levels — but what is acceptable? It depends

7 Lessons Learned From N.Y. Times Hack Attack, InformationWeek, August 29, 2013

AlertSite 11
by SMARTBEAR
largely on user expectations, and those expectations are set, in large
part, by the user’s experience with other sites, including your competi-
tors’.

Benchmarking
Benchmarking enables you to measure your Web performance against
others in similar verticals, markets and regions — data often included in
many WPM solutions. You can also subscribe to a number of industry
websites and publications to receive data focused on the performance
of Web and mobile sites, homepages and business transactions across
a wide variety of competitive industries. However, for direct competitors,
it’s often best to supplement industry data with your own benchmarks.

Creating your own set of performance benchmarks enables you to

establish use cases and user paths to get a better perspective on how
your customers’ actual online experiences are stacking up against the
competition.

A screenshot of benchmark reporting of Financial Services Organizations

from AlertSite

AlertSite 12
by SMARTBEAR
Combining transaction monitoring with performance benchmarks pro-
vides a more detailed approach to gauge where your site’s performance
ranks against competitors within the same vertical. You can accomplish
this by creating a transaction monitoring script that moves through
multiple pages within the competitor’s site, ideally toward a conversion
you’d like to measure on your own site. Then create a script following an
equivalent path using your own site.

Run these transactions at irregular intervals, and collect at least five

samples for each path. Response times will drift, and you’ll want to aver-
age the data before comparing benchmarks, but once established you’ll
have a running baseline of your competitors’ performance as it relates to
your own. These measurements will provide a framework against which
you can judge your own site’s performance, helping guide future site
optimizations.

Establishing baseline is also beneficial for addressing performance

variances across browsers. Firefox, IE and Google Chrome load source
code, images and third-party objects differently. Cross-browser moni-
toring quickly identifies browser-specific problems and performance
bottleneck you can address to ensure a consistent cross browser Web
experience for your users.

While benchmarking will provide you with good reference points, it’s
important to dig deeper. The key stat benchmarked is usually response
time or load time, but consider getting more granular. Metrics like time to
first paint or above the fold load time can help reveal where weaknesses
lie. If one site is generally faster or slower than another, differences in
this granular detail can help point to “why.”

Benchmarks only offer a high-level view of performance. Down in the

details you might find an interesting outlier or a strange pattern devel-
oping over a number of days, weeks or months. It could be an applica-
tion issue or a failing of an external service provider such as a CDN

AlertSite 13
by SMARTBEAR
or streaming vendor. Worst case, what appears to be a trivial outlier is
actually a sign of a security breach.

3rd Party Content

It’s becoming increasingly more common that performance problems
manifest as a result of someone else’s content. Ads are the culprits
everyone thinks of, but code snippets for services like Google Analytics,
Quantcast, and ValueClick — as well as social media widgets — are
often more cumbersome in the aggregate. Truly egregious third-party
content can make your page unusable with JavaScript errors, blocked
rendering, or HTTP timeouts keeping users from seeing your own con-
tent. Therefore, each of these elements should be isolated and mea-
sured separately via your WPM solution.

Looking at Internal Performance

We discussed the importance of measuring performance from multiple
worldwide locations if you conduct business internationally — knowing
how your site will perform vs. competitors in different countries is critical.
Monitoring from different locations also helps indicate if a problem is
a network issue isolated to a single location or more likely an applica-
tion problem experienced by all. However, one that’s overlooked far
too often is right behind your own firewall. The performance of internal
Web applications can be critical to users in customer-facing business
functions like customer service, sales, and support — monitoring that
performance needs to be top of mind for organizations relying on these
in-house applications to meet customer expectations and generate rev-
enue. An internal performance slowdown may not manifest on the public
facing website, but if its impact affects internal departments responsible
for servicing customers the user experience will almost certainly suffer.

AlertSite 14
by SMARTBEAR
Conclusion
Web Performance Monitoring goes well beyond simply keeping tabs on
your server — it’s a complex mix of techniques and technologies that
work together to continually improve the competitiveness of your busi-
ness. The fact is that slowdowns and outages will result in a measur-
able loss of revenue for your business — Web performance monitoring
mitigates the risks associated with poor performance by enabling you to
identify and address performance issues before your users are affected.

AlertSite 15
by SMARTBEAR
About AlertSite

AlertSite, a part of SmartBear Software, is a global leader in Web, API

and mobile performance monitoring solutions that continuously improve
the Web user experience. AlertSite uniquely provides both technical per-
formance measurement as well as visual user experience measurement
from more than 80 locations around the globe and from within your en-
vironment with InSite, a private monitoring location. AlertSite’s services
measure basic availability, Web performance using real instances of
IE, Firefox and Chrome browsers, API performance and mobile website
performance. Gain a real-time view of service quality in terms of avail-
ability, performance, consistency and the user experience.

Sign up for your complimentary 30-day trial here.

AlertSite 16
by SMARTBEAR
About SmartBear Software
More than one million developers, testers and operations profession-
als use SmartBear tools to ensure the quality and performance of their
APIs, desktop, mobile, Web and cloud-based applications. SmartBear
products are easy to use and deploy, are affordable and available for
trial at the website. Learn more about the company’s award-winning
tools or join the active user community at http://www.smartbear.com, on
Facebook or follow us on Twitter @smartbear and Google+.

SmartBear Software, Inc. 100 Cummings Center, Suite 234N Beverly, MA 01915
+1 978.236.7900 www.smartbear.com
©2013 by SmartBear Software, Inc. Specifications subject to change.
EBOK_AS_20131025-Monitoring-101