Ebook

Using Digital Experience Monitoring

for Cloud Performance Management

TM

1
What Does it Mean to be “in the Cloud”?
There’s a lot of talk about migrating applications to the cloud and
managing the cloud’s performance, but what exactly does this mean?
While we seem to be entering an era of “everything-as-a-service,” cloud
services are typically divided into three broad categories: software as
a service (SaaS), infrastructure as a service (IaaS), and platform as a
service (PaaS).
SaaS was one of the first cloud solutions to hit the market. It provides
the ability to consume information through a client interface, which
is often a browser. IT teams don’t have to manage applications,
hardware, security, or storage because everything is managed by
the vendor. The main categories of SaaS applications are business
productivity, business management, tools, and customer relationship
management.
PaaS gives some control back to the customer as it enables companies
to build and deploy applications upon an existing platform. IT
manages the applications in a hosted environment while everything
else is managed by the vendor.
IaaS is the final component and gives the most control to
organizations. They can manage applications, runtimes, security and
integrations, and databases. Servers, storage, and networking are
managed by the vendor. IaaS is often seen as a cost-effective way for
startups, as no capital investment is required. Categories under IaaS
include cloud management, content delivery networks, virtualization,
and storage and computing solutions.
If an organization is using a solution that falls into any of these
categories, it’s imperative to develop a monitoring strategy that is
tailored to the cloud.
2
State of the Cloud
Cloud spending is estimated to reach $383 billion by 2020, according to Gartner, with “cloud adoption strategies
influencing greater than 50% of IT outsourcing deals.” Many business units have embraced SaaS solutions because it
allows them to operate more independently and to support a geographically dispersed or remote workforce more easily.

The cloud provides flexibility and scalability. There’s no need to assign IT resources and purchase hardware to deploy a
new software solution—the business unit can usually sign a contract and start using a solution almost immediately.

3
 F5’s 2017 State of Application Delivery report reveals 20% of companies
have over 50% of their applications in the cloud. Organizations may not
even be aware of all the cloud solutions being used by all of its employees.
The number of cloud applications on which you rely typically varies based
on your job function, with marketing and HR using over two times the
number of cloud services as software development teams.

As applications migrate to the cloud, the role of IT must evolve as well. IT is

now tasked with managing and securing many applications, some of which
they know little about. Traditionally, IT has been responsible for deploying
an application and then ensuring availability. With SaaS, installation usually
isn’t required and the SaaS provider is responsible for availability—but
when issues arise, who is responsible for fixing them and how do you gain
visibility into what’s happening? Solutions from multiple vendors need
to be integrated to provide the functionality an organization needs. Add
to this customization and the need for security and governance, and it’s no wonder 57% of IT professionals surveyed by
BetterCloud say SaaS applications make their lives more difficult.

Where does that leave Ops? It depends on who you ask because there is a disparity between the business unit and IT.
IT can provide value-added services: advising on which applications should move to the cloud, integration of multiple
SaaS solutions, support, and providing
governance and SLA management.
Monitoring systems is already standard
practice for infrastructure and operations
(I&O) teams, but the existing arsenal of
tools may not be sufficient.

4
Why Monitor Cloud Performance?
Users’ expectations remain the same whether the application is hosted on premise or in the cloud. They expect the
software to be as well designed, easy to use, and reliable as on premise software, even though the application may be
further away and the sometimes-unstable nature of the Internet can affect performance and availability.

Don’t fall into the trap of thinking your SaaS provider is monitoring their application so you don’t have to. Yes, there is
likely a Service Level Agreement (SLA) in place, but how do you know if the terms are being met if you aren’t monitoring
the application yourself? Are you blindly trusting the vendor to tell you when there is a problem? Some vendors provide
their own monitoring solution to keep an eye on performance. AWS, for example, offers CloudWatch to monitor almost all
their services; customers can capture metrics such as latency, but monitoring content delivered by a service with the same
service may provide skewed results. You can’t assume that all of your users are requesting content from within the cloud
becuase they’re likely using anything from their mobile provider, wifi from a coffee shop, or their home Internet provider.
Monitoring from the cloud vendor should be augmented to get a more comprehensive view of the digital experience. You
need to treat your cloud applications just as you would any on premise application.

Traditional application and infrastructure monitoring tools aren’t much help when it comes to monitoring the cloud.
Tools that rely on instrumentation of code or systems like real user measurement and APM aren’t much use for SaaS
applications because source code is controlled by the SaaS provider, not the end user’s organization. Server and network
monitoring tools are great for the organization’s own data center, but don’t deliver much visibility of heavily virtualized
cloud environments that are controlled by a third party and scattered all over the world.

Applications must be monitored from your cloud provider’s and end-users’ perspectives to detect any bottlenecks.
Historical reporting on those service levels will hold providers to their SLAs.

5
Developing a Cloud Performance
Monitoring Strategy
Monitor from the user’s perspective

When you’re monitoring a cloud vendor, examine the various touch points. How are your employees and customers
interacting with the application? Are there tags embedded on a customer facing application? Is there a portal through which
employees access to perform a task? Both? Ensure you are monitoring the provider from vantage points that make the
most sense.

If there are tags on a customer-facing application, run synthetic tests from locations that represent your customers. If
internal employees are accessing a web-based portal, look for a way to test the application from internal locations. Access
the application the same way the users do, otherwise you won’t fully understand how the application is performing.
Cloud services encompass more than just SaaS and IaaS. Internet services like Domain Name Services (DNS) and Content
Delivery Networks (CDN) are also delivered by third-party providers and replace expensive infrastructure an organization
would otherwise have to provide themselves. These services are essential to the fast, reliable delivery of websites and
applications. Monitoring them is a necessity to ensure service delivery and SLA compliance.

CDN Monitoring

CDNs are often used to deliver improved performance to end users. Delivering content from locations closer to end users
reduces latency and reduces delivery times. If you are using a CDN, you want to ensure the performance from the CDN is
in fact faster. A CDN adds a layer of complexity to applications. When an issue occurs, the root cause could be found at the
CDN, origin, or something in between.

The best way to quickly identify the origin of the issue is to follow the strategy below:
• Monitor critical pages from the origin and the CDN
• Select the most popular or important resources and monitor them from both the origin and CDN
• Monitor at least one object that must be retrieved from the origin each time to identify if an issue is in the communication
between the CDN and origin. Appending a random number to the end of an object can achieve this goal.
6
DNS Monitoring

Monitoring DNS isn’t as simple as ensuring a browser can resolve a domain name into an IP address. When DNS fails, there
are multiple points at which the failure can occur – ISP or third-party DNS resolver, the root servers, the top-level DNS
servers, or the authoritative servers.

A DNS monitoring strategy needs to monitor all of these moving parts:

• When a failure occurs, each server in the DNS route needs to be queried to isolate where the issue resides.
• Monitor from the perspective of the end user. Query DNS as the end user would query it from their device. Test from a
variety of ISPs and third-party DNS resolvers such as Google’s public resolver.
• Test resolution of both IPv4 and IPv6 domains.
• Validate the correct results are being returned, not just that the response time is within appropriate thresholds.

API Monitoring

If you are utilizing the API of a cloud solution, this also must be monitored to ensure calls between the API and your
application are functioning and performing as expected. APIs extend the functionality of web applications and drive
automation. Developers take advantage of the interoperability of APIs to build modules that communicate seamlessly
irrespective of the system’s configuration. This provides endless possibilities to innovate and make a positive impact on
user experience. APIs have become an indispensable part of the digital world. A SaaS provider’s APIs may be used in several
different ways by an organization:

• Integrate a payment gateway with an ecommerce platform

• Extend the functionality of an application
• Manage and track productivity within the organization
• Share information across teams

These services enable organizations to extend their customer reach, increase revenue, and innovate faster. The
performance of these services has a direct impact on the user’s digital experience. Monitoring the performance of the APIs
provides the following:
• Statistics on service availability (uptime/downtime)
• Validation the correct data is returned
• Insight into error handling and appropriate alerting
• Knowledge of performance from different perspectives

Ignoring APIs when monitoring cloud applications can expose your organization to unnecessary risks. If APIs are being used,
they must be monitored. 7
Measure SLAs

SLAs protect customers from poor performance and provide objective grading criteria. Today, most cloud providers have
SLAs. Make sure you carefully review the document and understand the terms of the agreement. An SLA with a provider
should be:

• Attainable
• Repeatable
• Measurable
• Meaningful
• Mutually acceptable

Determine what tests can be set up to monitor your SLAs. Typically, part of monitoring includes setting up alerts when
thresholds are passed or errors are encountered. When an alert is generated this sets off a course of action to remedy the
situation. When an SLA is in breach there may not be an immediate course of action that needs to be taken. Determine
whether SLAs should generate alerts or focus primarily on daily/weekly/monthly reporting. Whether an alert should be
raised depends on the service level being monitored.

8
Critical Cloud Performance Management
Capabilities
As the role of IT evolves to address the growth of cloud, a different set of monitoring capabilities may be needed. The same
metrics and tools used for traditional applications may not apply for cloud-powered applications. Below are some capabilities
you may want to consider.

Capture Accurate Data

Sometimes it’s not feasible to install an agent everywhere or use real user monitoring to capture metrics from the end user’s
location. In those instances, ensure you are monitoring from locations representative of your users. Selecting a solution with
geographically dispersed monitoring locations can help highlight the level of performance your users are experiencing.
Monitoring locations should not be limited to the cloud. What happens if all your monitoring locations are with a single cloud
provider and that provider experiences an outage? Monitoring across a breadth of providers gives the most resiliency and
insight into how applications are performing.

When dealing with SaaS applications many of your users accessing applications are coming from centralized locations.
The ability to deploy monitoring agents in offices, warehouses, call centers, wherever users are connecting to critical SaaS
applications can show you the performance from the user’s perspective.

9
Identify Key Business Processes

Monitoring the home page or log-in page isn’t going to provide much insight for a SaaS application; you need to be able
to test the key business processes that users are performing. If your company is just starting out with a SaaS solution, you
may not know what transactions are the most important to measure. Selecting transactions from a library of common
scripts can help get you started.

Communicate Findings with Key Stakeholders

Don’t leave it up to the provider to inform you when an SLA has been breached. The ability to configure tests and reports
to quickly see on a daily, weekly, or monthly basis whether SLA thresholds are being met will help you enforce SLAs.
It is possible that all departments within an organization are using some type of cloud service. They don’t care about all
applications only the solutions that are relevant to them. Create and share dashboards for individual teams showing the
performance and availability of the cloud solutions relevant to them. Providing teams with an easy way to quickly see
whether the services they use are performing as expected can help reduce inquiries and increase efficiency.

10
Conclusion
While the cloud has opened new opportunities and made life easier for new technical solutions to be implemented, it’s not without challenges. The
traditional role of IT and monitoring needs to evolve to meet the changing needs of a cloud-first world. Take a look at your monitoring strategy to
see how it could be revised to provide more visibility and ensure the user experience is not negatively affected.

11
A Different Approach
to Digital Experience
Monitoring
Catchpoint is a leading digital experience intelligence
company that provides unparalleled insight into your
customer-critical services to help you consistently deliver 17 Smart Monitors
amazing digital experiences. Catchpoint is the only Real browser, multi-transaction, mobile, HTML code, API,
performance monitoring platform that provides integrated streaming, DNS, FTP, TCP, SMTP, ping, traceroute, SSH,
synthetic and real user monitoring, comprehensive test NTP, IMAP, web socket and MQTT.
types, real-time analytics, and a diverse node network to
help you continuously preempt performance issues and Deepest and broadest diagnostics: 100 days of object level
data; 3 years of raw aggregate data.
optimize service delivery. More than 400 customers in over
30 countries trust Catchpoint to strengthen their brands
and grow their businesses.

To request a free trial, visit

pages.catchpoint.com/freetrial.html

TM

12