Web Performance Optimization:

Key Caching Strategies

Author: Scott Price

LoadStorm
1

Table of Contents
Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Part 1: Web Server Caching. . . . . . . . . . . . . . . . . . . . 5
Part 2: Application Caching. . . . . . . . . . . . . . . . . . . . 8
Part 3: Data Caching. . . . . . . . . . . . . . . . . . . . . . . . . 11
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15

Introduction:
Web site optimization directly affects a
company's bottom line. A sudden traffic spike
that swamps a website's capacity can cost a
company thousands or even tens of
thousands of dollars per hour. Web servers
and Web applications should be built and
deployed from day one with performance at
the forefront of everyone's mind.

It is our hope that these

suggestions and best practices will
assist you in improving the speed of
your site. We view performance
engineering as an iterative process
whereby developers and testers will
run load tests, analyze measurements,
and tune the system incrementally. Our
goal is to help you make your site
faster and handle higher traffic.

Web site administrators and web application developers have a host of tricks
and techniques they can employ to deliver Web pages more quickly. Caching is the
#1 tuning trick in the web developers kit. Customers ask me weekly what I recommend
for speeding up their app. I always start with "caching, caching, and more caching". It's
like magic for a site.

What is Caching?

Caching refers to any mechanism that stores previously retrieved content for future
use. As I learned it in college back in the VAX/VMS operating systems class, it is
temporarily putting something into memory that you will use again in order to avoid
hitting the hard drive.
Computer scientists today are less concerned about saving every byte like we were
back then. Still, web applications are constantly re-using data and files; so why in the
world would we want to make an expensive hit to the database? Hard drives can be
10,000 times slower than memory because they are mechanical and must move to the
correct position and spin to the exact spot where data exists. Memory moves at the
speed of electricity.
The goal of caching is to increase the speed of content delivery by reducing the
amount of redundant work a server needs to perform. Putting a file in memory to
re-use it can save millions of drive accesses; thus, the speed of getting the browser
what the user needs is increased by magnitudes. Caching and performance go handin-hand. It's a no-brainer.

Part 1: Web Server Caching

Let's examine how a Web server can optimize content delivery both by controlling the
Web browser cache and by caching content in memory.
Why Care?
A ViSolve white paper shows a return on
investment of $61,000 for a $20,000 total cost of
ownership of only two caching servers!

Web Server Cache Control:

The most well known cache is the Web browser cache stored on a user's machine.
When a Web browser such as IE, Firefox or Chrome retrieves a resource, it doesn't
display it once and throw it away: it stores the content on the user's local hard drive.
A Web server can use a number of techniques (discussed below) to instruct the Web
browser when it is permissible to use this local copy of the content in lieu of
downloading the content again from the Web server.
A Web server delivers different types of data, including HTML code, images,
stylesheets, and Javascript libraries. Some of this information, such as the output of a
PHP or other CGI script, is highly dynamic and volatile, and can return different results
each time it is accessed. Other files, such as images and client-side scripts, may
change far less frequently. These files are prime candidates for caching.
Web server cache control uses HTTP headers to regulate which resources are pulled
from the cache, and how often. Setting effective cache control headers on content is
critical to the performance of a Web application because it directly affects how
browsers request files from your web server.

The Expires header tells a Web browser that a specif ic type of header expires at a
specif ic day and time. The Cache-control header uses a combination of caching
directives and age modif iers to instruct a Web client on whether a specif ic piece of
content can or cannot be cached, and for how long. The Cache-control header is
documented in section 14.9 of RFC2616. [1]
Web servers can also use the ETag header, which assigns a unique ID to each version
of a file, image, or other component. When the Web server delivers content, it stamps
it with an ETag value. Later, if the client believes the content might have expired, it
makes an HTTP request including an If-None-Match header, with the value of that
header set to the last ETag value for that component. If the content has changed since
that ETag value was issued, the server will respond with new content. Otherwise, if the
content has not changed, the server will response with a 304 HTTP status and an
empty HTTP response body. ETags prevent a Web server from re-delivering content
that is still fresh, thereby conserving server resources and reducing page load time.

Web Server Caching and Cache Proxies:

Web server caching is implemented in one of two ways. A Web server can cache
relatively static content in memory, greatly reducing the amount of time spent retrieving
content from storage. Some Web servers, such as G-WAN, are optimized for static
content, and automatically cache data in memory. Other Web servers allow memory
caching through configuration files. Apache, for instance, supports the mod_cache
module, which provides both a memory cache for static content (mod_mem_cache),
and a disk cache for caching content assembled from dynamic sources or retrieved
from an external provider (mod_disk_cache).
A web server cache can also take the form of a special server called a web cache
proxy, which intercepts client requests and serves content previously cached from the
actual host server where the application resides. These proxy caches are usually
operated in various locations worldwide, and requests are routed dynamically to the
appropriate proxy cache server based on the user's location. This user proximity
speeds content delivery to the user by reducing network latency. It also reduces the
number of requests made against the target server.

Cisco describes a proxy cache this way: When a browser wishes to retrieve a URL, it
takes the host name component and translates that name to an IP address.[2] A HTTP
session is opened against that address, and the client requests the URL from the
server.
When using a proxy cache, not much is altered in the transaction. The client opens a
HTTP session with the proxy cache, and directs the URL request to the proxy cache
instead.
Cache proxies work the same way that a Web browser's cache works: they use the
information included in HTTP headers such as Expires and Cache-control to
determine if a given component is fresh or stale. Setting accurate cache control
headers on content is critical for the success of a Web cache proxy.
Most high-volume Web projects use some form of Web cache proxy. For example,
Wikimedia deploys 50 instances of the Squid web cache proxy in three locations
worldwide to speed delivery of content to users.[3]

Web Server Caching Helps:

The conclusion is obvious: cache early and cache often. Caching is good - caching is
your friend. If you just built a web application, then my recommendation to you is to
turn on caching everywhere you can in the architecture.
While Web server caching and cache control are extremely powerful techniques for
reducing load on a Web server, they're not the only types of caching available. Read
Parts 2 and 3 for other types of caching.

Part 2: Application Caching

What is Application Caching?
Application caching stores
calculated components in
memory for future use,
either by the same user or
multiple users. Because of
the complex architecture of
modern Web applications,
application caching can
produce huge gains in Web
site response times.
A Web application is usually comprised of multiple architectural layers. At the simplest
level, Web applications consist of the client, which runs the application (usually a
series of HTML pages and associated technology); and the server, which processes
data and generates output. When we zoom in on the server side, however, we find
that most non-trivial applications consist of several additional layers. Typically, these
layers include the UI layer, which generates the user interface (usually HTML); the
business logic layer, which implements business rules; and the data layer, which
stores and retrieves data from one or multiple data sources.
In examining an online bookstore- The layers would break down as follows:

UI Layer: The browsing and shopping cart interface shown to the users.

Business Logic Layer: All rules pertaining to processing orders - calculating

sales tax and shipping, combining and splitting orders, gift orders, tracking
shipments, etc.

Data Layer: The catalog of books, record of orders, and order workflow storage.

Obviously, this bookstore is highly dynamic: the server must generate a slightly
different interface for each user based on the user's account preferences and current
shopping activity. Exchanging data between these layers and generating every
component required for every request decreases site response times. Not every
component, however, needs to be generated fresh every time. For example, each Web
page may have its header and footer broken out into static components that can be
cached by the application.
The shopping site may also elect to cache dynamically generated data that change
infrequently, such as selection lists of current shipping rates.
Even the user's current shopping cart status can be cached, as there is no need for
the server to regenerate that data until the user invalidates it by changing the contents
of her cart.
Storing components in memory for later use can represent a huge performance
savings for the application, as it reduces database retrieval requests, numeric
calculations, and string concatenation operations.

Considerations in Caching:
Most application caching takes place on the server side. When caching data on the
server, applications must balance a number of factors:
Scope:
Who can reuse the cached component? Some components, such as shopping
cart content, may be specif ic to the current user. Other components, such as a
list of shipping rates, may be global, and can be placed in a memory pool
shared by all server connections. For global components, applications that run
in distributed server environments must implement a caching mechanism that
can be accessed across multiple Web servers. This can be accomplished either
through some centralized cache (such as a Web Service that accesses a
common data set), or by synchronizing the cache across all servers.

Server Resources:
Cached data reduces both processing and disk access time at the expense of
system memory. Application caches require a mechanism to scavenge items out
of the cache should memory become scarce. Applications must also take care
not to exhaust system memory themselves through aggressive caching.
Invalidation and Expiration:
Just as with server and client resource caching, application caching requires a
mechanism to invalidate a cached item. Applications can assign expiration times
to these components, just as Web servers do with files served over HTTP.
Alternatively, components can be expired in response to an event, such as a
database update or user action. In our shopping cart example, the user's
shopping cart component can be expired whenever the user modif ies her cart.

Application Caching from Server Applications:

Application caching in modern Web applications is handled programmatically, and
differs greatly based upon the application environment used. Most server-side
application platforms offer a built-in caching library. In ASP.NET, the Cache class
provides a facility for caching data and setting expiration conditions. The class also
supports supplying a connection string so that data can be shared across multiple
servers. Similar caching frameworks exist for other systems, including the Java
Caching System (JCS) for Java[4] and the Alternative PHP Cache (APC) for PHP[5].

The Danger of Application Caching:

While application caching can yield enormous benefits, it can also hide severe
performance problems. Development teams should load test the performance of their
applications prior to implementing any caching scheme to isolate performance
bottlenecks. It's a good idea to implement targeted caching only after other
performance issues introduced by inefficient algorithms and redundant data queries
have been remedied.

10

Part 3: Data Caching

In parts 1 and 2 of this Web
Performance Optimization eBook,
we've examined how developers can
employ Web server caching and
application caching to speed up their
Web applications. In this section, we'll
see how caching entire data sets can
increase platform efficiency.

What is Data Caching?

Database caching is a species of application caching that caches the result of one or
more database queries in the application server's memory.
For our purposes, we use application caching to refer to caching any component in
an application server's memory, and data caching to refer to caching a collection of
data for future querying and sorting.
There are two main approaches to data caching:
Data set caching: Data returned from a database is stored in a data set, an inmemory representation of data that mimics the column/row structure of a
relational database.
In-memory databases: An in-memory database is a relational database that
operates completely in a server's RAM, as opposed to storing data to a
hard drive. In-memory databases can be used for fast access to data
previously retrieved from a traditional, disk-based RDBMS.

11

Data Set Caching:

One of the best architectures for data set caching is
found in Microsoft's .NET platform. The DataSet class
in .NET implements a disconnected data set. Once a
DataSet is filled with the data returned from a SQL
query, the connection to the underlying database is
broken. This makes DataSets ideal for caching in the
Application object.

David Burgett discusses using cached DataSets in detail in an article in MSDN

Magazine. In Burgett's example, a developer of a credit card authorization Web
service for businesses retrieves the list of authorized customers into a DataSet and
caches the DataSet in memory for future reference.[6]
What happens when a customer changes part of their customer record? In this case,
the developer can maximize the benefits of caching by changing the data directly in
the cached DataSet, and then push the change down to the underlying database.
Instead of invalidating the entire DataSet and re-querying the database for all
customer records (a costly operation for performance), the developer need only
commit changes to a single customer record. In his article, Burgett further
demonstrates how the application can use Web Service calls to keep all of the
DataSet caches in a server farm synchronized.
While .NET contains the best support for data set caching, it can be utilized (or at least
emulated) in other platforms. PHP programmers can retrieve the results of their
database queries as associative arrays, which can be cached in shared memory using
the Alternative PHP Cache (PHP). Java contains more direct support for data caching
using Service Data Objects (SDO).[7]

12

In-Memory Databases and Data Caches:

Some database vendors support an even more sophisticated version of caching via inmemory databases.
The Oracle In-Memory Database Cache supports caching critical subsets of Oracle
relational data tables in the application tier. This cache relies upon Oracle's TimesTen
In-Memory Database, which maintains its databases as "embedded databases"
entirely within system memory.[8]
TimesTen uses checkpointing to persist in-memory data to disk for recoverability, and
transactional replication to keep data consistent across multiple servers. Server
replication and checkpointing both insure against data loss in the case of server
failure.

Other in-memory database

systems include SQLite and IBM's
solidDB. MySQL Cluster may also
be configured as an in-memory
database.
Some developers argue that
embedded databases are the
wave of the future. As writer Greg
Linden points out, most highly
performant sites like Facebook
already cache so much data that
up to 75% of their total data
retrieval is from RAM anyway![9]

13

Temporary Tables and Database Connections:

In the past, it was common for client/server developers to utilize temporary tables
stored in a database server's memory. The temporary table could hold the output of
queries that were the result of complex JOIN operations. However, most temporary
tables are associated with a specif ic database connection. Since nearly all Web-based
applications use database connection pooling, using temporary tables isn't feasible.
Some database systems support memory-based database tables. For instance,
MySQL can create temporary tables with the MEMORY command. However, these
tables are specif ic to a single instance of the database server and don't support rowlevel locking.

Data Caching Benefits:

Moving data to and from a database is typically one of the bottlenecks of a web
application's performance because it involves physical devices actuating mechanical
arms that are exponentially slower than electricity flowing through memory chips.
Thus, putting data into cache memory has enormous performance gains.
In general, developers will achieve greater speed boosts by storing data sets in the
application tier, and using the techniques described above to keep these cached data
sets synchronized across application and database servers.

14

Summary:
There are many ways to improve the speed of your site. This eBook is designed to
give you practical suggestions to try in your system environment. Web performance
optimization is an engineering discipline which can have a tremendous impact on
revenue and customer satisfaction. Caching is probably the biggest weapon in the
arsenal of performance engineers, and it is usually relatively easy to utilize.

Our load testing tool is great at measuring

the response times and errors from your
site, but it doesn't tell you how to tune the
components of your architecture to improve
performance. These tips and tricks will
hopefully assist you in taking the metrics
LoadStorm provides and make tangible
changes.

Get a FREE account and storm your site with 25 concurrent V-Users

15

References
1. http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9

2. http://www.cisco.com/web/about/ac123/ac147/ac174/ac199/about_cisco_ipj_arc
hive_article09186a00800c8903.html

3. http://meta.wikimedia.org/wiki/Wikimedia_servers
4. http://jakarta.apache.org/jcs
5. http://php.net/manual/en/book.apc.php
6. http://msdn.microsoft.com/en-us/magazine/cc188799.aspx
7. http://www.ibm.com/developerworks/java/library/j-sdo/
8. http://www.oracle.com/technetwork/database/timesten/overview/timesten-imdb086887.html

9. http://glinden.blogspot.com/2009/11/put-that-database-in-memory.html
10. http://loadstorm.com/2009/web-performance-tuning

16