Syntel CQA Forum Effective Testing for E-Commerce
Effective Testing for E-commerce
Introduction
What is e-commerce? E-Commerce is defined
as the software and business processes
required to allow businesses to operate solely
or primarily using digital data flows.
Why is testing important in the e-commerce
environment?
 The first and primary reason is because e-
commerce is, by its very nature, business
critical.
 The second reason is that e-commerce is
a massive and growing market place but
one which requires large up-front
investment to enter successfully. There
are already 5.8 million web sites
worldwide.
 The third reason is because the history of
e-commerce development has been
littered with expensive failures, at least
some of which could have been avoided
by better testing before the site was
opened to the general public.
The Testing Challenges
A successful e-commerce application is:
•Usable. Problems with user interfaces lose
clients.
•Secure. Privacy, access control,
authentication, integrity and non-
repudiation are big issues.
•Scaleable. Success will bring increasing
demand.
•Reliable. Failure is unthinkable for a
business critical system.
•Maintainable. High rates of change are
fundamental to e-commerce.
•Highly available. Downtime is too
expensive to tolerate.
Technical Issues
The development process for e-commerce
has unique characteristics and some
associated risks. It is generally recognized
that a ‘web year’ is about 2 months long. For
this reason, Rapid Application Development
(RAD) techniques predominate in the e-
commerce environment, and in some cases
development is even done directly in a
production environment rather than in a
separate development environment. These
changes increase risk and create new
challenges for testers, because time
pressures militate against spending a longer
time testing sites before they are released.
10718249.doc
CQA Doc No 22
At the same time, the technical environment
of front-end systems is changing very rapidly,
so change is imposed on e-commerce sites
even when the site itself is not changing. This
requires more regression testing than would
be expected in a conventional application to
ensure that the site continues to function
acceptably after changes to browsers, search
engines and portals. New issues have also
come to the fore for testers, notably security
of transactions and the performance of web
sites under heavy load conditions.
Front End Systems
Static Testing. The front end of an e-
commerce site is usually a web site that
needs testing in its own right, it must also
offer an acceptable level of service on one or
more platforms, and have portability between
chosen platforms. It should be tested against
a variety of browsers, to ensure that images
seen across browsers are of the same quality.
Usability is a key issue and testing must
adopt a user perspective. It is also important
to gain confidence in the security of the site.
Many of these tests can be automated by
creating and running a file of typical user
interactions – useful for regression testing
and to save time in checking basic
functionality.
Dynamic Testing. The services offered to
customers must be systematically explored,
including the turnaround time for each
service and the overall server response. This,
too, must be exercised across alternative
platforms, browsers and network
connections. E-commerce applications are
essentially transaction-oriented, based on
key business processes, and will require
effective interfacing between intranet-based
and extranet-based applications.
Back End Systems
The back end of e-commerce systems will
typically include ERP and database
applications. Back end testing, therefore, is
about business application testing and does
not pose any new or poorly understood
problems from a business perspective, but
there are potential new technical problems,
such as server load balancing. What is
essential, however, is to apply the key front
end testing scenarios to the back end
systems. In other words, the back end
systems should be driven by the same real
transactions and data that will be used in
front end testing. The back end may well
prove to be a bottleneck for user services, so
performance under load and scalability are
key issues to be addressed. Security is an
Page 1 of 3
Syntel CQA Forum Effective Testing for E-Commerce
issue in its own right, but also has potential
to impact on performance.
Middleware and Integration
Integration is the key to e-commerce. In
order to build an e-commerce application,
one or more of the following components are
usually integrated:
•Database Server
•Server-side application scripts/programs
•Application server
•HTML forms for user interface
•Application scripts on the client
•Payment server
•Scripts/programs to integrate with
legacy back-end systems
The process of developing an e-commerce
site is significantly different from developing
a web site – commerce adds extra levels of
complexity. One highly complex feature is
that of integration.
If an application is being built that uses a
database server, web server and payment
server from different vendors, there is
considerable effort involved in networking
these components, understanding
connectivity-related issues and integrating
them into a single development (executable)
environment. If legacy code is involved, this
adds a new dimension to the problem, since
time will need to be invested in
understanding the interfaces to the legacy
code, and the likely impact of any changes.
It is also crucial to keep in mind the steep
learning curve associated with cutting-edge
technologies. Keeping pace with the latest
versions of the development tools and
products to be integrated, their compatibility
with the previous versions, and investigating
all the new features for building optimal
solutions for performance can be a daunting
task. Also, since e-commerce applications on
the web are a relatively new phenomenon,
there are unlikely to be any metrics on
similar projects to help with project planning
and development.
The maintenance tasks of installing and
upgrading applications can also become very
involved, since they demand expertise in:
•Database administration.
•Web server administration.
•Payment server administration.
•Administration of any other special tools
that have been integrated into the site.
10718249.doc
CQA Doc No 22
Technical support should also be borne in
mind.
Correctly functioning back-end and front-end
systems offer no guarantees of reliable
overall functionality or performance. End-to-
end testing of complete integrated
architectures, using realistic transactions, is
an essential component.
Ten Key Principles of Effective E-
Commerce Testing
Principle 1. Testing is a risk
management process
Effective testing adopts a strategy that is
tailored to the type of application or service
being tested, the business value of the
application or service, and the risks that
would accompany its failure Plans should
include measures of risk and value and
incorporate testing and other quality-related
activities that ensure development is
properly focused on achieving maximum
value with minimum risk.
Principle 2. Know the value of the
applications being tested.
To manage risk effectively, we must know the
business value of success as well as the cost
of failure. The business community must be
involved in setting values on which the risk
assessment can be based and committed to
delivering an agreed level of quality.
Principle 3. Set clear testing objectives
and criteria for successful completion
(including test coverage measures).
The test programme must be properly
planned, with test scripts giving precise
instructions and expected results. There will
also need to be some cross-referencing back
to the requirements and objectives, so that
some assessment can be made of how many
of the requirements have been tested at any
given time
Principle 4. Create an effective test
environment.
Cross-platform testing is, naturally, an
important part of testing any multi-platform
software application. In the case of e-
commerce, the term ‘cross-platform’ must
also extend to include ‘cross-browser’.
In order to ensure that a site loads and
functions properly from all supported
platforms, as much stress and load testing as
possible should be performed.
Principle 5. Test as early as possible in
the development cycle.
Page 2 of 3
Syntel CQA Forum Effective Testing for E-Commerce CQA Doc No 22
In the case of an e-commerce site, a fault be immediately expensive in terms of lost
found after shipping will have been detected revenue and even more expensive in the
as a failure of the site by the marketplace, longer term if disaffected users seek
which is potentially as large as the number of alternative sites. Yet the time pressures in the
Internet users, hence that the earlier faults e-commerce world militate against the
are detected, the cheaper the cost of thorough testing usually associated with
rectification. business criticality, so a new approach is
needed to enable testing to be integrated
Principle 6. User Acceptance Testing
into the development process and to ensure
(UAT).
that testing does not present a significant
E-commerce users are becoming time burden.
increasingly intolerant of poor sites, and
technical issues related to functionality,
performance or reliability have been cited as
primary reasons why customers have
abandoned sites. Early exposure of users to
sites with problems increases the probability
that they will find the site unacceptable, even
if developers continue to improve the site
during beta testing.
Principle 7. Regression testing. Web-
based applications that reference external
links need regular regression testing, even if
their functionality does not change, because
the environment is changing continuously
Principle 8. Automate as much as
possible.
The key is to take testing processes
sufficiently seriously that you document them
and control them so that automation
becomes a feasible option – then you select,
purchase and install the tools.
Principle 9. Capture test incidents and
use them to manage risk at release
time.
All incidents found must be recorded via an
incident management system (IMS), which
can then be used to ascertain what faults are
outstanding in the system and what the risks
of release might be. Outstanding incidents
can be one of the completion criteria that we
apply, so the ability to track and evaluate the
importance of incidents is crucial to the
management of testing.
Principle 10. Manage change properly to
avoid undoing all the testing effort.
Things change quickly and often in an e-
commerce development and management of
change can be a bottleneck, but there is little
point in testing one version of a software
application and then shipping a different
version; not only is the testing effort wasted,
but the risk is not reduced either.
Conclusions
Testing is crucial to e-commerce because e-
commerce sites are both business critical and
highly visible to their users; any failure can
10718249.doc Page 3 of 3