Case Study: Penetration-Testing Tool Use

Gartner RAS Core Research Note G00155811, John Pescatore, 4 May 2008, R2742 05232009

Organizations with the technical capabilities to buy and use

penetration-testing tools can reduce risk and gain business
benefits. Penetration-testing services can be used by organizations
that do not have the expertise or personnel available.
Key Findings
• Penetration testing provides a level of risk assessment above vulnerability scanning, but it
requires investment in technical expertise and regular use to provide security benefits.
• Penetration tests should be formally scheduled and augmented with ad hoc testing, driven
by changes in the IT, business or threat environments.
• There is the risk of affecting business systems during penetration testing, but this risk can
be minimized by thorough preparation and planning.
• Open-source, penetration-testing tools are available for organizations that do not have the
budget to buy commercial products; however, the open-source products will require
significantly higher levels of staffing.

Recommendations

• Penetration testing should be part of a vulnerability management program for all

enterprises that may be targeted by financially motivated attacks.
• Enterprises that already buy and use vulnerability assessment products with well-defined
vulnerability assessment processes are candidates for augmenting those processes with
penetration-testing tools.
• Enterprises that outsource vulnerability assessments should look to penetration-testing
services, not products.
2
WHAT YOU NEED TO KNOW
The growth in financially motivated, targeted threats has driven the
need to augment standard vulnerability assessment efforts with
penetration testing. For many organizations, using external
consultancies to conduct periodic penetration tests will be the
most appropriate approach. Organizations with sufficient technical
capabilities and mature vulnerability assessment programs, and
those that use commercial or open-source, penetration-testing
tools, can provide higher levels of security and quicker reactions to
changing threat environments.
CASE STUDY
Introduction
Company A is a global manufacturer in the power/energy and
transportation fields. It has an annual turnover of approximately
$20 billion, with 70,000 employees in 70 countries. The transport
business unit hired its first full-time information security person (its
IT security director) in 2005. In late 2007, this position evolved into
the security architect position, responsible for security strategies
for the entire company.
The Challenge
When the new security architect came on board in 2005, he knew
he needed a way to quickly baseline the actual security status of
the organization’s IT systems and networks. He considered
bringing in an outside consultancy to perform a security audit and
penetration test, but that would cost around $50,000 for a one-
time analysis. The security architect felt he needed to create a
process that would remain in use during the long term, and
procure tools to implement that process. Corporate management
supported him and approved funding.
Approach
The transport group had been using vulnerability assessment
scanning tools, but the security architect felt those tools resulted in
“information overload,” with long lists of vulnerabilities but no
prioritization by the likelihood that a vulnerability could actually be
accessed and used to cause damage to mission-critical systems.
He also knew that threats on the Internet were getting more
© 2008 Gartner, Inc. and/or its Affiliates. All Rights Reserved. Reproduction and distribution of this publication in any form without prior written permission is
forbidden. The information contained herein has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy,
completeness or adequacy of such information. Although Gartner’s research may discuss legal issues related to the information technology business, Gartner
does not provide legal advice or services and its research should not be construed or used as such. Gartner shall have no liability for errors, omissions or
inadequacies in the information contained herein or for interpretations thereof. The opinions expressed herein are subject to change without notice.
complex and targeted, and believed that “hacker in a box” testing
approaches were needed. He had experience with open-source,
penetration-testing tools such as The Metasploit Project,
Immunity’s Canvas and early versions of Saint’s Exploit, and
believed acquiring penetration-testing tools would provide the
security baseline visibility and a more targeted presentation of what
the most-critical vulnerabilities were.
The security architect felt that buying a commercially supported
product would be the best way to go, rather than using open-
source tools, because of the need for management, reporting and
support capabilities. He surveyed the security market and came
across the Core Impact product from Core Security Technologies.
He evaluated the product’s capability, did not see any other
commercial products at the time with equivalent capabilities and, in
2006, acquired the Core Impact penetration-testing product.
Although the security architect did not do a competitive evaluation,
other commercial products are available (see Note 1).
Results
Once the procurement was processed, it took a few days to obtain
and install the software, and take a half-day training class from
Core Security. The security architect is the primary user of Core
Impact and initially ran penetration tests on a near-daily basis to
become familiar with the tool and to create the initial security
baseline. He has now settled into running the tool quarterly to
support regular reporting, plus additional runs after any major
change on the network or upon special request. The typical
penetration-testing run is done against 40,000 nodes on Company
A’s network. The results are provided to the IT compliance officer
for compliance reporting, and to the IT operations group for
patching and other configuration mitigation efforts.
The security architect warns the operations staff when penetration
tests will be launched to avoid any appearance of an actual
incident that might cause business disruption if incident response

Note 1
Penetration-Testing Tools
The major commercial penetration-testing products available
today include:
• Core Security Core Impact (www.coresecurity.com)
• Immunity Canvas (www.immunitysec.com)
• Saint Exploit (www.saintcorporation.com)
Rapid7’s NeXpose (www.rapid7.com) has many of the
features of a penetration-testing tool, but it is more of a
broad vulnerability-scanning product.
The major open-source, penetration-testing tools include:
• BackTrack (www.remote-exploit.org)
• The Metasploit Project (www.metasploit.org)
3
actions were initiated. During the course of using the Core Impact
product for a year, the security architect has only had one server
experience a production disruption caused by penetration testing –
a server on a high-speed network that was sensitive to workload
disruption. The security architect was able to easily tune the tests
to avoid any future disruptions.
Although production disruption, in this case, was minor, it serves
as an example of the need for testing and phased production
deployment. The security architect believes the detailed effects of
information provided by penetration-testing tools is helpful in
convincing IT and corporate management when mitigation actions
need to be prioritized.

The SecurityForest Exploitation Framework

(www.securityforest.com) also has penetration-testing
capabilities, but does not seem to be actively supported.
4
Critical Success Factors
• The security architect’s experience with penetration-testing
tools enabled him to rapidly initiate a penetration-testing
program and minimize how much time it takes each quarter to
run the tests and interpret the results.
• Information provided by penetration-testing tools makes it
easier to convince management when immediate or costly
security actions need to be taken.
• Minimizing disruption to production systems requires some
ongoing effort, but it pays off.
Lessons Learned
• Being able to explain the impact of vulnerabilities to the
business makes it easier to gain support for necessary
changes.
• The risk of trying to do surprise penetration tests outweighs the
benefits.
• By using a controlled, repeatable process, penetration testing
can provide increased value.