VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

FEBRUARY 2006

QualysGuard

www.westcoastlabs.org

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Contents

QualysGuard

Test specifications ....................................................................................3 Vulnerabilities ..........................................................................................6 The product ..............................................................................................5 Test report ................................................................................................10 West Coast Labs conclusion ..................................................................20 Security features buyers guide ............................................................21 Appendix..................................................................................................22

West Coast Labs, William Knox House, Britannic Way, Llandarcy, Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001. www.westcoastlabs.org
www.westcoastlabs.org

QUALYSGUARD

Test specifications

The aim of this Technology Report is to evaluate solutions in the field of Vulnerability Assessment. Participants in the report may include online services, appliances and software tools. TEST ENVIRONMENT Participants in the technology report were invited to provide a vulnerability assessment of a heterogeneous network, together with proposals and recommendations for remediation. The network set up by West Coast Labs for evaluation of solutions comprised 24 distinct hosts, including routers, managed switches, network servers and client machines. Web applications were installed on relevant servers. A variety of Operating Systems were used on the network, on different hardware platforms. A small number of virtual hosts were included. In building the network, some of the servers were installed with default settings. Various levels of patching were applied. In addition a number of common misconfigurations were made in setting up the servers, and in deploying particular services. Every host on the test network was imaged, and restored to its start state before each round of testing for individual solutions. The test network was protected by a router. ACLs were set on the router to restrict access to the test network from IP addresses specified by the participating vendor, if appropriate. Where the solution under test was an appliance or software solution then the router was configured to block all access from the internet for the period of test. The test network was available to each solution for 2 days. The final report, containing the results of the Vulnerability Assessment and any recommendations are addressed in the Test Results that follow.

www.westcoastlabs.org

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Test specifications (continued)

Appliances were provided to WCL in the default shipping state. WCL engineers configured appliances in accordance with documentation provided. Software solutions state the desired specification and OS of the hardware on which the software is to be installed. WCL engineers installed and configured software in accordance with documentation provided. All participating solutions were provided together with documentation supplied to a normal user. WCL evaluation of the Vulnerability Assessment Report Vulnerabilities on the target network were classified under 4 headings: CRITICAL VULNERABILITIES those that allow an attacker with minimal knowledge or skill to compromise the integrity of the network. This may include gaining control of a server or network device, gaining illegitimate access to network resources or disrupting normal network operations. SEVERE VULNERABILITIES those that allow illegitimate access to, or control over, network resources, but that require considerable knowledge or skill on the part of the attacker. NON-CRITICAL VULNERABILITIES those that allow attackers to gain access to specific information stored on the network, including security settings. This could result in potential misuse of network resources. For example, vulnerabilities at this level may include partial disclosure of file contents, access to certain files on hosts, directory browsing, disclosure of filtering rules and security mechanisms. INFORMATION LEAKS these allow attackers to collect sensitive information about the network and the hosts (open ports, services, precise version of software installed etc.)

www.westcoastlabs.org

QUALYSGUARD

Test specifications (continued)

Each product was assessed on: The ease of deployment of the solution The number of vulnerabilities correctly identified in each class The completeness of the report, including identification of any network changes made The clarity of presentation of the findings The clarity of advice on remediation WCL also comments on the level of technical knowledge required to understand and act on the information contained in the final report. Participants in the Technology Report will be eligible for the Checkmark certification for Vulnerability Assessment. In order to achieve the Standard Checkmark Certification, the candidate solution must identify at a minimum 100% of the Critical Vulnerabilities and 75% of the Serious Vulnerabilities. However, those developers identifying 100% of the Critical Vulnerabilities and a minimum 90% of the Serious Vulnerabilities will be awarded the Premium Checkmark Certification for Vulnerability Assessment. All solutions must also provide accurate advice on mitigating the risks posed by the vulnerabilities.

www.westcoastlabs.org

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Vulnerabilities

So that the test network would mirror that found in many businesses, a variety of operating systems, on different hardware platforms, were included. A Windows domain was set up with three servers and a mix of workstations running Windows XP and Windows 2000 professional. Some Sun Servers running Solaris 2.8 provided web services and file storage, assorted Linux boxes running Mandrake and RedHat distributions, and a Mac completed the mix. Some of the servers were installed with default settings and varying levels of patching were applied: some hosts were patched fully up to date while others had been left out of the process. Also, a number of common misconfigurations were made in setting up servers, and deploying particular services. For example, Windows servers were configured with open network shares, ftp servers with anonymous write access, smtp servers configured as open proxies. These are configuration errors that can have profound effects on network security but can easily be implemented by a hard-pressed administrator as a temporary quick fix to a connectivity problem. On the Windows 2000 PDC we installed TightVNC as a service without tunnelling through SSH, SQLServer with a blank SA password, Active Directory, and IIS 5.0 with the demo applications. The BDC had Exchange 2000 and Active Directory installed. DNS was provided by the remaining Windows 2003 server. DNS was configured to allow zone transfers. In addition, IIS5.0 was installed with demo applications, and a vulnerable web application that was specially crafted in-house. The server was also running Unreal Tournament GOTY edition (version 436) along with the UT web interface running on an unusual high port. There were user shares available on the wwwroot and ftproot directories and a world-writable FTP server. One of the Sun Blade servers had a Virtual Learning Environment (VLE) installed. The VLE had a default admin username and password as well as being installed with an old
www.westcoastlabs.org

QUALYSGUARD

Vulnerabilities (continued)

version and vulnerable version of Apache. Vulnerabilities included SSH access, Apache installations, Samba and a writable FTP directory. Each of the user workstations was patched to a different level using official Microsoft Service Packs, historical patches and Windows Update. These machines then had different applications installed, ranging from Unreal Tournament client and TightVNC through to IIS 5.0 and remote admin. Some machines were included in the Windows Domain. Back Orifice was installed on one machine on a high end port. An HP printer was added with default settings and open to administrative access via telnet and HTTP, a Cisco router configured with default settings, default username/password and open web admintool and an Apple Mac Power G3 running OS 8.6. If changes were made to the default settings, over all these devices passwords were set to be blank or easily guessable. Our test network thus consisted of a series of machines with differing hardware specifications, operating systems, patch levels, and software installations, and multiple vulnerabilities.

www.westcoastlabs.org

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

The product

QUALYSGUARD ENTERPRISE Qualys describe QualysGuard Enterprise as a scalable vulnerability management solution, which enables organizations to measure and reduce risk by providing a proactive solution to track and remediate security vulnerabilities used for exploitation. According to CERT, 99% of attacks exploit known vulnerabilities. QualysGuard Enterprise is an enterprise class, on demand solution, which is positioned by Qualys as being suited to large, distributed networks that require support for an unlimited number of IPs, appliances for internal scanning and users with hierarchical authorization rights. Qualys sayabout the product... QualysGuard enables security managers to strengthen the security of their networks effectively, conduct automated security audits and ensure compliance with internal policies and external regulations - with no infrastructure to deploy or manage. www.qualys.com

www.westcoastlabs.org

QUALYSGUARD

The product (continued)

Qualys sayabout the QualysGuard Business Benefits. QualysGuard on demand platform gives users an automated way to map global assets, identify vulnerabilities on their networks, prioritize remediation according to business risk and achieve regulatory compliance - with no infrastructure to deploy or manage. QualysGuard gives organizations the ability to mitigate risks by automating the proactive identification and prioritized remediation of security vulnerabilities based on risk to business operations and to ensure regulatory compliance via automated auditing, indelible audit trails, plus the validity and assurance that comes with third-party assessment. The on demand architecture offers significant economic advantages with no capital expenditures, extra human resources or infrastructure to support and maintain. www.qualys.com Qualys sayabout the QualysGuard Technical Benefits... QualysGuard allows organizations to audit their networks with the highest degree of accuracy, data integrity and ease of use while delivering the lowest total cost of ownership. Companies receive daily updates about new security vulnerabilities, full security trending reports, and access to verified remediesall without the cost and burden of deploying and maintaining complex software. QualysGuard has the most comprehensive KnowledgeBase of vulnerability signatures in the industry (5,000+), and performs over 6 million scans per month with a 99.997% accuracy rate. Its immediate deployment capabilities and strong security model enables security teams to perform scans on geographically distributed and segmented networks both at the perimeter and behind the firewall. www.qualys.com

www.westcoastlabs.org

10

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Test report

Introduction
QualysGuard is a Vulnerability Assessment tool aimed at large distributed networks. It consists of a series of one or more Scanner Appliance devices placed within the corporate network. These are accessed and scans are launched via a web-based management tool. The compact hardware arrived at West Coast Labs with a Quick Start Guide, Administrators Guide, a Rackmount kit, power and Cat5 cables, and a set of documents relating to the latest news and the regulatory compliance of the device.

www.westcoastlabs.org

QUALYSGUARD

11

Test report

Installation and Configuration

The installation was a straightforward three-stage process. After having been provided with login credentials to the web interface, the set up of the hardware following the clearly formatted manuals proved to be simple networking can be set up using the LCD screen and navigational buttons on the fascia of the unit. These keys are responsive and do not have the problem of key lag common to this kind of interface. The product then needs to be activated by first logging into the web application and then into the unit itself.

www.westcoastlabs.org

12

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Test report

The Interface
The web application is the main interface point of the solution and has been designed with ease of use in mind. The interface is far from utilitarian, however, and has an understated elegance that serves it well. Upon logging in, the user is presented with a selection of the latest vulnerabilities from the knowledgebase with relevant information such as category, Bugtraq ID and, crucially, the severity. These knowledgebase entries are updated regularly so that the users can be assured that the functionality of the solution is as up to date as possible. Basic help is provided at this point via a pop up window that offers a helpful set of pointers that are in the form of a QuickStart Guide. Each of the links offered here direct the user to the relevant section of the interface and the overall layout provides a suitably structured introduction to performing asset maps and vulnerability scans. Further help can be accessed at any point during the use of the interface either by the main Help link at the top as part of the general menu structure, or via the Quick Help button that appears on every page. These serve different functions, as the Quick Help relates specifically to the screen that the request is made from whereas the main Help is more general and covers use of the entire interface, split into chapters in a similar format to the standard Windows help files.

www.westcoastlabs.org

QUALYSGUARD

13

Test report

The Interface (continued)

The main menu for the system consists of several sections: Home, Map, Scans, Reports, Remediation, and Preferences with further links for Support and Help. When attempting to discover a networks liabilities, the starting point for any new customer should be the Map section. This allows for discovery scans to be made by IP range or domain name. A quick process to set underway, the interface makes it as simple as possible by guiding the user through it in stages. A screen within the set up of each Map process includes a tick box that must be checked to ensure that the user has the legal right to scan the IP range entered, and the scan can then be undertaken. Length of the scan will obviously depend on the number of hosts within the range to be scanned, but notification emails can be configured so that an administrator is aware of the successful completion of this phase. Once the scans have completed they can be viewed either as text or in a diagrammatic format. The former is in list format with tick boxes along side each entry to enable selection of each asset for scanning or insertion into Asset Groups, whilst the latter is a well thought out and presented interactive view. This signed java applet allows the user to drag individual devices around the interface in order to lay the target network out in different arrangements to best suit the presentation and can also be used to launch scans against individual devices.
www.westcoastlabs.org

14

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Test report

The Interface (continued)

Once the devices on the network have been discovered via mapping, they may then be scanned either as individual concerns or as Asset Groups. The building of a scan is initiated from the Scans menu and has been made incredibly easy by using similarity to the mapping interface. Again, it functions by specifying an IP range or asset group, the scanner to be used (whether the default for the group, the internal appliance, or Qualys own scanners), a title and an option profile for the scan. Three initial profiles are provided: Initial (default), SANS top 20 and RV-10. It is easy to alter the settings for each of these scans if so desired, or custom scans can be constructed using a link found under the Preferences menu. This allows the user to specify a title, and then various sets of options for Scanning or Mapping including the levels of scanning for TCP and UDP ports, whether to scan dead hosts, performance levels, load balancing detection, the degree of brute password forcing to apply, the different types of vulnerability detection and various types of authentication to try. There are also sets of advanced options that relate to corporations that use IDS systems and some further options related to the types of packets sent.

www.westcoastlabs.org

QUALYSGUARD

15

Test report

The Interface (continued)

Using Qualys own appliances rather than an in house device is a viable option for scanning external IP addresses, however private IP ranges may only be scanned if there is a Scanner Appliance in place within the network. The choice of which scanner to use makes it possible to split the workload so that one device does not necessarily become overloaded with requests, and it is possible to set a default Scanning Appliance for a given Asset Group to make this process even easier. Once the scans have been executed, an administrator may choose to receive a notification by email. This is turned on by default and may be disabled under the Preferences menu. The email contains a link to the online report as well as a summary of the principle points of concern. These contain the assigned title of the scan, the start time and duration, target groups, number of hosts scanned and number of hosts active, option profile and the user that initiated the scan. Further to this there are summaries of discovered Vulnerabilities, Potential Vulnerabilities, and Information Gathered. These are grouped by severity, and a differential figure between the current scan and the last scan for each level of severity is also provided.

www.westcoastlabs.org

16

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Test report

Reports
The online reports are well formatted and are available in several flavours, from an executive summary for non-technical management to a technical report including recommended resolutions to give to the corporate IT staff. For those reports that cover a large range of assets there a drop down menu at the top of each generated report allowing the user to see a summary of vulnerabilities, groups of assets broken down by IP address, or both. Each report may be saved in several different formats: PDF, XML, zipped HTML, or an MHT web archive are available and may be downloaded to a local machine. The majority of the reports are taken up by the descriptions of the vulnerabilities and their remediation organised on a per asset basis. The Summary section for each report, however, consists of clearly presented graphical representations of the severity of vulnerabilities, operating systems detected, and services detected along with a textual synopsis to back these images up. The section of the report given over to Detailed Results contains three major sections Vulnerabilities, Potential Vulnerabilities and Information Gathered. The description of each liability contained within the reports is given a severity rating between one and five and is organised in order of most dangerous and highest rated first. This gives a corporate IT department the ability to tackle the most important problems immediately but it is important to be aware that the other vulnerabilities should not be overlooked just because they come lower down the scale.
www.westcoastlabs.org

QUALYSGUARD

17

Test report

Reports (continued)
Each report also includes a detailed data on a per vulnerability basis. This data includes similar data to the knowledgebase entries seen upon first login such as BugTraq ID, CVE ID and category. Alongside this there is an assessment of the threat of each, an impact evaluation that describes how the vulnerability may be exploited, remediation advice that includes links to external web sites where appropriate, and a Result section that shows returned values if appropriate. Alongside each description there is also a status for the vulnerability and a drop down menu allowing the administrator to either ignore the vulnerability or create a ticket using the inbuilt ticketing system. This allows the administrator to assign the remediation of the problem to any user registered on the system and set a deadline as well as provide some descriptive text to accompany the ticket. The ticket itself consists of data regarding the assignation of the ticket, the vulnerability details taken from the report, and a section for the user to add further comments and apply actions such as resolving or reassigning if their permissions allow. These tickets can then be viewed under the Remediation section of the interface that offers a variety of filters that can be applied to the tickets. These filters include user, asset, date range, status, vulnerability and severity and allow for a detailed summation to be constructed of the current state of remediation across the network.
www.westcoastlabs.org

18

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Test report

Reports (continued)
The Preferences section covers various areas including as previously noted the construction of custom scan parameters. It is possible to alter assets by assigning them to specific users and changing the way they are tracked via IP address, DNS host name or NetBIOS host name. It is also easy to adjust Asset Groups in various ways including organising them into Business Units. Scans may be scheduled from within this section of the interface so that a long scan that may potentially interfere with network traffic can be set to run overnight on a one-off or regular basis. User permissions are also set and assigned here. There are several levels of users from Manager down to Contact and each may be assigned the responsibility for different asset groups. The level of interface interaction that a user gets depends upon their permissions and certain tabs and sections within the QuickStart Guide are not available to given levels of user privileges. There is also the possibility within this section of the interface to look at usage logs for the interface. These include date and time, action (such as logging in to the interface or launch / completion of a scan), the user to which the entry refers, the IP address of login, and their role. This gives a good way of tracking access to the system from different locations and users.

www.westcoastlabs.org

QUALYSGUARD

19

Test report

System Oversight
Oversight of the entire system comes from the section labelled Home. This contains the Knowledgebase that is displayed upon initial login, but also includes the Dashboard. This is a useful overview of the vulnerabilities, open tickets by severity level, top ten open tickets and top ten vulnerabilities. Further information is provided by another screen called Account Info - this includes details of the latest scans run and when the next scheduled scans are due. There is also a link to email the assigned Qualys contact for the corporation and the number of IP addresses registered in the corporate account. This section also includes various version numbers including the Web Application. The scanner operating system version and signature database version are provided for both Qualys external scanners and for any internal Scanner Appliances that are registered, in order to ensure that the latest versions are available. It is also possible from within the Home interface to run a Risk Matrix report mapping given vulnerabilities against assets or Asset Groups this is a useful tool allowing newly released vulnerabilities to be run against registered devices for an instant risk assessment. Finally, there is a section called Resources that contains release notes, access to support documents and a group for Tips and Techniques.

www.westcoastlabs.org

20

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

West Coast Labs Conclusion

QualysGuard is a comprehensive vulnerability assessment and remediation solution. The installation and set up is well documented and trouble-free and the interface offers a deceptively simple user experience. The quality of scans and subsequent remediation advice is paramount in solutions of this nature, and QualyGuard delivers admirably. During our testing QualysGuard detected all of the Critical and the majority of the Serious vulnerabilities with ease, and we are therefore pleased to announce that QualysGuard has been awarded the Vulnerability Assessment Premium Checkmark. The ability to assign tickets within the interface ensures that administrators and those responsible for the security of a corporate network can keep on top of the workload and have information at their fingertips whenever it is needed. From design to user interaction, QualysGuard offers everything necessary for the user to improve the security of their network in a very short time frame. This solution should be considered by any corporation looking to mitigate the risks to their network through a thorough liability detection system.

West Coast Labs, William Knox House, Britannic Way, Llandarcy, Swansea, SA10 6EL, UK. Tel : +44 1792 324000, Fax : +44 1792 324001. www.westcoastlabs.org
www.westcoastlabs.org

QUALYSGUARD

21

Security features buyers guide as stated by Qualys

Unlimited number of Network Maps Unlimited scanning of servers and workstations 24x7 email and telephone Customer Support Scheduled and on-demand Security Audits VPN and wireless access point scanning Remediation workflow management with automatic trouble ticket creation Executive summary reports for managers Detailed technical reports Vulnerability ticket reporting with full remediation instructions Differential reports with trending graphs Differential network inventory reports Built-in PCI compliance reports for self certification Full remedy information for each vulnerability Distributed Scanning with centralized data repository for reporting Ability to create multiple users with flexible access privileges for distributed management API/SDK capabilities for automation and integration with other security products Internal and external scanning provides a 360-degree view of network vulnerabilities CVE, CVSS and OVAL standards support Automatic, daily updates to vulnerability KnowledgeBase (over 5000 unique checks) 100% non-intrusive detection techniques Inference-based scanning engine optimized for speed and bandwidth efficiency Scans configurable for optimum performance Both trusted and non-trusted scanning capabilities Scans configurable for optimum performance Six-Sigma scanning quality Export reports to HTML, MHT, PDF and XML formats Executive Dashboard to track progress and enforce compliance End-to-end encryption of vulnerability data Immediate deployment capabilities www.qualys.com
www.westcoastlabs.org

22

VULNERABILITY ASSESSMENT SOLUTIONS TECHNOLOGY REPORT

Appendix

Vulnerability Assessment Premium Level Certification

Within the framework of the testing carried out in this Technology Report, those developers identifying 100% of the Critical Vulnerabilities and a minimum 90% of the Serious Vulnerabilities are awarded the Premium Checkmark Certification for Vulnerability Assessment. http://westcoastlabs.org/cm-briefingdocs.asp

www.westcoastlabs.org