Вы находитесь на странице: 1из 47

Operations Run Book

Enter Clients Name Here

Prepared By:
Version:
Publication Date:

Managed Services, Champion Solutions Group


1.00
02/08/15 7:58:44 AM

Copyright 2005 Champion Solutions Group


All rights reserved. The information contained in this document is the proprietary information of Champion Solutions Group and may not
be used, duplicated, or disclosed except for its intended purpose. All company or product names mentioned are used for identification
purposes only, and may be trademarks of their respective owners.

Operations Run Book


2/8/2015 7:58:44 AM

DOCUMENT IDENTIFICATION INFORMATION


Document Name:

Operations Run Book for Enter Clients Name Here

Version:

1.00

Date Created:
Created By:
Date Published:

October 3, 2005, 2:05 PM

Security Classification:

Restricted for use by IBM/CSG Managed Services Clients

Creation Software:

Microsoft Word 2003

Contributors:

MSOC Team

CHANGE HISTORY
Ver.

Date

Change Description

0.01

Initial draft for review by MSOC team

1.00

Initial version for publication

Approval

DOCUMENT REVIEW
Name/Title

Signature

Date

DISTRIBUTION LIST
IBM Managed Services
Champion Managed Services
All IBM/CSG Managed Services Clients
DOCUMENT LOCATION
This document is available via the Champion Portal at https://www.championpulse.com.
Clients Final:

The Clients copy is stored on the CMS portal under their specific document area.

Template

To obtain the internal template from the CMS portal, please click here, or enter
the following URL in your Web browser:

(CMS Use Only):

https://portal.championpulse.com/C15/MSOCPoliciesNProcedures/Templates/Op
erations_Run_Book.doc.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - i

Operations Run Book


2/8/2015 7:58:44 AM

Table of Contents
Introduction........................................................................................................................................... 1
Contact Information............................................................................................................................... 1
Champion.......................................................................................................................................... 1
IBM.................................................................................................................................................... 1
Escalation Process............................................................................................................................ 2
Champion Group / IBM Web Portal................................................................................................... 3
Champion Group / IBM Phone Support............................................................................................. 3
Infrastructure......................................................................................................................................... 3
Facility Overview............................................................................................................................... 3
Site restrictions.............................................................................................................................. 3
Fire and emergencies.................................................................................................................... 3
Shipping To The Facility................................................................................................................ 3
Directions...................................................................................................................................... 4
Shared Common Areas................................................................................................................. 4
Hardware Configuration.................................................................................................................... 5
Operating Procedures Overview........................................................................................................... 6
Introduction....................................................................................................................................... 6
Remote Accessibility......................................................................................................................... 7
Purpose......................................................................................................................................... 7
Scope............................................................................................................................................ 7
General Policy............................................................................................................................... 7
Requirements................................................................................................................................ 7
Enforcement.................................................................................................................................. 8
Operating System Permissions......................................................................................................... 8
Purpose......................................................................................................................................... 8
Scope............................................................................................................................................ 8
Policy............................................................................................................................................. 8
Ownership and Responsibilities.................................................................................................... 8
General Configuration Guidelines................................................................................................. 9
Compliance................................................................................................................................... 9
Server Setup..................................................................................................................................... 9
Overview....................................................................................................................................... 9
Scope............................................................................................................................................ 9
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - ii

Operations Run Book


2/8/2015 7:58:44 AM

Policy............................................................................................................................................. 9
Ownership and Responsibilities.................................................................................................. 10
Backup Configuration...................................................................................................................... 11
Software...................................................................................................................................... 11
Policies........................................................................................................................................ 11
Data Restoration Process............................................................................................................... 12
Purpose....................................................................................................................................... 12
Overview..................................................................................................................................... 12
Incremental Backups................................................................................................................... 12
Incremental Restores.................................................................................................................. 12
Database Restores...................................................................................................................... 12
Tape Handling and Retention.......................................................................................................... 12
Retention Policies........................................................................................................................ 12
DRM Tape Handling.................................................................................................................... 12
Responding To Alerts...................................................................................................................... 14
Change Management.......................................................................................................................... 14
Types Of Change Requests............................................................................................................ 14
Severity And Priority........................................................................................................................ 15
Change Request Classifications...................................................................................................... 15
Scheduled Client Change............................................................................................................ 16
Scheduled CSG Change............................................................................................................. 16
Emergency Changes................................................................................................................... 16
Who Is Authorized To Request A Change?......................................................................................16
How Is A Request Submitted?......................................................................................................... 16
Scheduled Maintenance Windows.................................................................................................. 17
Change Control Board..................................................................................................................... 17
Decision Categories........................................................................................................................ 17
Turnaround Time............................................................................................................................. 17
Problem Management......................................................................................................................... 17
Business Hours............................................................................................................................... 17
Contacting Support (Champion MSOC).......................................................................................... 17
Telephone.................................................................................................................................... 17
Champion Portal.......................................................................................................................... 17
Severity And Priority Levels............................................................................................................. 17
Trouble Ticket Workflow.................................................................................................................. 20
Monitoring Standards.......................................................................................................................... 21
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - iii

Operations Run Book


2/8/2015 7:58:44 AM

PURPOSE....................................................................................................................................... 21
REFERENCE.................................................................................................................................. 21
OVERVIEW..................................................................................................................................... 21
Thresholds...................................................................................................................................... 21
IIS Services................................................................................................................................. 21
SQL Server 2000......................................................................................................................... 23
Storage Area Network Switches.................................................................................................. 24
Webservers................................................................................................................................. 24
Network Intrusion Detection System............................................................................................... 24
Purpose....................................................................................................................................... 24
Scope.......................................................................................................................................... 24
General / Policy........................................................................................................................... 24
Enforcement................................................................................................................................ 24
Operating System Patches / Service Packs....................................................................................25
Microsoft...................................................................................................................................... 25
AIX.............................................................................................................................................. 25
Linux............................................................................................................................................ 25
Disaster Recovery........................................................................................................................... 25
Appendix A Windows Server Security Checklist.............................................................................26
Linux Security Checklist.............................................................................................................. 33

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - iv

Operations Run Book


2/8/2015 7:58:44 AM

Introduction
Welcome to the Managed Services Operations Center (MSOC) for Champion Solutions Group (CSG).
As an IBM business partner, the MSOC has been established for the purpose of providing managed
services for customers. This document serves as a centralized repository for all policies, procedures,
and supporting documents that are associated with the day-to-day operations of the MSOC. The
administrators and engineers are provided the ability to quickly and easily navigate to documentation
needed to perform assigned duties accordingly.

Contact Information
Champion
Managed Services Operations Center (To submit a request for service)
Telephone:

(888) 997-7789

Web Portal

https://www.championpulse.com

IBM
Name & Title

Telephone

E-mail

Enter PM Name Here

Enter PMs Phone # Here

Enter PMs E-mail Here

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 1

Operations Run Book


2/8/2015 7:58:44 AM

Escalation Process
The escalation process describes the information flow in case of non-compliance with minimum
service levels. This escalation process applies to severity 1 calls only.
The following escalation sequence is to be utilized if a service is not delivered in a specific timeframe.
ESCALATION

15 Minutes

MSOC Call Center


888-997-7789

30 Minutes MSOC Supervisor,


Eric Schneider
561-251-6240
45 Minutes

AND

IBM Project Manager,


Enter PM Name Here
Enter PMs Phone # Here
Enter PMs E-mail Here

AND

IBM Project Executive,


Enter PEs Name Here
Enter PEs Phone Here
Enter PEs E-mail Here

MSOC Manager,
Jay Kobert
954-646-2784

1 Hour President, Managed


Services
Ian Sutcliffe
561-997-2900, XT 262

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 2

Operations Run Book


2/8/2015 7:58:44 AM

Champion Group / IBM Web Portal


Champion Managed Services has designed a web portal called The Pulse, for our customers
intended to provide various types of information, such as but not limited to:
1) Customer Infrastructure Documentation
2) Procedures/Processes
3) System Monitor Tools
4) On-line Service Requests
Champion Managed Services Portal URL:
https://www.championpulse.com

Champion Group / IBM Phone Support


Customers can directly contact the Managed Services Operations Center directly via the telephone
by dialing:
(888) 997-7789

Infrastructure
Facility Overview
The customers environment is maintained in the Champion Managed Services facility located in the
IBM Atlanta BellSouth eBHC (eBusiness). For the purpose of this document, we will refer to eBHC as
the facility.
The facility maintains several security features for your protection. Security technology may include
biometric readers, cyberlocks, and interior and exterior motion-activated video surveillance cameras
in selected areas.

SITE RESTRICTIONS
Smoking is not allowed in the facility. Unauthorized recording devices, including cameras and
video recorders, are not permitted.

FIRE AND EMERGENCIES


The center maintains a fire suppression system. Emergency announcements are made by the
facility manager. During a fire emergency, all visitors must report to the front parking lot and wait
for the Onsite Operations staff to give a fire status. Emergencies should be reported promptly to
the Onsite Operations staff.

SHIPPING TO THE FACILITY


Any request for shipments must be submitted through the Champion Managed Services
Operations Center (MSOC). The details for requesting service (submitting a ticket) are located in
the procedure titled Creating A Request For Service on Champion Managed Services portal
(https://www.championpulse.com).

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 3

Operations Run Book


2/8/2015 7:58:44 AM

Be prepared to provide the following shipping information to the MSOC when scheduling the
delivery:
Name of carrier
Way bill number

Expected date and approximate time of arrival


Number of packages

Approximate weight and dimensions

Specific handling instructions

If the shipment is going to be delayed, contact the MSOC to modify the shipping information.
All carriers must be instructed that all deliveries must indicate Inside Delivery.
Shipments must be addressed to:
BellSouth c/o IBM Site Manager
Customer name/identifier
BellSouth Trouble Ticket Number
675 W. Peachtree Street NW
Atlanta, GA 30308-1989

DIRECTIONS
The address is:
675 W. Peachtree Street NW
Atlanta, GA 30308-1989
From Atlanta Hartsfield Airport
1. Follow the airport exit signs to Camp Creek Parkway.
2. Merge onto I-85 N toward I-75 N/ATLANTA.
3. Take the US-19/SPRING STREET exit (exit number 249D) toward US-29/W.
PEACHTREE STREET.
4. Take the ramp toward US-19/US-29 N/US-78/W. PEACHTREE STREET.
5. Turn SLIGHTLY RIGHT onto LINDEN AVENUE NW.
6. Turn LEFT onto W. PEACHTREE STREET NW.

SHARED COMMON AREAS


The facility has a common area located past the mantrap. The common area is shared by all
customers of the IBM e-business Hosting Center and has the following amenities:

Eating area

Vending machines

Coffee machine

Restrooms

Conference room

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 4

Operations Run Book


2/8/2015 7:58:44 AM

Hardware Configuration
Part No.

Qty.

Description

Copyright 2005 Champion Solutions Group


Operations Run Book

Server Name

Operating
System

Page - 5

Operations Run Book


2/8/2015 7:58:44 AM

Part No.

Qty.

Description

Server Name

Operating
System

Operating Procedures Overview


Introduction
Please note that the procedural content of this section is presented on a general, high-level basis.
Please refer to the Champion Managed Services portal (https://www.championpulse.com) for the
detailed, step=by-step procedures.
The following serves as an overview of policies, procedures, and supporting documents that are
associated with the day-to-day operations of the Managed Services Operations Center (MSOC). It is
made available to the administrators and engineers, and provides them with the ability to quickly and
easily navigate to the documentation that is needed to perform assigned duties accordingly.
Each procedure is structured to lead the engineer and management through steps to ensure the rapid
and efficient completion of a particular task. In addition to the steps, general overviews are provided
for clarity. After having completed a specific procedure several times, and have become familiar with
its background, you will be able to use the document as a reference guide and proceed directly to the
steps required.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 6

Operations Run Book


2/8/2015 7:58:44 AM

Remote Accessibility
PURPOSE
The purpose of this policy is to define standards for connecting to Champion Managed Service's
network and any hosted network environment that Champion manages from any host. These
standards are designed to minimize the potential exposure to Champion Managed Services, and
managed network infrastructures, from damages which may result from unauthorized use, outdated / insecure encryption methods, and unsupported methods of connection to Champion
Managed Services resources. Damages include the loss of sensitive or company confidential
data, intellectual property, damage to public image, infrastructure device and/or Operating system
configurations, and damage to critical Champion Managed Services internal systems.

SCOPE
This policy applies to all Champion Managed Service customers, customer clients, employees,
contractors, vendors and agents that require connection to the Champion Managed Service
network and customer-hosted network environments. Remote access implementations that are
covered by this policy include, but are not limited to, dedicated internet circuits, dial-in modems,
frame relay, ISDN, DSL, VPN, SSH, and cable modems, etc.

GENERAL POLICY
1. The following policies outline details about different methods of accessing network resources
via remote access methods, and acceptable use of Champion Managed Service's managed
networks:
A. Acceptable Encryption Policy
1) ESP-3DES
2) Hash / ESP Authentication = MD5, SHA, AES-128, AES-192, AES-256
3) D-H group = 2
B. Virtual Private Network (VPN) Policy
1) Site to Site = IPSEC
2) Remote Access VPN = PPTP (Microsoft Client)
2. Based on business and application requirements for administration; the following additional
methods are acceptable once a secure tunnel has been established or requests from the
customer with acknowledgement of their insecurities can be established.
A. Microsoft Terminal Services / Remote Desktop Protocol
B. Secure Shell
D. PCAnywhere
C. Telnet
E. RealVNC / VNC

REQUIREMENTS
1. Secure remote access must be strictly controlled. Control will be enforced via submission of a
change request through the MSOC from authorized personnel from the customer.
2. At no time should anyone provide their login or email password to anyone.
3. Customers must submit all encryption details with the specific source to destination for the
customer network. Details include the following: Peer IP Address, Pre-Shared Key, Specific
host / network to Specific host / network destination.
4. Firewall change requests must be submitted to the MSOC. No firewall change requests will
be completed without the completion of a Firewall Rule Request Form.
5. Frame Relay must meet minimum authentication requirements of DLCI standards.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 7

Operations Run Book


2/8/2015 7:58:44 AM

6. Non-standard hardware configurations and security configurations must be approved by


Champion Managed Services MSOC.
7. All hosts that are connected to Champion Managed Services managed environments and
networks via remote access technologies must use the most up-to-date anti-virus software.
This includes personal computers. Third party connections must comply with requirements.
8. Customer personal equipment that is used to connect to Champion Managed Service's
managed networks is not supported.

ENFORCEMENT
If any of the above requirements are not met, Champion Managed Services will be responsible
for damages that may be caused from the misuse of remote access policies. Service requests
that do not comply with the policies in this run book may be subject to rejection by Champion
Managed Services.

Operating System Permissions


PURPOSE
The purpose of this policy is to establish standards for the base configuration of server equipment
that is supported by Champion Managed Services. Effective implementation of this policy will
minimize unauthorized access to customers proprietary information.

SCOPE
This policy applies to server equipment supported by Champion Managed Services.

POLICY
Champion Managed Services recommends the utilization of the server security best practices
(see Appendix A). All server security considerations for application management will be defined
by the customer, which may or may not affect SLA availability credits.

OWNERSHIP AND RESPONSIBILITIES

All servers supported by Champion Managed Services are owned by the MSOC. The MSOC
is divided into technical verticals to ensure efficient problem resolution. The technical
verticals include:
o Network
o Server
o Storage
o

Data Management

Servers are registered within Champions enterprise management system. At a minimum, the
following information is required to positively identify a given system:
o Server contact(s) and location, and a backup contact
o

Hardware and Operating System/Version

Primary functions and applications

Information in the enterprise management system is kept up-to-date.

Configuration changes for production servers follow the appropriate change management
procedures.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 8

Operations Run Book


2/8/2015 7:58:44 AM

GENERAL CONFIGURATION GUIDELINES

The latest security patches must be installed on all systems as business permits.

Trust relationships should be avoided whenever possible.

Always use standard security principles of least required access to perform a function.

If a methodology for secure channel connection is possible, privileged access must be


performed over secure channels, such as encrypted network connections using SSH or
IPSec).

Servers are physically located in an access-controlled environment.

COMPLIANCE

Audits will be performed on a regular basis by Champion Managed Services.

Audits will be managed by the internal audit group.

Server Setup
OVERVIEW
The purpose of this policy is to establish standards for the base configuration on server
equipment that is managed by Champion Managed Services. Effective implementation of this
policy will minimize server setup time and ensure stability across environments.

SCOPE
This policy applies to server equipment owned by the customer and managed by Champion
Managed Services. It defines the process of how the logical operating systems are setup in the
Champion managed environment and prepped to accommodate customer SOR.

POLICY
1. Champion Managed Services support and install the following operating system
manufacturers and versions.
A. Microsoft
i. Windows 2000 Server
ii. Windows 2000 Advanced Server
iii. Windows 2003 Standard Server
iv. Windows 2003 Web Server
v. Windows 2003 Enterprise Server
B. Linux
i. Red Had, versions 9.0, AS 2.1, and AS 3.0
C. AIX, version 5.1 or later
D. VMWare
i. ESX
2. Once a server has been integrated with all work order hardware resource allocations (such as
Processors, RAM, NICs, HBAs, etc.), the following processes are followed:
A. Surveys of hardware chassis light indicators are conducted prior to operating system
logical configuration.
B. Successful posts with bios confirmation of installed hardware on servers are conducted
prior to operating system logical configuration.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 9

Operations Run Book


2/8/2015 7:58:44 AM

C. Bios, RAID, etc. firmware versions are verified and updated with the latest available and
recommended versions from the manufacturer.
D. Disk partitions are built as necessary based on customer requirements regarding OS
partitions, and/or data partitions. (Pre-determined by customer and IBM/Champion
Managed Services technical teams prior to build dates.)
E. Servers are installed with the requested Operating System. (Pre-determined by
customer and IBM/Champion Managed Services technical teams prior to build dates.)
F. At the time of server operating systems installation, all available security and critical
updates from the OS manufacturer are applied unless otherwise agreed to in writing by
the customer and IBM/Champion.
G. IPs will be bound to the network interface cards with verified server name instances per
the customer requirements.
H. Operating system resources will be kept on the servers for necessary OS-based
applications. For example: i386 directory will be kept on a windows 2003 server root
drive unless customer requests otherwise.
I. For monitoring reasons, a local or domain account is created for perfmon statistics to be
used for proactive monitoring. The use of a local account or domain account is
determined by the projected network environment that the server will be participating in.

OWNERSHIP AND RESPONSIBILITIES


Champion Managed Services (CMS) maintains the server hardware and operating system
instances. All applications are the responsibility of the customer.
In order to proactively manage the environment, a standard maintenance window, between 3:00
AM and 6:00 AM Eastern Standard Time each Sunday, is reserved for internal, Champion
maintenance. The time at which Client-requested changes are made is determined during the
change request process.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 10

Operations Run Book


2/8/2015 7:58:44 AM

Backup Configuration
The policies discussed in this section are established standards, but can vary per a mutual
agreement to a customer request.

SOFTWARE
Server
TIVOLI Storage Manager (TSM) 5.x
Client
AIX, TSM Client Module 5.x
AIX, Agent Client 3.x
AIX, ORACLE (TDP Agent)
Linux, TSM Client Module 5.x
Linux, Agent Client x.x
Linux, ORACLE (TDP Agent)
Windows, TSM Client Module 5.x
Windows, Agent Client x.x
Windows, ORACLE ***
Licensed, installable physical media is required for ALL of the above.

POLICIES
Policies are rules that are set at the IBM Tivoli Storage Manager server to manage client data.
Policies control how and when client data is stored, for example:
How and when files are backed up and archived to server storage
How space-managed files are migrated to server storage

The number of copies of a file and the length of time copies are kept in server storage

The standard policy consists of a standard policy domain, policy set, management class, backup
copy group, and archive copy group. The attributes of the default policy are as follows:
Backup Policy
Daily incremental backups will be taken

An incremental backup is performed only if the file has changed since the last backup.
Up to two backup versions of a file on the clients system are retained in server storage.
The most recent backup version is retained for as long as the original file is on the client
file system. All other versions are retained for up to 30 days after they become inactive.

One backup version of a file that has been deleted from the clients system is retained in
server storage for 60 days.

DRM Policy

Duplicate copies of daily backups will be sent to offsite storage.

The offsite storage retention policy follows the backup retention policy above.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 11

Operations Run Book


2/8/2015 7:58:44 AM

Data Restoration Process


PURPOSE
The purpose of this section is to describe the steps necessary to restore data from backups.

OVERVIEW
In order to understand how to restore data from backups it is necessary to understand how
backups are organized. Backups are organized on all systems with the following concept. All
non-database related files are backed up under a nightly incremental backup. Database related
files are backed up either using the associated TDP (Tivoli Data Protection) agent, or cold
database backup.

INCREMENTAL BACKUPS
Tivoli Storage Manager (TSM) uses a incremental strategy where the first time an incremental
backup is run against the server everything is backed up. Everything means all data except
database related files, which are excluded in a list maintained in the TSM client path. After the
first backup of all files that have had their timestamp updated are incrementally backed up. This
incremental method ensures that a full restore is available at all times.

INCREMENTAL RESTORES
Incremental restores may be requested, and must include:

Source
Destination

Path
Date of source and path

Node

Estimated amount of data

DATABASE RESTORES
Oracle Database restores in an UNIX environment is handled by the customer via custom RMAN
scripts. The RMAN script connects to the TSM Server and opens channel(s) to tape drives and
passes the restore request between Oracle and TSM.

Tape Handling and Retention


RETENTION POLICIES
Policies are rules that are set at the IBM Tivoli Storage Manager server to manage client data.
Policies control how and when client data is stored, for example:

How and when files are backed up and archived to server storage

How space-managed files are migrated to server storage

The number of copies of a file and the length of time copies are kept in server storage

The standard policy consists of a standard policy domain, policy set, management class, backup
copy group, and archive copy group.

DRM TAPE HANDLING


Champion's Managed Service facility in Atlanta uses the services of Iron Mountain for the storage
of media offsite. This procedure explains how to send tapes from the Atlanta facility to off site
storage, and how tapes are retrieved from the off site storage vault when the retention policy
expires.
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 12

Operations Run Book


2/8/2015 7:58:44 AM

Champion Managed Services standardizes on IBM LTO2 tape technology.


The application that is used to manage these libraries is the Disaster Recovery Manager (DRM)
within the Tivoli Storage Manager (TSM), which enables these libraries. A copy is made of all
backups that are scheduled, which are controlled by the DRM.
The retention policies set in TSM controls the DRM process that:
1. Sends tapes off site, and
2. Requests tapes to be returned from vault storage.
Sending Tapes Offsite (Onsite-to-Offsite)
To send tapes offsite, the following tasks must be performed:
1. Copies are made of daily scheduled backups.
2. Disaster Recovery Manager (DRM) sets the state on these copies indicating they are
ready for offsite storage.
3. An automated process will run at 13:00 daily and move the DRMedia to the I/O trays of
the libraries.
4. An Onsite to Offsite Report will be e-mailed to Champion Managed Services Division and
the IBM On-site System Administration (OSSA) team daily at 13:00.
5. IBM OSSA team will verify the tapes in the I/O ports with the DRMedia Report upon
removal.
6. IBM OSSA team will transfer the DRMedia to Iron Mountain for offsite storage.
7. IBM OSSA team will provide a hard copy of the DRMmedia Report to the Iron Mountain
courier.
8. IBM OSSA team will fax a copy of the signed Iron Mountain report to Champion Managed
Service Division. The report indicates that the DRMedia has been received.
Retrieve Tapes From The Vault
To retrieve tapes from the vault (Iron Mountain), the following tasks must be performed:
1. A daily Vault Retrieve Report will be automatically generated at 13:00 and e-mailed to
Champion Managed Services Division and the IBM OSSA team. (See list above.)
2. Using the Iron Mountain online tool, the IBM OSSA team will make the appropriate
entries to order the tapes described in the Vault Retrieve Report. Iron Mountain will return
the requested DRMedia the following day.
3. Upon the receipt of the tapes from Iron Mountain, the IBM OSSA team will verify that the
DRMedia that is retrieved from the vault (Iron Mountain) matches from the previous days
Vault Retrieve Report.
4. IBM OSSA team will log all tapes received in the Daily Tape Transfer Log.
5. IBM OSSA team will send a copy of the Daily Tape Transfer Log to Champion.
6. IBM OSSA team will place the Vault Retrieve DRMedia in a storage cabinet, which is
provided by Champion.
7. Champion personnel will load the Vault Retrieve DRMedia into the tape libraries.
8. Champion personnel will run the DRM program which processes the DRMmedia volumes
into the proper state in the libraries.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 13

Operations Run Book


2/8/2015 7:58:44 AM

Responding To Alerts
1. Verify that the alert is a legitimate alert and not just a monitoring time out.
2. If it is during working hours, check with team to see if anyone is working on the system.
3. Create a ticket in heat.
4. Contact customer to make them aware of the alert and to see if they are doing anything to the
system. In the event that the alert is acknowledged by the customer as a valid alert, proceed
with the following:
Request from the customer if alert constitutes a priority one ticket.
In the event that it is a priority one ticket, contact the corresponding IBM Project
Manager and inform them of the status of the ticket.

Enter priority into Heat.

5. Notify the customer that you are creating a ticket and assigning it to an engineer.
6. Contact the appropriate engineering team to notify them of the alert.
NOTE: In the event of the alert being a Priority 1, maintain an hourly contact with the engineer
and customer with status updates until the alert is cleared.

Change Management
The Managed Services division of Champion Solutions Group (CSG) performs monitoring services for
Clients. These services are performed through CSG's Managed Services Operations Center (MSOC),
which is staffed around the clock by MSOC engineers whose primary responsibilities are to:

Monitor, maintain, and protect the MSOC, the Client's environment and networks, and the
equipment
Fulfill the Client's requests for modifications and problem reports as needed

Perform backups and other procedures in compliance with appropriate service level agreements

There are occasions or circumstances that require changes to the Client's environment or monitoring
needs. In order to maintain Client satisfaction by providing timely responses to such requests, this
section provides an overview of the change management policy, and offers guidelines for preparing a
change request.
For the steps required for accepting and monitoring a Client's change request, please refer to the
appropriate procedures that reside on the Champion Managed Services portal
(https://www.championpulse.com):

Creating A Request For Service

Change Control Process

Types Of Change Requests


Change requests may be submitted by the Client, IBM, or CSG. Depending on the service level
agreement, the types of changes cover the following environments:

Operating System
Storage

Copyright 2005 Champion Solutions Group


Operations Run Book

Network
Database

Page - 14

Operations Run Book


2/8/2015 7:58:44 AM

The Champion Managed Service Operations Center (MSOC) provides the Client with the ability to
submit service requests by utilizing the Helpdesk System on the MSOC Portal via the Internet, or by
telephone. It is recommended that priorities 1 and 2 should be submitted via telephone.
Either process provides the capability of submitting any of the three types of requests:
Administrative Request (AR) Does not affect production systems, and implementation times are
minimal. For example: new user, change password, change rights . . .
Problem/Fix Request (PF) This is similar to an administrative request except that you may be
experiencing some outage problems. For instance, a server or a network may be down.
Change Request (CR) A change request is associated with a change in the environment. For
example, if you have a piece of hardware that is already installed, but is not currently configured in
the infrastructure, a change request is required in order to make the hardware part of the steady state
infrastructure.

Severity And Priority


When you submit any of the three types of service requests, you are required to assign a priority
level, which identifies criticality of the problem and the level of support that is needed. The four priority
levels and their definitions are:
Priority 1:

Critical Impact/Urgent The product, service, or network is not usable or affects


the customer's core business. It is Champions objective to respond to all priority
1 requests within fifteen (15) minutes.

Priority 2:

Emergency Request The Client needs a quick response for reasons defined by
the requester. It is Champion's objective to respond to all priority 2 requests
within four (4) hours.
Reminder: Per contract, the Client is permitted only three priority 2 requests per
month, which are included in the Client's monthly fees. There will be
additional charges if the Client exceeds the limit of three priority 2
requests per month.

Priority 3:

Major Impact An important function or service is not available, however the


environment can still be used. It is Champion's objective to respond to all priority
3 requests within 48 hours and escalated appropriately.

Priority 4:

Minor Impact/Informational The product, service, or network is not seriously


affected and is not currently affecting the customers core business. Or, this level
of severity comprises shortcomings, suggestions, or questions. It is Champion's
objective to respond to all priority 4 requests within one week and escalated
appropriately.

Change Request Classifications


Change Management includes any type of change that affects the Client's environment. Examples of
this include emergency reboots, scheduled reboots, and other changes performed by either the Client
or CSG that affect the server, network, or storage during, or outside of, a normal maintenance
window.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 15

Operations Run Book


2/8/2015 7:58:44 AM

SCHEDULED CLIENT CHANGE


When changes are known in advance, the customer should notify the MSOC manager. The
MSOC manager begins the Change Management process by opening a ticket in Heat. The
MSOC manager is the primary contact for the Client and continues to monitor and track the
change request until it is completed.

SCHEDULED CSG CHANGE


When CSG requires the utilization of the regular maintenance window, the MSOC manager
begins the Change Management process, including opening a ticket in Heat. The MSOC
manager contacts the Client to inform them of the upcoming change. The MSOC manager is the
primary contact for the Client and continues to monitor and track the change request until it is
completed.

EMERGENCY CHANGES
If a Client needs to make modifications that require a reboot of the system, the Client should call
the MSOC manager to implement this change immediately. The MSOC manager verifies that the
person calling is authorized to request changes of this nature. Once the Client is verified, a ticket
is opened to track the change. The Client receives a ticket number for tracking purposes. The
MSOC manager is the primary contact for the Client and continues to monitor and track the
change request until it is completed.
NOTE: With proper authorization, an MSOC Administrator may perform this function for the
MSOC manager.

Who Is Authorized To Request A Change?


The individuals who are authorized to request a change differ depending on which entity originates
the request.
Client

The Client's authorized requesters are pre-defined during phase IV of the post-sales
process. During this phase, the MSOC manager is responsible for contacting the Client
to obtain the name(s) of the individual(s) who are authorized to request changes.

IBM

For a current list of the IBM authorized requesters, see Contact Information on the
Champion portal (https://www.championpulse.com).

CSG

The CSG authorized requesters include:


Operations Manager

President, Managed Services

NOTE: Emergency requests will be handled on an ad hoc basis and must be thoroughly
documented to include the requester's complete identification, which may
require a specific authorization code.

How Is A Request Submitted?


Most change requests are to be submitted via the CSG Managed Services Web site at
https://www.championpulse.com. However, emergency requests can be handled on an ad hoc basis,
via the telephone, but must be thoroughly documented to include the requester's complete
identification, which may require a specific authorization code.
NOTE: A request for change must be received no later than end-of-day on Wednesday to be
reviewed and considered by the Change Control Board on the Thursday of the same week.
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 16

Operations Run Book


2/8/2015 7:58:44 AM

Any request received after end-of-day Wednesday will not be addressed until Thursday of the
following week.
When a request is received, use the Requests For Service procedure to accommodate the Client's
needs.

Scheduled Maintenance Windows


Champion Managed Services (CMS) maintains the server hardware and operating system
instances. All applications are the responsibility of the customer.
In order to proactively manage the environment, a standard maintenance window, between 3:00
AM and 6:00 AM Eastern Standard Time each Sunday, is reserved for internal, Champion
maintenance. The time at which Client-requested changes are made is determined during the
change request process.

Change Control Board


The Change Control Board (CCB), which meets each Friday, consists of personnel from IBM and
CSG. The current members are:
CSG

IBM

Operations Manager
MSOC Supervisor

Technical Leads
Senior Application/Web Development Lead

IBM Project Manager


Project Manager - Business Partners

Site Project Executive

NOTE:

A request for change must be received no later than end-of-day on Wednesday to be


reviewed and considered by the Change Control Board on the Thursday of the same
week. Any request received after end-of-day Wednesday will not be addressed until
Thursday of the following week.

AUTHORS NOTE:
The NOTE above refers to a specific timetable for holding the CCB meeting(s). We
need to clarify the frequency at which they are actually held. Is it on Monday,
Wednesday, and Friday? Is it everyday?
There are other sections that may require modifications too. AND, the policies and
procedures will require the same modifications.

Decision Categories
There are three categories into which a decision may be defined:
Approved

The Client is informed when the change will be performed.

Declined

The Client is informed why the change was declined. This may require that a
completely new sales process begin in order to define specific requirements.

Postponed

The Client and the CCB mutually agree to postpone the change until a later date.
This may require that a completely new sales process begin in order to define

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 17

Operations Run Book


2/8/2015 7:58:44 AM

specific requirements.

Turnaround Time
While an immediate acknowledgement is provided to the Client, the turnaround time for scheduled
changes is within one week from the day the request is reviewed by the CCB. Many tasks associated
with a request may be performed the same day, or within one week from the day the request is
approved.
A request for change must be received no later than end-of-day on Wednesday to be reviewed and
considered by the Change Control Board on the Thursday of the same week. Any request received
after end-of-day Wednesday will not be addressed until Thursday of the following week.
NOTE: Emergency requests will be handled on an ad hoc basis and must be thoroughly documented
to include the requester's complete identification, which may require a specific authorization
code.

Problem Management
Business Hours
Champions Managed Service Operations Center (MSOC) is available 24 hours per day, 7 days per
week. The MSOC provides you with the ability to submit service requests by utilizing the Helpdesk
System on the MSOC Portal via the Internet, or by telephone.

Contacting Support (Champion MSOC)


The following contact information is provided for the customers for the purpose of creating a request
for service and incident reporting:

TELEPHONE
Toll Free: 888-997-7789
Local / Direct: 561-997-7789

CHAMPION PORTAL
1. Log on to the Champion Portal. https://www.championpulse.com
2. Click the Support button at the top of the page.
3. From the left navigation panel, click Helpdesk System. In an effort to protect your records, you
may need to enter your user name and password to enter this restricted area.
Instructions for Creating A Request For Service is provided on the portal under Documents.

Severity And Priority Levels


When a service request is submitted, a priority level must be selected and assigned to the
request. Each level identifies the criticality of the problem and the level of support that is needed.
The four priority levels and their definitions are:
Priority 1:

Critical Impact/Urgent The product, service, or network is not usable or affects


the customer's core business. It is Champions objective to respond to all priority
1 requests within fifteen (15) minutes.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 18

Operations Run Book


2/8/2015 7:58:44 AM

Priority 2:

Emergency Request The Client needs a quick response for reasons defined by
the requester. It is Champion's objective to respond to all priority 2 requests
within four (4) hours.
Reminder: Per contract, the Client is permitted only three priority 2 requests per
month, which are included in the Client's monthly fees. There will be
additional charges if the Client exceeds the limit of three priority 2
requests per month.

Priority 3:

Major Impact An important function or service is not available, however the


environment can still be used. It is Champion's objective to respond to all priority
3 requests within 48 hours and escalated appropriately.

Priority 4:

Minor Impact/Informational The product, service, or network is not seriously affected


and is not currently affecting the customers core business. Or, this level of severity
comprises shortcomings, suggestions, or questions. It is Champion's objective to
respond to all priority 4 requests within one week and escalated appropriately.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 19

Operations Run Book


2/8/2015 7:58:44 AM

Trouble Ticket Workflow

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 20

Operations Run Book


2/8/2015 7:58:44 AM

Monitoring Standards
PURPOSE
The purpose of this document is to outline the thresholds that are associated with standard
monitoring activities performed for the Champion Managed Services Clients.

REFERENCE
Please contact the Manager of Managed Services Operations Center to clarify any process within this
procedure and for any concern beyond its scope.

OVERVIEW
Champion Managed Services provides monitoring services based on the standard, global thresholds
for the activities listed below under Thresholds. While the standard monitoring thresholds are listed
below, Champion Managed Services is able to provide modified settings for the alerts associated with
particular monitoring activities. Any alternative parameters will be assessed on an as-needed basis.

Thresholds
The following are the current standard monitoring functions and their respective thresholds.

IIS SERVICES
The following are the thresholds for IIS Services:

PerfMon Status
Warning Level Alert

Critical Level Alert


Down >2 Minutes

Page File
Warning Level Alert
>= 90% 5 Minutes

Critical Level Alert


>= 99% 5 Minutes

Memory Usage
Warning Level Alert
>= 80% 5 Minutes

Critical Level Alert


>= 90% 5 Minutes

Disk Usage
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 21

Operations Run Book


2/8/2015 7:58:44 AM

Warning Level Alert


<= 10% Immediate

Critical Level Alert


<= 1% Immediate

CPU Usage
Warning Level Alert
>= 90% 3 Minutes

Critical Level Alert


>= 95% 3 Minutes

DNS Server Response


Warning Level Alert
>= 10000msec 5 Minutes

Critical Level Alert


>= 10000msec 5 Minutes

PortMonitor Status (LDAP, Web, E-mail)


Warning Level Alert

Critical Level Alert


Down >5 Minutes

Ping
Warning Level Alert

Critical Level Alert


Down >3 Minutes

Ping Response Time


Warning Level Alert

Critical Level Alert


>=300msec 3 Minutes

Switch SNMP Status


Warning Level Alert

Copyright 2005 Champion Solutions Group


Operations Run Book

Critical Level Alert


Down Immediate
Page - 22

Operations Run Book


2/8/2015 7:58:44 AM

SQL SERVER 2000


The following are the thresholds for SQL Server 2000:

PerfMon Status
Warning Level Alert

Critical Level Alert


Down 5 Minutes

Processor Time
Warning Level Alert
>= 80% 5 Minutes

Critical Level Alert


>= 90% 3 Minutes

Buffer Cache Hit Ratio


Warning Level Alert
<= 90%

Critical Level Alert

Cache Hit Cache Count


Warning Level Alert
<= 84%

Critical Level Alert

Conflicts For New


Warning Level Alert
>= 1/sec

Critical Level Alert

Log Used
Warning Level Alert
>= 80% 5 Minutes

Copyright 2005 Champion Solutions Group


Operations Run Book

Critical Level Alert


>=97% 5 Minutes

Page - 23

Operations Run Book


2/8/2015 7:58:44 AM

STORAGE AREA NETWORK SWITCHES


The following are the thresholds for SAN switches:

Port Status
Warning Level Alert

Critical Level Alert


>=2 Immediate

WEBSERVERS
The following are the thresholds for Webservers:

Available Memory
Warning Level Alert
<=128M

Critical Level Alert

Network Intrusion Detection System


PURPOSE
With the increased complexity of security threats, achieving efficient network intrusion security is
critical to maintaining a high level of protection. Vigilant protection ensures business continuity
and minimizes the effects of costly intrusions.

SCOPE
Champion Managed Services provides network intrusion detection on all incoming and outgoing
internet traffic. Signature updates are applied as supplied by the manufacturer within one week
of posting for validation and testing purposes. IDS logs are available to the customer via service
request ticket submission.

GENERAL / POLICY
Champion Managed Services monitors activity logs and responds as alert thresholds are met
pertaining to manufacturer supplied signatures. The IDS system is designed to accurately
identify and classify known and unknown threats targeting your network, including worms, denialof-service (DoS), and application attacks. Multiple detection methods are employed, thus
ensuring comprehensive coverage. The methods include stateful pattern recognition, protocol
analysis, traffic anomaly detection, and protocol anomaly detection. The IDS technology
implemented by Champion Managed Services uses multilayer protection options to prevent an
attack from successfully reaching targets. After the attack is accurately identified and classified,
the system can stop the attack before damage occurs.

ENFORCEMENT
In the event of any type of threat that is deemed to require attention and actions, Champion
Managed Services Network team will assess the activity, deem necessary actions, and contact
the customer. The customer contact will be constructed with information of attack, necessary

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 24

Operations Run Book


2/8/2015 7:58:44 AM

actions needed to be performed, and verification of needed information as they pertain to


customer proprietary applications.

Operating System Patches / Service Packs


MICROSOFT
Patches and service packs are deployed from Shavlik during the specified scheduled
maintenance windows. Please reference the section heading Scheduled Maintenance Windows
for more information.
NOTE: Some patches may require a reboot at this time.

AIX
Patches and service packs are to be installed during the specified scheduled maintenance
windows. Please reference the section heading Scheduled Maintenance Windows for more
information.
NOTE: Some patches may require a reboot at this time.

LINUX
Patches and service packs are to be installed during the specified scheduled maintenance
windows. Please reference the section heading Scheduled Maintenance Windows for more
information.
NOTE: Some patches may require a reboot at this time.

Disaster Recovery
All "hardened" IBM facilities are of an enterprise class nature, complete with redundant power
including generator back-ups. All data management is maintained using multiple copies of critical
data to be stored onsite in the hardened facility, as well as an alternative location offsite.
Disaster Recovery is not included in the basic managed service offering and will not be addressed
unless otherwise agreed to in writing between the customer and IBM.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 25

Operations Run Book


2/8/2015 7:58:44 AM

Appendix A Windows Server Security Checklist1


The following checklist is a recommended windows security checklist that Champion Managed
Services suggests customers to request implementation on new network environments. Three
security consideration checklists are listed below for customer ease of application and operating
system assessment needs.
Basic Security Considerations
Provide Physical Security for the machine
Most security breaches in corporate environments occur from the inside. Culprits can be
well meaning "power users" who configure their co-workers PCs, to disgruntled employees,
or they can be full blown corporate spies that are working at your company. It may not be
practical to physically secure every workstation in your environment, but your servers need
to be in a locked room with monitored access. Consider placing surveillance cameras in
your server rooms and keeping the tapes for 30 days. For desktops, install a lock on the
CPU case, keep it locked, and store the key safely away from the computer at a secure
location. (i.e. a locked cabinet in the server room)
Disable the Guest Account
Windows 2000 finally disables the guest account by default, but if you didn't build the
image yourself, always double check to make sure the guest account is not enabled. For
additional security assign a complex password to the account anyway, and restrict its logon
24x7.
Limit the number of unnecessary accounts
Eliminate any duplicate user accounts, test accounts, shared accounts, general
department accounts, etc., Use group policies to assign permissions as needed, and audit
your accounts regularly. These generic accounts are famous for having weak passwords
(and lots of access) and are at the top of every hacker's list of accounts to crack first. This
can be a big problem at larger companies with understaffed IT departments. An audit at a
Fortune 10 company I worked for revealed that 3,000 of their 15,000 active user accounts
were assigned to employees who no longer worked for the company. To make matters
worse, we were able to crack the passwords on more than half of those inactive accounts.
Create 2 accounts for Administrators
I know this goes against the previous caveat, but this is the exception to the rule. Create
one regular user account for your Administrators for reading mail and other common tasks,
and a separate account (with a more aggressive password policy) for tasks requiring
administrator privileges. Have your Administrators use the "Run As" command available
with Windows 2000 to enable the access they need. This prevents malicious code from
spreading through your network with admin privileges.
Rename the Administrator Account
Many hackers will argue that this won't stop them, because they will use the SID to find the
name of the account and hack that. Our view is, why make it easy for them. Renaming the
Administrator account will stop some amateur hackers cold, and will annoy the more
determined ones. Remember that hackers won't know what the inherit or group
permissions are for an account, so they'll try to hack any local account they find and then
1

Copyright 2004 Microsoft Corporation. All rights reserved

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 26

Operations Run Book


2/8/2015 7:58:44 AM

try to hack other accounts as they go to improve their access. If you rename the account,
try not to use the word 'Admin" in its name. Pick something that won't sound like it has
rights to anything.
Consider creating a dummy Administrator account
Another strategy is to create a local account named "Administrator", then giving that
account no privileges and impossible to guess +10 digit complex password. This should
keep the script kiddies busy for a while. If you create a dummy Administrative account,
enabled auditing so you'll know when it is being tampered with.
Replace the "Everyone" Group with "Authenticated Users" on file shares
"Everyone" in the context of Windows 2000 security, means anyone who gains access to
your network can access the data. Never assign the "Everyone" Group to have access to a
file share on your network, use "Authenticated Users" instead. This is especially important
for printers, who have the "Everyone" Group assigned by default.
Password Security
A good password policy is essential to your network security, but is often overlooked. In
large organizations there is a huge temptation for lazy administrators to create all local
Administrator accounts (or worse, a common domain level administrator account) that uses
a variation of the company name, computer name, or advertising tag line. i.e.
%companyname%#1, win2k%companyname%, etc. Even worse are new user accounts
with simple passwords such as "welcome", "letmein", "new2you", that aren't required to
changed the password after the first logon. Use complex passwords that are changed at
least every 60 -90 days. Passwords should contain at least eight characters, and preferably
nine (recent security information reports that many cracking programs are using the eight
character standard as a starting point). Also, each password must follow the standards set
for strong passwords .
Password protect the screensaver
Once again this is a basic security step that is often circumvented by users. Make sure all
of your workstations and servers have this feature enabled to prevent an internal threat
from taking advantage of an unlocked console. For best results, choose the blank
screensaver or logon screensaver. Avoid the OpenGL and graphic intensive program that
eat CPU cycles and memory. Make sure the wait setting is appropriate for your business. If
you can get your users in the habit of manually locking their workstations when they walk
away from their desks, you can probably get away with an idle time of 15 minutes or more.
You can keep users from changing this setting via Group Policy.
Use NTFS on all partitions
FAT and FAT32 File systems don't support file level security and give hackers a big wide
open door to your system. Make sure all of your system partitions are formatted using
NTFS.
Always run Anti-Virus software
Again, this is something that is considered a basic tenet of security, but you would be
surprised at how many companies don't run Anti-Virus software, or run it but don't update it.
Today's AV software does more than just check for known viruses, many scan for other
types of malicious code as well.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 27

Operations Run Book


2/8/2015 7:58:44 AM

Secure your Backup tapes


It's amazing how many organizations implement excellent platform security, and then don't
encrypt and/or lock up their backup tapes containing the same data. It's also a good idea to
keep your Emergency Repair Disks locked up and stored away from your servers.

Mid Level Security Measures


Use the Security Configuration Toolset included with Windows 2000 to configure
policies.
Microsoft provides a Security Configuration Toolset which provides plug in templates for the
MMC that allow you to easily configure your policies based on the level of security you
require. The template includes a long list of configurable options (many of which appear on
this checklist) and also includes a useful security analysis tool. For more information,
download the documentation here. If your workstation is not part of a domain, you can still
enable policies by using the Poledit.exe file from the Windows 2000 Server CD-ROM. For
more information, check out Microsoft Knowledge Base Article: 269799 - How to Secure
Windows 2000 Professional in a Non-Domain Environment.
Don't allow unmonitored modems in your environment
One of the easiest hacks in the world is finding a company's phone number prefix and
suffix range and wardialing for a modem that picks up. After weeding through the fax
machines, you can either look for an unsecured workstation with RAS enabled, or one with
Symantec's PC Anywhere loaded on it. If either one is configured incorrectly, you can
easily gain access to the local machine and work up from there. If you have a digital phone
system, get a list of every analog line that comes into your workplace and find out where it
goes! Every PC hooked to a modem is a security risk. Make sure they're configured
correctly and audited regularly.
Shut down unnecessary services
Unnecessary services take up system resources and can open holes into your operating
system. IIS, RAS, and Terminal Services have security and configuration issues of their
own, and should be implemented carefully if required. There are also several malicious
programs that can run quietly as services without anyone knowing. You should be aware of
all the services that all run on your servers and audit them periodically. The default services
allowed in a Windows NT 4.0 C2 certified installation are:
Computer Browser
Microsoft DNS Server
Netlogon
NTLM SSP
RPC Locator

TCP/IP
NetBIOS
Helper
Spooler
Server
WINS
Workstation

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 28

Operations Run Book


2/8/2015 7:58:44 AM

RPC Service

Event Log

Windows 2000 has not been submitted for C2 certification by Microsoft, so an updated list
of services is not available. What services are deemed unnecessary may vary based on
the function of your server and/or workstations. Please test your specific configuration in a
lab environment before enabling it in your production network. A list of services available in
Windows 2000 Server (as well as their default settings) can be found here
Shut down unnecessary ports
This is a judgment call based on your needs and risks. Workstations aren't normally at risk
behind a firewall, but never assume your servers are safe! A hackers first attempt at
rattling the doors and windows usually involves using a port scanner. You can find out a list
of open ports on your local system by opening the file located at %systemroot
%\drivers\etc\services. You can configure your ports via the TCP/IP Security console
located in the TCP/IP properties (Control Panel > Network and Dial Up Connections >
Local Area Connection > Internet Protocol (TCP/IP) > Properties > Advanced >
Options > TCP/IP Filtering) To allow only TCP and ICMP connections, configure the UDP
and IP Protocol check boxes to "Permit Only" and leave the fields blank. A list of default
ports for Windows 2000 Domain Controllers can be found here
Enable Auditing
The most basic form of Intrusion Detection for Windows 2000 is to enable auditing. This will
alert you to changes in account policies, attempted password hacks, unauthorized file
access, etc., Most users are unaware of the types of doors they have unknowingly left
open on their local workstation, and these risks are often discovered only after a serious
security breach has occurred. At the very minimum, consider auditing the following events:
Event

Level of Auditing

Account logon events

Success, failure

Account management

Success, failure

Logon events

Success, failure

Object access

Success

Policy change

Success, failure

Privilege use

Success, failure

System events

Success, failure

Set permissions on the security event log


The event log files are not protected by default, so permissions should be set on the event
log files to allow access to Administrator and System accounts only.
Store all sensitive documents on file servers
Although most new workstations come with some very large drives, you should consider

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 29

Operations Run Book


2/8/2015 7:58:44 AM

storing all of a users data (documents, spreadsheets, project files, etc.,) on a secured
server, where the data is backed up regularly. Modify the parameters for the "My
Documents" folder to always point to the users network share on a secured server. For
laptop users, enable the "Make available offline" capabilities to synchronize the folder's
content.
Prevent the last logged-in user name from being displayed
When you press Ctrl-Alt-Del, a login dialog box appears which displays the name of the
last user who logged in to the computer, and makes it easier to discover a user name that
can later be used in a password-guessing attack. This can be disabled using the security
templates provided on the installation CD, or via Group Policy snap in. For more
information, see Microsoft KB Article Q310125
Check Microsoft's web site for the latest hotfixes
Nobody writes 30 million lines of code and is going to have it perfect the first time, so
updating service packs and hotfixes can go a long way to plug security holes. The problem
is that hotfixes and service packs aren't regression-tested as thoroughly as service packs
and can come with bugs of their own. You should always test them on a comparable, non
production system before deploying them. Check Microsoft's TechNet Security Page
frequently for the latest hotfixes and decide which ones you need to roll out. Tip: Our home
page at LabMice.net always features Microsoft's latest hotfix to save you time.

Advanced Security Settings


Set a power on password
This should be mandatory for all laptop users, but is rarely done in most environments for
servers and workstations because it doesn't allow you to remotely log on and reboot a
machine to the point that the Operating System will restart. Keep in mind that an intruder
who can physically open your computer's central processing unit (CPU) can adjust
hardware switches to disable the power-on password, and could also temporarily install a
drive and boot another OS, bypassing all of your security settings. If this is a concern for
your company, consider locking the case (if the model permits it) or using removable hard
drives that are locked up every night.
Disable DirectDraw
This prevents direct access to video hardware and memory which is required to meet the
basic C2 security standards. Disabling DirectDraw may impact some programs that require
DirectX (games), but most business applications should be unaffected. To disable it edit the
Registry HKLM\SYSTEM\CurrentControlSet\Control\GraphicsDrivers\DCI and set the value
for Timeout (REG_DWORD) to 0
Disable the default shares
Windows NT and Windows 2000 open hidden shares on each installation for use by the
system account. (Tip: You can view all of the shared folders on your computer by typing
NET SHARE from a command prompt.) You can disable the default Administrative shares
two ways. One is to stop or disable the Server service, which removes the ability to share
folders on your computer. (However, you can still access shared folders on other
computers.) When you disable the Server service (via Control Panel > Administration
Tools > Services), be sure to click Manual or Disabled or else the service will start the
next time the computer is restarted. The other way is via the Registry by editing
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 30

Operations Run Book


2/8/2015 7:58:44 AM

HKeyLocal
Machine\SYSTEM\CurrentControlSet\Services\LanManServer\Parameters. For
Servers edit AutoShareServer with a REG_DWORD Value of 0. For Workstations, the edit
AutoShareWks. Keep in mind that disabling these shares provide an extra measure of
security, but may cause problems with applications. Test your changes in a lab before
disabling these in a production environment. The default hidden shares are:
Share

Path and Function

C$ D$ E$ Root of each partition. For a Windows 2000 Professional computer,


only members of the Administrators or Backup Operators group can
connect to these shared folders. For a Windows 2000 Server
computer, members of the Server Operators group can also connect
to these shared folders
ADMIN$

%SYSTEMROOT% This share is used by the system during remote


administration of a computer. The path of this resource is always the
path to the Windows 2000 system root (the directory in which
Windows 2000 is installed: for example, C:\Winnt).

FAX$

On Windows 2000 server, this used by fax clients in the process of


sending a fax. The shared folder temporarily caches files and
accesses cover pages stored on the server.

IPC$

Temporary connections between servers using named pipes


essential for communication between programs. It is used during
remote administration of a computer and when viewing a computer's
shared resources

NetLogon This share is used by the Net Logon service of a Windows 2000
Server computer while processing domain logon requests.
PRINT$

%SYSTEMROOT%\SYSTEM32\SPOOL\DRIVERS Used during


remote administration of printers.

Disable Dump File Creation


A dump file can be a useful troubleshooting tool when either the system or application
crashes and causes the infamous "Blue Screen of Death". However, they also can provide
a hacker with potentially sensitive information such as application passwords. You can
disable the dump file by going to the Control Panel > System Properties > Advanced >
Startup and Recovery and change the options for 'Write Debugging Information" to None.
If you need to troubleshoot unexplained crashes at a later date, you can re-enable this
option until the issue is resolved but be sure to disable it again later and delete any stored
dump files.
Enable EFS (Encrypting File System)
Windows 2000 ships with a powerful encryption system that adds an extra layer of security
for drives, folders, or files. This will help prevent a hacker from accessing your files by
physically mounting the hard drive on another PC and taking ownership of files. Be sure to
enable encryption on Folders, not just files. All files that are placed in that folder will be
encrypted. For more information check out our EFS Resource Center

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 31

Operations Run Book


2/8/2015 7:58:44 AM

Encrypt the Temp Folder


Applications use the temp folder to store copies of files while they are being updated or
modified, but they don't always clean the folder when you close the program. Encrypting
the temp folder provides an extra layer of security for your files.
Lock down the Registry
In Windows 2000, only Administrators and Backup Operators have default network access
to the registry, however you may wish to tighten this down even further. To restrict network
access to the registry, follow the steps listed in TechNet Article Q153183
Clear the Paging File at shutdown
The Pagefile is the temporary swap file Windows NT/2000 uses to manage memory and
improve performance. However, some 3rd party programs may store store unencrypted
passwords in memory, and there may be other sensitive data cache as well. You can clear
the pagefile at shutdown by editing the Registry Key
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory
Management and changing the data value of the ClearPageFileAtShutdown value to 1
Disable the ability to boot from a floppy or CD ROM on physically unsecured
systems.
There are a number of 3rd party utilities that pose a security risk if used via a boot disk
(including resetting the local administrator password.) If your security needs are more
extreme, consider removing the floppy and CD drives entirely. As an alternative, store the
CPU in a locked external case that still provides adequate ventilation.
Disable AutoRun for CD-ROM drives on physically unsecured systems.
One of the easiest ways for a hacker with physical access to a company's PC's to distribute
malicious code is via the CD-ROM. By creating a custom CD with a payload set to launch
from the autorun feature in any machine, a hacker can affect any number of unlocked
systems without ever leaving a fingerprint or touching a keyboard. Or he/she can simply
leave a few of these lying around the office marked "MP3's", or "Payroll Data" and wait for
an unsuspecting user to simply pick it up and insert it into their machine. You can disable
this function by editing the Registry and changing the HKEY_LOCAL_MACHINE
\SYSTEM \CurrentControlSet \Services Cdrom subkey and set the AutoRun value to 0
Remove the OS/2 and POSIX Subsystems
If you are not using these subsystems (and people rarely do), removing them may improve
performance and also closes a potential security risk.

To remove the OS/2 and POSIX subsystems:


1. Delete the \winnt\system32\os2 directory and all of its subdirectories.
2. Use the Registry Editor to remove the following registry entries:
Key:

HKEY_LOCAL_MACHINE\SOFTWARE

Subkey: Microsoft\OS/2 Subsystem for NT


Entry:

delete all subkeys

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 32

Operations Run Book


2/8/2015 7:58:44 AM

Key:

HKEY_LOCAL_MACHINE\SYSTEM

Subkey: CurrentControlSet\Control\Session Manager\Environment


Entry:

Os2LibPath

Value:

delete entry

Key:

HKEY_LOCAL_MACHINE\SYSTEM

Subkey: CurrentControlSet\Control\Session Manager\SubSystems


Entry:

Optional

Values: delete entry

Key:

HKEY_LOCAL_MACHINE\SYSTEM

Subkey: CurrentControlSet\Control\Session Manager\SubSystems


Entry:

delete entries for OS2 and POSIX

The changes take effect the next time the computer is started. You might want to update
the emergency repair disk to reflect these changes.
Consider using SmartCard or Biometric devices instead of passwords.
The more stringent your password policy is, the more likely your users will begin keeping
paper password lists in their desk drawers, or taped to the bottom of their keyboard.
Windows 2000 supports these devices, so consider the costs vs. risks of your most
sensitive data.
Consider implementing IPSec
Basically, IPSec provides encryption for network sessions using the Internet Protocol (IP)
and promises to offer transparent and automatic encryption of network connections. For
more information, click here

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 33

Operations Run Book


2/8/2015 7:58:44 AM

LINUX SECURITY CHECKLIST

The following is a recommended security checklist for Linux servers. This document should be
used as a guide to the installation and configuration of Linux Servers in conjunction with an
agreed security plan for the identified systems. The document is designed for use by experienced
system administrators. Some of the settings may be dependant on the patch levels of the
components in use, and therefore differencies may exist between this document and the actual
file paths and access control settings on your machine. Most of the points below can be
addressed by running security scripts made specifically for every system (fx. Harden_suse), but
due to the general nature of these scripts or applications it is not advised to use them without
proper testing.
Initial Installation
Install the Latest Patches
In most cases distribution vendors will provide an update facility for the distribution of patches.
The latest system patches should be installed prior to operational deployment. Particular
attention should be paid to those network services that the operating system makes available to
remote clients (eg: Web (Apache), Mail (sendmail/postfix/imapd), and so on.
It is also recommended that the system be updated with newly realeased patches as soon as
operational circumstances allow.
Bypassing the vendor, and installing patches directly from the application provider (eg: from
apache.org) may also be appropriate in some circumstances, where the problem in question is
significant, or the distribution vendor response to security issues is poor.
Latest Patches can be found at
Debian ftp://ftp.debian.org/debian/dists/stable-proposed-updates/
RedHat ftp://ftp.redhat.com/pub/redhat/linux/updates
SuSe ftp://ftp.suse.com/pub/suse/i386/update/
In order to stay updated with latest vulnerabilities on Solaris systems and patches required for it
Sun issues a security bulletin. To receive security bulletins directly from:

Debian http://www.debian.org/MailingLists/subscribe#debian-security-announce

RedHat https://listman.redhat.com/mailman/listinfo/redhat-watch- list

SuSe http://www.suse.com/us/private/support/mailinglists/index.html

File Systems
Per default Linux mounts remote or local filesystems are mounted with read-write privileges with
possibility to have suid or sgid files.
In order to prevent that filesystems that don't require extra privileges should be limited.
In /etc/fstab there should be an entry nosuid or noexec for external devices like cdrom or
filesystems in that specific row in the fourth field .
Time Settings
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 34

Operations Run Book


2/8/2015 7:58:44 AM

All the servers should have the same time settings in order to be able to evaluate logs properly.
1. There should be a time-zone entry in /etc/sysconfig/clock containing
ZONE=Europe/Berlin. Or in Debian /etc/timezone should contain Europe/Berlin
2. There should be a NTP system installed with timeservers configured for synchronisation
(fx. /etc/ntp.conf should contain server a.b.c.d prefer)
Timeservers in OssBss are 10.130.200.70 or 10.130.200.80
o In Management network 10.10.8.70 or 10.10.8.80

o In internet network there are official time servers at


http://www.eecis.udel.edu/~mills/ntp/clock1.htm
Software Selection
If system should be freshly installed, there should be core installation used and only those
packages added that are required for operation of the system. All the external packages that
cant be patched should be kept updated to the latest operational version (fx. SSH package
should be version 3.4.1 or higher).
All the unnecessary modules should be also removed.
Minimize boot services or daemons
All the unnecessary daemons or services starting at boot time (/etc/rc*.d) should be removed or
disabled. They can also be listed with chkconfig list on all systems except Debian.
FX.A service can be disabled with chkconfig level 3 lpd off or just removed from
/etc/rc.d/rc3.d/S12lpd
Message Text for users attempting to log on
/etc/motd
Place the following message (or a similar one) into this file. It contains a message that will be
printed after a successful login.
This is a private computer facility. Access for any reason must be specifically
authorized by the owner. Unless you are so authorized, your continued access and
any other use may expose you to criminal and/or civil proceedings. Usage may be
monitored.
/etc/issue
Place the following message (or a similar one) into this file. It contains a message that will be
printed during the login process.
This is a private computer facility. Access for any reason must be specifically
authorized by the owner. Unless you are so authorized, your continued access and
any other use may expose you to criminal and/or civil proceedings. Usage may be
monitored.
/etc/issue.net
Place the following message (or a similar one) into this file. It contains a message that will be
printed during the login process.
This is a private computer facility. Access for any reason must be specifically
authorized by the owner. Unless you are so authorized, your continued access and
any other use may expose you to criminal and/or civil proceedings. Usage may be
monitored.
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 35

Operations Run Book


2/8/2015 7:58:44 AM

NOTE:

The users may see both the /etc/motd and the /etc/issue messages when they login.

SSH daemon should be configured to display the message by putting this line into sshd_config:
Printmotd yes
Privileged Account Login Source
In order to ensure security of the root account there should be limitations placed on the source of
login.
Root should be able to log into the system only locally (via console or with su command).
This can be ensured by :
1. In /etc/nologin there should be all the administrative accounts
2. In /etc/security/access.conf there should be a line
-:ALL EXCEPT wheel shutdown sync:console
-:ALL EXCEPT root:ALL EXCEPT console
3. In sshd_conf put line PermitRootLogin no
Network driver configuration
Make the following adjustments to the /etc/sysctl.conf to protect the machine from some types of
network attacks.

1.
2.
3.
4.
5.
6.
7.

net.ipv4.ip_forward = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

and protect the configuration file:


chown root:root /etc/sysctl.conf

chmod 0600 /etc/sysctl.conf

Disable multicasting:
ifconfig [interface] -allmulti -multicast
System Network Services
Network Services Summary
All the unnecessary network services should be switched off.
1. 1./etc/inetd.conf should not contain any entries unless specifically required by
applications.
Here is a quick rundown of the risks associated with services started in /etc/inetd.conf :
ftp: enables an FTP server that introduces a variety of insecurities and is the cause of many
intrusions. Disable this and use SSH instead to transfer files between systems.
telnet, shell, login, exec: allows users from other systems to log into and run commands on your
machine. This is useful, but the more useful something is, the more likely it is that someone will
find a way to exploit it. Disable these services and, if you do need to allow remote logins, use
SSH instead.
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 36

Operations Run Book


2/8/2015 7:58:44 AM

comsat: a daemon which is used to notify users of newly arrived email. There are alternate
means of doing the same thing, and there are occasional rumors of security problems with
comsat. Unless you have some overwhelming need for this, turn it off.
talk: allows users to communicate by typing at each others terminals.
uucp: Nobody uses uucp anymore - disable this. While you are at it, you may as well turn off
execute permission on the uucp-related shell commands.
tftp: FTP without any security. This should be needed only if your system will be used for booting
workstations. If this is the case, you must invoke the daemon with the -s flag, as in:
tftp dgram udp wait root in.tftpd -s /tftpboot
If you don't, tftp can be used to retrieve any file from your system, anonymously. Also make all the
files in the bootfile directory read-only. Finally, restrict access to the service using TCPwrappers
and
IPFilter/IPChains.
finger: this gives out information on who is loggedin, or people's phone numbers and offices.
Unfortunately this information can be used by a potential intruder to find accounts to attack. You
may wish to disable this, run a custom finger daemon, or restrict access to it using TCPwrappers
and
IPFilter/IPChains.
systat, netstat: these services give out information about your system. The comments for finger
apply to these.
time : Gives out the system time to any remote host that asks for it. Probably safe but can be
disabled without impacting the system.
echo, discard, daytime, chargen: these are used for testing, and are generally safe, though
there have been reports of TCP packets with forged IP source addresses being used to trick a
system into sending echo packets to itself, causing a packet storm on the local ethernet segment.
Disable them and only turn them on while testing.
rexd - this is the Remote Procedure Call mechanism. It has minimal authentication, so disable it
and use SSH instead.
walld: allows people to send messages to all logged in users. Useful, but easily abused.
ttdbserverd (tooltalk): used by some convenient desktop elements but not important from a
system operation standpoint. Some versions of this service contain serious remote exploits and
should be disabled (dsabling this service causes virtually no operational degradation).
rpc.cmsd (calendar manager) : used to share calendar information over the network but not
important from a system operation standpoint. Some versions of this service contain serious
remote exploits and should be disabled.
others : Other services such as sadmind (once found to be vulnerabale to remote root exploit)
and kerberos can be disabled without impacting the system.
There should not be any services listening on the network unless required by applications.

Fx. XFree86 listens on port 6000+n, where n is the display number. This
connection type can be disabled with the -nolisten option (see the Xserver(1) man
page for details).
File Transfer

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 37

Operations Run Book


2/8/2015 7:58:44 AM

Ftp service should be disabled and secure ftp or secure copy should be used. /etc/ftpusers
should contain all the account names except those that should be allowed to access the system
via FTP.
Electronic Mail
There should be no email service running on the system for local use (email servers have email
agents installed as application and should follow the application security part of this document).
Domain Name Service
There should be no DNS servers running on the system (DNS servers should be treated as an
application and should follow application security part of this document).
Remote shell / copy services
All the systems should have the latest SSH installed in order to allow remote administration of the
server with encrypted interconnection. Sshd_config should contain also these features:

Protocol 2
UsePrivilegeSeparation yes
ServerKeyBits 768
SyslogFacility AUTH
LogLevel INFO
LoginGraceTime 600
PermitRootLogin no
StrictModes yes
RSAAuthentication yes
PubkeyAuthentication yes
RhostsAuthentication no
IgnoreRhosts yes
RhostsRSAAuthentication no
HostbasedAuthentication no
PermitEmptyPasswords no
PasswordAuthentication yes
PrintMotd yes
PrintLastLog no
MaxStartups 10:30:60
ReverseMappingCheck yes

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 38

Operations Run Book


2/8/2015 7:58:44 AM

File Share Facilities


There should be no file sharing facilities (like nfs or cifs), unless required by the application .
Firewall Selection
Every system should have a filtering capability at disposal for later use. Filtering capabilities
should enable limitation of certain IP addresses to certain services.
1. 1.IPTables http://www.iptables.org/
2. 2.IPChains http://www.netfilter.org/ipchains/
System Accounts and User Rights
Account Characteristics
By default, Linux operates on the assumption that all users are local users. Specific package
installation needs to be done on most Linux distributions in order to facilitate a distributed
authentication framework such as LDAP.
Other features:
1. User home directories should be mode 755 or more restrictive
2. No user dot-files should be group/world writable
3. Standard Password policy should be put in /etc/login.defs
Standard Accounts
Several of the accounts in /etc/passwd are unnecessary. In order to secure them youshould:
1. effectively disable them.
2. ensure these accounts cannot use ftp, cron or at.
3. remove valid shells from daemon accounts.
4. Put them into /etc/nologin
Unauthenticated Access
There should be no possible unauthenticated access enabled on the system.
1. /etc/hosts.equiv, /root/.rhosts, /etc/ssh/shosts /root/.netrc should be empty
2. No empty password fields in password files
Appropriate Administrative Authentication
Access to root account via su should be only possible from wheel group. All users that are system
administrators should have separate accounts and be in wheel group and only they can login to
the console. All this can be configured in
/etc/security/access.conf
Authentication Configuration
In order to enforce system authentication to use standard unix authentication facility /etc/pam.d
should contain these entries:
# PAM configuration
# Authentication management
login auth required
su auth required
other auth required
Copyright 2005 Champion Solutions Group
Operations Run Book

/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
Page - 39

Operations Run Book


2/8/2015 7:58:44 AM

# Account management
login account required
su account required
other account required

/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1

# Session management
su session required
other session required

/usr/lib/security/pam_unix.so.1
/usr/lib/security/pam_unix.so.1

# Password management
other password required

/usr/lib/security/pam_unix.so.1

If http authentication is supposed to be used then there should be extra entries specifying http
authentication facility.
File and Object Access
Umask settings
Set user file creation mask
In each of the files /etc/csh.cshrc and /etc/profile, there should be an invocation of the umask
command. This invocation should be positioned immediately after the initial comments. The
value passed to umask is an octal mask of the mode bits that are not set when a file is
created. Acceptable values are 022, 026 (suggested) and 027. Each of these has advantages
and disadvantages. Please read the umask manual page prior to selecting the value to be
set.
Set FTP file creation mask
Add the following line at the end of the /etc/proftpd.conf file. This line contains the default umask
value that will be used by FTP when a file is created.
UMASK=022
Set daemon umask umask 022
In /etc/init.d/functions add a line umask 022 (redhat)
In /etc/rc.status add a line umask 022 (others)
Permissions tightening
1. Minimize file or object access to only groups or users that will access them (fx. Oracle
daemon should be executable by user oracle only)
2. Crontab access restrictions should be put into /etc/cron.allow (debian, redhat) or
/var/spool/cron/allow (SuSe).
3. At access retrictions should be put into /etc/at.allow.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 40

Operations Run Book


2/8/2015 7:58:44 AM

SUID or SGID files.


Check for setuid files, and modify them, as appropriate. The command to check for these files is:
find / -perm -04000 -type f -exec ls -ld {} \;
-rwsr-x--- 1 root trusted /bin/ping
-rwsr-x--- 1 root trusted /bin/su
-rwsr-x--- 1 root trusted /usr/bin/crontab
-rwsr-xr- x 1 man root /usr/bin/man
-rwsr-xr- x 1 root root /usr/bin/rcp
-rwsr-xr- x 1 root root /usr/bin/rlogin
-rwsr-xr- x 1 root root /usr/bin/rsh
-rwsr-x--- 1 root shadow /usr/bin/gpasswd
-rwsr-x--- 1 root trusted /usr/bin/newgrp
-rwsr-xr- x 1 root shadow /usr/bin/passwd
-rwsr-x--- 1 root trusted /usr/bin/sudo
-rwsr-xr- x 1 root root /usr/sbin/traceroute
-rwsr-xr- x 1 root root /usr/lib/pt_chown
Check setgid files
Check for setgid files, and modify them, as appropriate. The command to check for these files
is:
find / -local -type f -perm -2000 -exec ls -ld {} \;
-rwxr-sr-x 1 root tty /usr/bin/write
-rwxr-sr-x 1 root tty /usr/bin/wall
NOTE: A server should be checked for setgid files after patches are updated, and after thirdparty packages (source or binary) are installed. A list of all the SUID and SGID files
should be maintained from the point of clean installation in order to detect deviations
from correct system state.
System access configuration files locked
In every users directory there should be configuration files created (.rhosts or authorized_keys)
with root as the owner and writable only by root in order to keep the control of the users system
access.
System Auditing
Auditing Overview
All the messages and log information should be centrally processed on a remote log server.
In OSSBSS there is a syslog server at disposal. Syslog.conf entry should look like this:
*.*
@10.130.200.40
or
*.*
@10.10.8.40
Initial Installation
Syslog messages sent to a centralized log server
1./etc/syslog.conf should contain this entry *.* @logserver
Turn on cron logging
2./etc/default/cron should contain CRONLOG=YES
Copyright 2005 Champion Solutions Group
Operations Run Book

Page - 41

Operations Run Book


2/8/2015 7:58:44 AM

Protecting the audit configuration files


Integrity verification service should be done to a remote host. Logs should be stored safely on a
read only media or on a secured media not accessible by all the system users.
Monitoring User access.
Create a log for authentication information. It should contain all the necessary authentication
information for access auditing.
echo "auth.info\t\t\t/var/log/authlog" >>/etc/syslog.conf
touch /var/log/authlog
chown root:root /var/log/authlog
chmod 600 /var/log/authlog
Application security
Patches
Install all the latest patches and fixes for the application. If possible upgrade application to the
latest version.
Minimize services offered
Switch off or remove services that are not required to perform applications function in the system.
Fx. Apache web server with basic HTML functionality required doesnt need CGI, imap module or
Java servlets configured.
Configure users and authentication
Every user should have his/her own login with proper authentication method used. Their
permissions should not include more than necessary (read only user should have only read only
access) and if necessary for every role a user has there should be an account with proper
permissions for that role.
Fx. User XY ; role: application tester (read only access to data)
User XY ; role application developer (read write to development environment)
User XY ; role application administrator (full access to application configuration)
Auditing
If possible application should produce a log documenting its tasks that are performed as well as
requests, queries or user input.
1. major or critical application errors
2. users login/logout
3. modification of application settings (security settings, logging settings, operational
settings)
4. (optional) business data modifications
Object access, permissions
If possible application should have restrictions on objects (files, items or tables) so that only
authorized sources can read, modify or delete them.
Fx. Access to SNMP system IDs in NetCool database should be limited to NetCool administrator
only.
Oracle progressor database access granted only to progressor application.

Copyright 2005 Champion Solutions Group


Operations Run Book

Page - 42