NSS Final SOP

 Configure Layer 2 switch –
Scope: This SOP is used to Configure Cisco Layer 2 Switch.

This SOP is not used to configure the other cisco devices like Router, ASA
firewall etc.
Roles:  User
 Requestor
 IT support team
Prerequisites:  Service ticket has been raised.
 Ticket has been assigned to the support engineer
 Necessary approvals has been taken.
 Hardware
 Switch Login details.
 Other network information.
Inputs:  Service Desk ticket raised and assigned to Network WAN and Internet
Support team.
Controls:  Tickets SLAs

 Colliers Standard policy.
Outputs:  Switch configured as per colliers standard.
 Updates in the ticket.
1. Once we received new switch we need to login it by default credentials – (cisco/cisco).

2. Then check Switch RAM and Slots etc.
3. Configure the Hostname for Switch.
Note – Naming convention will like Region, Location, and Device Number.
4. Configure the Interface according to Requirements.

5. Configure the Radius server for AD credentials on Switch, so user can access Switch with their
AD credentials.
6. Configure the Protocols like EIGRP, OPSF.
7. Configure the Standard configuration will be set for our Environment.
8. Finally we need to configure Line protocol on router so Remote users can login to the device.
 VSP static routing on core switch –
Scope: Configuration of VSP static routing on Core switch.

This SOP is not used to configure the other static routing.
Roles:  User
 Requestor
 IT support team
 Switch Login details.
 VSP server IP address.
 Next hop IP address.
Support team.

Outputs:  VSP static routing configured on Core switch as per colliers standard.
1. Login to core switch with the help of AD credentials.

2. Then we need to configure Static routes on core switch for VSP and that Static route should be
pointing towards next hop as checkpoint firewall.
3. For reference please check below snapshot.
4. In the above snapshot 32.65.X.X is the standard IP address range for VSP server and at the en
you can see checkpoint firewall interface IP address.
Troubleshoot VLAN issue on switch –

1. We have to login to switch by AD credentials.
2. In our environment all switches have VTP running.
Note – VTP means Vlan Trucking protocol . Which is used to propagate Vlan in our topology.
3. Core switch must be VTP Master mode and all other access switches must be in Client mode.
Core Switch – (Server Mode)
Access Switch – (Client Mode)
4. If VLAN is not propagate properly in network then we need to check VTP domain name,
Password of VTP.
5. Make sure that link between Core switch and access switch must be in Trunk mode.
From trunk interface all VLAN must be recognized properly which are present on core switch.
6. To configure VLAN on switch follow below steps.

Go to Configuration Mode – Vlan – Number of VLAN
7. To check vlans on switch.

8. To configure Layer 3 interface on switch or VLAN interface on switch.
Go to configure mode –
Interface Vlan (number )
9. To check that interface on switch.
 IOS version upgrade on Router Or Switch –
Scope: This SOP is used for IOS up gradation of Cisco Router & Switch.
Roles:  User
 Requestor
 IT support team
 Switch/Router Login details.
 IOS version copy.
 TFTP server.
Support team.

Outputs:  IOS upgraded.
1. To Download IOS Image.
To download a system image you must have an account at Cisco.com to gain access to the following
websites. If you do not have an account or have forgotten your username or password, click Cancel
at
the login dialog box, and follow the instructions that appear.
If you know the Cisco IOS release and feature set you want to download, go directly to
http://www.cisco.com/kobayashi/sw-center/index.shtml.
For more information before selecting the Cisco IOS release and feature set, go to the Software
Download Center at:
http://www.cisco.com/public/sw-center/index.shtml.
For more information about Loading and Managing System images, go to
http://www.cisco.com/en/US/docs/ios/fundamentals/configuration/guide/cf_system_images.html.
2. Saving Backup Copies of Your Old System Image and Configuration.
Router> enable
Router# copy nvram:startup-config ftp:

The configuration file copy can serve as a backup copy.
Router# dir flash0:
Learn the name of the system image file.
Router# copy flash0: ftp:

Copy the system image file to a server. This file can serve as a backup copy.
Example –
Router# copy nvram:startup-config tftp:
Remote host[]? 192.0.0.1

Name of configuration file to write [rtr2-confg]? rtr2-config-b4upgrade
Write file rtr2-confg-b4upgrade on host 192.0.0.1?[confirm] <cr>
![OK]
Router# copy flash0: tftp:

Source filename [running-config]?
Address or name of remote host []? 192.0.0.1
Destination filename [router-confg]? running-config
983 bytes copied in 0.048 secs (20479 bytes/sec)
Router# dir flash0:
Directory of flash0:/
1 -rw- 48311224 Mar 2 1901 11:32:50 +00:00
c3900-universalk9-mz.SSA.XFR_20090407
2 -rw- 185667 Jan 27 2021 09:03:54 +00:00 crashinfo_20210127-090354
3 -rw- 983 Feb 14 2021 12:41:52 +00:00 running-config
260173824 bytes total (211668992 bytes free)
Router#
3. Copying the System Image into Flash Memory
Router# copy tftp flash0:
When prompted, enter the IP address of the TFTP
Address or name of remote host []? 10.10.10.2
Source filename []? c2900-universalk9-mz.bin
Destination filename []? c2900-universalk9-mz.bin
4-
Router# configure terminal
Router(config)# no boot system
Router(config)# boot system flash0: c2900-universalk9-mz.bin.
Router(config)# exit
Router# show version
Cisco Internetwork Operating System Software
.
.
.
Configuration register is 0x0
Router#
Router# copy run start

Proceed with reload? [confirm] y
 Printer/Scanners not accessible on network
Scope: This SOP is used of Printer/Scanner issue trouble shooting.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Printer/Scanner IP address.
Support team.

Outputs:  Printer/Scanner issue resolved.
1. Login to the switch with AD credentials.

2. We need to check first VLAN present on switch.
3. Printer and Scanner are always are in DATA Vlan. So then we need to check ARP table of switch.
4. If ARP entry does not show then check the interface of Switch is directly connected to Printer or
scanner.
5. The interface connected to printer is always needs to be in Access Mode for DATA Vlan.
6. Check if Printer/ Scanner is getting proper IP address from DHCP pool or not.
 Troubleshoot Meraki wireless issue
Scope: This SOP is used of Meraki Wireless issue trouble shooting.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Meraki Dashboard login details.
Support team.

Outputs:  Meraki issue resolved.
How to Login to Meraki Dashboard
Steps –
1. Go to URL - https://n104.meraki.com/login/dashboard_login
2. Enter your credentials for Meraki Dashboard. (Username, Password)
In our environment we have receive multiple alert for Meraki , To work on that alerts we will used
below steps and Sample alerts –
Meraki is unreachable
1. login to Meraki dashboard and check the location first. Which is mention in alert.
2. Check the mention AP is Green or Red on the dashboard.
3. If it is red then login to Access switch. And check whether switch is learning MAC address of AP
on the switch or not.
4. If No then check switch port where AP is connected. Check switch port is in SHUTDOWN or
NOSHUTDOWN.
Check configuration of Port.
5. Then Check Power and Physical connectivity for AP.
Meraki is Switch from Gateway to Repeater Mode.
Follow the step above problems.

Only in this issue Gateway is not assigned to AP. At this time AP will take one of the other AP as
gateway to send Traffic.
Users complain slowness for one of SSID.
Login to meraki Dashboard, check SSID presents on that AP or not.

Then check on switch VLAN is configure for wireless.
Check Switch Ports for the AP. And check errors on the port.
Check load on the Switch port.
Then go to tab Traffic Analytics and check traffic usage on it.
Users are not able to connect to the Access Point via any of SSID.
Check SSID is OK and present on the AP.

Then check configuration for SSID.
Check all SSID is showing ON or not. As well as check VLAN present on AP and on switch.
If any of SSID is not ON then check all configuration on all AP.
If we want to check Access Points Details and Health.
1. Login to the Meraki Dashborad .

2. Select Wireless Tab and Select Access Point tab.
Above snapshot shows Access Points good Health. See Connectivity tab it shows all Green means
this AP is UP and Stable.
Above snapshot shows Access Points are not stable and it was down for particular reason.
If we need to check which client is connected to the AP and Overall usage –
Go to Network-wide Tab and Select Clients.

If we want to check particular clients details then click on clients Hostname, we will get all detail
information about that client.
 Troubleshooting DHCP & DNS issues on Switch.
Scope:  This SOP is used of DHCP & DNS issue trouble shooting.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Switch details.
Support team.

Outputs:  DHCP/DNS issue resolved.
1. Login to the core Switch.

2. Check DHCP pool on the core switch.
3. If IP are not release from core switch then check IP excluded address and DHCP pool
configuration.
 Generate reports from NetQos and UIM to verify the bandwidth utilization
Scope:  This SOP is used for generating NetQos & UIM report.
Roles:  User
 Requestor
 IT support team
 NetQos server login credentials.
Support team.

Outputs:  NetQos report prepared.
Below screenshot shows how to capture and analyses traffic from a particular site and link.
 First login to NetQos tool using Internet Explorer or Google Chrome. (Google Chrome is the
preferred application).
http://10.168.176.76
 Use the User account or Administrative account to login in to the below page.
 Once logged in, click the any site router from the list of routers showed in the below screenshot.
 Expand the site router or interface by clicking the site you want to capture the traffic utilization.
 Once we expand the site link, we will see LAN and WAN interfaces, and these information is
captured from the router by the NetQos using NetFlow and SNMP protocol.
 This information also covers interface description, Speed, interface active status.
 Interface Active status in the below screenshot means “NetFlow” is configured in that interface.
 In the above Edmonton site, WAN interface Gig0/1.136 is configured with NetFlow and we are
going to capture traffic utilization for this interface.
 Click interface “Gig 0/1.136” from the above screenshot. Below are the lists of screenshot that
appear in the same page.
 The reporting page get open and displays the overview of the traffic utilization report that
covers traffic send and received using protocols, traffic send and received using TOS, traffic send
FROM local host, traffic send TO remote host and TOP Talkers.
 We need to define the interval and the time-period for the report that we are generating.
 Below screenshot will guide us to set the date and time and the period for which we are
generating the report.
 In this example, we are generating report for the traffic utilized on 6th September from 8 am to
6 pm PST for the Edmonton site.
 In the below screen, we set the date as 6th September 8 am PST to 6 pm PST and then click
“SET”
 Once we set the period and date and time, the report will get generated for the traffic send and
received using protocols, traffic send and received using TOS, traffic send FROM local host,
traffic send TO remote host and TOP Talkers.
 We need to define the interval and the time-period for the report that we are generating.
 Below screenshot will guide us to set the date and time and the period for which we are
generating the report.
 In this example, we are generating report for the traffic utilized on 6th September from 8 am to
6 pm PST for the Edmonton site.
 In the below screen, we set the date as 6th September 8 am PST to 6 pm PST and then click
“SET”
 Once we set the period and date and time, the report will get generated for the traffic send and
received using protocols, traffic send and received using TOS, traffic send FROM local host,
traffic send TO remote host and TOP Talkers.
 The below screen shot is for the Inbound Protocol traffic. Right side we can see the total link
bandwidth as 10 Mbps and the right side of the graph shows list of protocols with their color
coding.
 Report says Inbound HTTP traffic was on peak at 10 pm PST using bandwidth of 7 Mbps.
 While Inbound HTTPS traffic was at peak at 10 pm PST using bandwidth of around 6 Mbps.
 The below screen shot is for the Outbound Protocol traffic. Right side we can see the total link
bandwidth as 10 Mbps and the right side of the graph shows list of protocols with their color
coding.

 Report says Outbound HTTPS traffic was on peak at 5:45pm PST using bandwidth of 800 Kbps.
 While Outbound HTTP traffic was at peak at 9pm PST using bandwidth of around 800 Kbps.
 Below TOS report is a continue of the above protocol report.
 The below TOS report is based on the QOS defined in the router.
 TOS inbound report says the Default traffic was on peak at 10 pm PST using bandwidth of 7
Mbps.
 TOS outbound report says the traffic marked with AF21 was on peak at 10 pm PST using
bandwidth of 7 Mbps.
 While the TOS outbound report says the Default traffic was on peak at 6 pm PST using
bandwidth of 750 Kbps.
 While the TOS outbound report says the traffic marked with AF21 was on peak at 6 pm PST and
9 pm PST using bandwidth of 800 Kbps above.
 Below screenshot shows the traffic generated FROM particular host is a continue report of the
earlier Protocol and TOS traffic report.
 This screenshot shows the traffic generated FROM a particular local host. The Host generating
traffic will show either in hostname or IP address. While sending the report to end user we need
to resolve the IP address to their hostname using windows “Nslookup” or “ping –a <ip address>.
Then send the end user both IP address and their Hostname. Sample mail is addressed in the
end of the document.
 The below screenshot says 132.245.3.220 host is generating the most amount of traffic.
 The host 132.245.3.220 is generating 2.33 GBytes of traffic, which is alone using 20.16 % of
Edmonton site total bandwidth of 10 Mbps.
 This screenshot shows the traffic generated TO a particular local host.
 The below screenshot says host edmokeatingm2.na.corp.local (207.61.65.67) is receiving the

most amount of traffic.
 The host edmokeatingm2.na.corp.local (207.61.65.67) is receiving 4.49 GBytes of traffic, which
is 35.65 % of Edmonton site total bandwidth of 10 Mbps.
 Below report for the TOP Talkers is a continue report of the earlier report of Protocol and TOS
report and Host generating and receiving traffic report.

 The below screenshot of Top Talkers report is a combination of earlier screenshot of Host FROM
and Host TO report.
 In the below report, the host 132.245.3.220 is communicating to host
edmokeatingm2.na.corp.local (207.61.65.67) using 2.28 GBytes of traffic which is 16.61 % of the
Edmonton site total bandwidth of 10 Mbps.
 You can get into the forensic report of what actual time the host 132.245.3.220 was
communicating to host edmokeatingm2.na.corp.local (207.61.65.67), by clicking the traffic
report.
 Last 2 Hrs period for the communication happened from host 132.245.3.220 to host
edmokeatingm2.na.corp.local (207.61.65.67).
 Last 8 Hrs. periods for the communication happened from host 132.245.3.220 to host
edmokeatingm2.na.corp.local (207.61.65.67).
 Till now this document explained us the “Overview” of the traffic utilization.
 We can also take separate report based on Protocols, TOS, Hosts and Conversation by selecting
from the drop down menu as showed in the below screenshot.
 The report for Protocols, TOS, Hosts and Conversation will be same as explained in this
document earlier, because “Overview” document contains the collective reports of all these
components.
 By clicking “Print” option at the right top corner of the page, we can save the traffic utilization
report in a PDF format and then send it to the end user.
 DHCP configuration on Switch –

Scope:  This SOP is used of DHCP configuration on Switch.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Switch details.
 Subnet info.
Support team.

Outputs:  Configured DHCP on core Switch.
 Colliers have DHCP scope for three services: DATA, VOICE and WIRELESS.
 If the site is a Cisco environment, then there are sites that have DATA, Voice and Wireless DHCP
scope configured in the Cisco Core switch.
 If the site is a Cisco environment and the site has its own Domain Controller, then DATA and
Wireless DHCP scope is configured in Domain Controller, while Voice DHCP scope still remains in
Cisco Core switch.
 If the site is a Non-Cisco environment and the site has its own Domain controller, then DATA and
Wireless DHCP scope is configured in the Domain Controller, while Voice DHCP scope is
configured in the Cisco CE router.
 If the site is a Non-Cisco environment and there is no Domain controller exists, then DATA,
Wireless and Voice DHCP scope is configured in the Cisco CE router.
 MPLS Link Upgrade –
Scope:  This SOP is used for up gradation of MPLS link on CE Router.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Router details.
 MPLS link info.
Support team.

Outputs:  MPLS link up graded.
Phase 1 –
Test & Turn Up Circuit :-
interface GigabitEthernet0/2
description 20 Mb Link to ATT AVPN
no ip address
ip flow ingress
ip flow egress
load-interval 30
shutdown
duplex full
speed 100
!
interface GigabitEthernet0/2.50
description ATT_AVPN_10.254.160.46/30_ .MMEC.940747..ATI.
bandwidth 20000
encapsulation dot1Q 50
ip address 10.254.160.45 255.255.255.252
ip flow ingress
ip flow egress
Phase 2 –
NO service-policy output QOS-TO-MPLS
Exit
After complete the Phase 1 we will run the BGP session
IP route 0.0.0.0 0.0.0.0 10.254.160.26
We add this route because we dnt want to lose connectivity with ATT
Do NOT save the configuration now, in case if we want to revert back, we can restart the router.
Remove the existing BGP process using command

col-att-torrance-ce01#conf t
col-att-torrance-ce01(config)# No router bgp 65244
Configure the new BGP process with new AS number…below is the sample configuration:
col-att-torrance-ce01(config)# router bgp 65246

bgp log-neighbor-changes
redistribute connected
no synchronization
bgp log-neighbor-changes
redistribute eigrp 10 route-map EIGRP-TO-BGP
neighbor 10.254.160.46 remote-as 13979
neighbor 10.254.160.46 soft-reconfiguration inbound
no auto-summary
exit
col-att-torrance-ce01(config)# router eigrp 10

network 10.252.160.96 0.0.0.7
redistribute static
redistribute bgp 65246 metric 100000 1 255 255 1500
passive-interface default
exit
Now change the bandwidth shaping under QOS Policy-map configuration. The sample configuration
is as below:
policy-map QOS-TO-MPLS
class all-data
shape average 20000000
service-policy BANDWIDTH
Now enable the new interface Gig 0/1 and its sub-interface using “NO SHUT” command.
no shut
exit
interface GigabitEthernet0/2.50
no shut
service-policy output QOS-TO-MPLS
exit
Check the routing table in the CE router using command

Show ip route
Check whether your CE router is learning BGP routes from its next hop PE router new IP
address.
If your CE router is learning the BGP routes from its neighbor PE router, then you can delete the
default route that you have added in the CE router earlier, using command
no Ip route 0.0.0.0 0.0.0.0 10.254.160.26 ----- > removing the temporary default route.
If your site is a Cisco environment and the Cisco Core switch is running EIGRP, then check whether
EIGRP is learning External EIGRP routes in your Cisco Core switch, using command
Show ip route
Now ask the ISA to carry out following Post configuration testing.
Ping and traceroute to another MPLS site or servers.
Carry out Post Speed test and compare it with the speed test done during the Pre-testing period and
see the difference.
Lync
Outlook
Send and receive mail
IP Phone calls
Voice mails
Check the NetQos report and see whether the new 10 Mb is highlighted in to the NetQos report for
that particular site. This may take some time to get reflected into the NetQos report analyser.
Configure SNMP-Server trap command to get traps into the new interface where circuit is
terminated in the CE router. Router will not accept physical interface, as it will only accept sub-
interface, because IP address is configured inside sub-interface. Below is the sample command
snmp-server trap-source GigabitEthernet0/2.50 ----- > Enabling SNMP traps on new circuit.
This command will help GLS to get traps in their monitoring tool.
SAVE the router configuration now, once the Post-testing is completed successfully.
 Create Network diagram in VISIO and update time to time
Scope:  This SOP is used to create VISIO diagram.
Roles:  User
 Requestor
 IT support team
 VISIO software.
 Network details.
Support team.
Outputs:  VISIO diagram updated.
1. Select the Microsoft Visio Program from Start button.

2. Then select Tab Detailed Network Diagram.
3. Select the Network Device tab in that you need to select devices as per your requirements.
4. Save the diagram with .Vdx extensions.
 Coordinating with service providers for circuit outages – Log a ticket and follow-up with
service providers for resolution
We have different service provider as well as Vendor in our environment. As per requirements we
need to coordinate with them. So how to coordinate with them if any ongoing issue is there.
1. ATT – MPLS service provider.

2. GLS – Network Monitor team.
3. Cisco – Device Vendor.
4. Comcast , ATT T&T UP – Local ISP.
We have information stored in our database as per vendor we need to call them and give them
details about issue.
Suppose we need to raised ticket with any vendor we need to follow below steps-
1. Call to vendor or Service provider.
2. We have to give them details which include- Name , Organization , Location , Account Number
which is associated with them , Email , Call back number.
3. Once we create ticket with them you will receive ticket details with mail.
4. So whenever after create ticket we need to take follow up on that ticket on call we need to just
give ticket number to vendor.
 Site network down issue
When any location is down we will received alert as per below.

1. We have to first check from our machine whether this IP is reachable or not.
2. In our machine click on Run and select command prompt.
3. Type ping command and IP address of location.
4. This confirms us that location is unreachable. Now we need to check with local ISA whether
there is any power outage at location or is there any physical connection loss.
5. If any power outage at location we need to wait till power comes up.
6. If there is no power outage then we need to check the location step by step.
7. All cable connections with device needs to be check , try to ping internet connection from core
switch.
8. If ping is resolved then check trace route for global DNS (i.e. 4.2.2.2).
9. At which ip address this trace route will stop. Login to that device and check what is issue or try
to reach from that location to internet.
 Configure wireless AP in Meraki dashboard
1. Login to to the Meraki Dashboard with help of login credentials.

2. Click on add AP tab on that portal.
3. Then in the portal there is search inventory fill box. Enter there meraki access point MAC
address.
4. Then it will show MAC address as per given table. Select the MAC address of the access point
and click on claim.
5. Before add Meraki access point in portal select network for location.
6. Then check whether access point is shown under the prefer location.
7. The access point should show green in color. If that access point is shown as red then check
physical connection with the access point.
 How to Configure Router and Switch in Colliers Environment –
1. We need to check Router details like RAM, Slot etc.
2. Once we received fresh Router we need to login it by default credentials – (cisco/cisco).
3. Configure the Hostname for router.
Note – Naming convention will like Region , Location , Device Number.

4. Configure the Interface according to Requirements.
5. Configure the Radius server for AD credentials on Router , So user can access router with their
AD credentials.
6. Configure the Protocols like EIGRP , OPSF , BGP.

7. Configure the Standard configuration will be set for our Environment.
8. Finally we need to configure Line protocol on router so Remote users can login to the device.
Now how to configure Protocols on Router / Switches –
Scope:  This SOP is used for protocol configuration on Router.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Router details.
 Subnet info.
Support team.

Outputs:  Configured Routing protocol on Router.
EIGRP:
1. Colliers uses EIGRP to advertise the internal LAN subnets within local area network of each site.
So EIGRP is configured in both Core switch and CE router using Autonomous system number 10
using command
router eigrp 10
netwok 10.0.0.0
2. EIGRP then redistribute these internal LAN subnets into BGP to reach the MPLS cloud using
command:
router bgp 65263
redistribute eigrp 10
3. There are also sites where we have static routes configured in the Core switch and we are
redistributing those static routes into EIGRP.
4. There are also sites where we have static routes configured in the Core switch and we are
redistributing those static routes using “route-maps and prefix-list” into EIGRP after filtering the
static routes. This is explained in the earlier section of this document.
5. There are also sites where we use command as “EIGRP STUB”, “redistribute Connected”, “EIGRP
STUB Static”, using command
router eigrp 10
redistribute connected
eigrp stub static
6. EIGRP STUB command is used to configure a router as a stub where the router directs all IP
traffic to a distribution router.
7. The redistribute connected command allows the EIGRP Stub to send connected routes. If the
connected routes are not covered by a network statement, then it will be necessary to
redistribute connected routes with the redistribute connected command under the EIGRP
process. This option is enabled by default.
8. The EIGRP STUB Static command allows the EIGRP Stub to send static routes. Without this
command EIGRP will not send any static routes, including internal static routes. It will still be
necessary to redistribute static routes with the redistribute static command.
Advertising default route using EIGRP:
Assume if the Core switch have EIGRP advertising the internal LAN subnets internally and also
redistributing those routes to BGP.
But at the same time there are sites where we have ANIRA router implemented for failover to MPLS and
this failover get kicked-up using default route with higher distance value as 250. Once the MPLS cloud is
unreachable via BGP and thus External EIGRP stops working, ANIRA router take up the primary role. But
there are sites that have two default routes running, one for ANIRA with higher distance value and
another default route pointing to CE router inside interface with default distance value as “one”.
So when primary MPLS and BGP goes down and External EIGRP stops advertising external routes inside
LAN, then the default static route with default distance value “one” get kicked-up and tries to reach CE
router inside interface and the packet get dropped, reason as the MPLS link is down and the packet does
not get a way to reach next-hop or destination. Thus ANIRA router never turns up. So we need to
manually remove the default static route pointing to CE router inside interface with default distance
value as “one” in Core switch.
To avoid this, we can advertise default route using EIGRP instead of default static route. Because default
static route distance value is greater than Internal EIGRP which is “90”.
Following are the steps to advertise default route using EIGRP.
9. Tell the CE router, if anything that we don’t know send it to PE router using command
IP default-network 10.254.x.0
10. How to reach this default network ? In order to reach the default network we use following
command.
IP route 10.254.x.0 255.255.255.0 10.254.x.x
11. Above command says that all the packets destined for 10.254.x.0 will be sent to PE router inside
interface 10.254.x.x
12. In CE router test this traffic using command “show IP route”.
13. You will see EIGRP routes marked as “D”, while default routes advertised using EIGRP will be
marked as “D*” and the gateway of last resort will be set to “PE router inside interface”.
14. Now we will be able to reach any network in MPLS cloud through CE router. But we will not be
able to reach any network in MPLS cloud through Core switch. So we need to advertise the
default network inside EIGRP using command in CE router.
router eigrp 10
network 10.254.x.0
15. Now if we check the routing table inside Core switch using command “show IP route” we will see
default route marked as “D*” . This means anything that Core switch don’t know, it has to reach
10.254.x.0 network and to reach that 10.254.x.0, it has to reach CE router inside interface.
16. Now Core switch will also be able to reach any network in the MPLS cloud.
17. This process helps to reduce the manual configuration of default static route inside the LAN.
BGP:
1. Colliers uses basic BGP configuration in all the CE routers.
2. Colliers uses BGP routing protocol in CE router to form a neighbor relationshIP with its PE router
using command “neighbor 10.254.140.42 remote-as 13979”.
3. BGP running on CE router learns all the routes coming from different sites via MPLS cloud. This
can be verified using command “show IP route” or “show IP route bgp”.
4. BGP in the Colliers network is configured and running on default attribute called Autonomous
system path (AS path).
5. AS number is provided by ISP AT&T to Colliers to get it configured in our CE routers.
6. AS number assigned by AT&T on all PE routers for Canada region is 8034.
7. AS number assigned by AT&T on all PE routers for America region is 13979.
8. If the site is a Cisco LAN environment, then the internal LAN connectivity VLAN between CE
router and Core switch is advertised by CE router to PE router using BGP network command
“network 10.252.140.72 mask 255.255.255.248”
9. BGP learn rest of the internal LAN subnets from internal LAN routing protocol like EIGRP using
redistribute command “redistribute eigrp 10”, which CE router then advertises it to it neighbor
PE router.
10. If the site is Non-Cisco LAN environment, then the internal LAN connectivity and LAN subnets
are directly advertised by BGP in CE router to its neighbor PE router using BGP network
command “network 10.252.140.72 mask 255.255.255.248”
11. BGP also advertises CE router Loopback interface using network command “network
10.255.194.17 mask 255.255.255.255”
12. PE router then populate all these routes learned from CE router to all its neighbor in the MPLS
cloud, thus each and every router in the Colliers MPLS cloud will populate its routing table with
all the sites routes.
13. There are also sites where we have static routes configured in the CE router and we are
redistributing those static using “route-maps and prefix-list” into BGP after filtering the static
routes. This is explained in the earlier section of this document. Same is also applied while
filtering EIGRP routes into BGP.
14. The return traffic uses BGP from MPLS cloud to reach the site CE router and then these routes
are redistributed inside EIGRP to forward the traffic to the internal LAN subnets using
redistribute command inside EIGRP in the CE router
router eigrp 10
network 10.252.140.0 0.0.0.255
redistribute bgp 65263 metric 100000 1 255 255 1500
 SNMP configuration of devices - Router, Switch
Scope:  This SOP is used for SNMP configuration on Router & Switch.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Router & Switch details.
 SNMP String, SNMP server details.
Support team.

Outputs:  Configured SNMP on Router & Switch.
 We need to configure Radius configuration Router and Layer 3 Switches. That help us to login to
device with AD credentials.
We need to Enable AAA services on Device.
aaa new-model
!
!
aaa authentication login vtylogin group radius local-case
aaa authorization exec vtylogin group radius local
aaa accounting exec vtylogin start-stop group radius
aaa session-id common

exit
Need to configures Lines on device.
line vty 0 15
exec-timeout 0 0
login authentication vtylogin
authorization exec vtylogin
accounting exec vtylogin
transport input telnet ssh
Radius Configuration –
ip radius source-interface Loopback0
radius-server dead-criteria tries 5
radius-server host 10.168.176.14 auth-port 1812 acct-port 1813 key Cr0ok3dDeployment42!
radius-server timeout 15
radius-server directed-request
logging userinfo
SNMP –
snmp-server community coll01 RO
snmp-server community keystone RO
snmp-server community coll01s RW
snmp-server community glsmon RO
snmp-server community glsiiabdfi RO
snmp-server host 10.168.175.212 coll01
snmp-server host 10.255.1.99 glsmon

snmp-server host 10.168.176.249 keystone
snmp-server trap-source “loopback IP”
snmp-server location “office location”
Netflow -
ip flow-export source Loopback0
ip flow-export source “MPLS Link”
ip flow-export version 9
ip flow-export destination 10.168.175.212 9995
NTP –
ntp server 10.255.1.1
snmp-server enable traps snmp authentication linkdown linkup coldstart warmstart
snmp-server enable traps vrrp
snmp-server enable traps ds1
snmp-server enable traps tty
snmp-server enable traps eigrp
snmp-server enable traps ospf state-change
snmp-server enable traps ospf errors
snmp-server enable traps ospf retransmit
snmp-server enable traps ospf lsa
snmp-server enable traps ospf cisco-specific state-change nssa-trans-change
snmp-server enable traps ospf cisco-specific state-change shamlink interface-old
snmp-server enable traps ospf cisco-specific state-change shamlink neighbor
snmp-server enable traps ospf cisco-specific errors
snmp-server enable traps ospf cisco-specific retransmit
snmp-server enable traps ospf cisco-specific lsa
snmp-server enable traps aaa_server

snmp-server enable traps atm subif
snmp-server enable traps bgp
snmp-server enable traps config-copy
snmp-server enable traps config
snmp-server enable traps entity
snmp-server enable traps event-manager
snmp-server enable traps frame-relay
snmp-server enable traps frame-relay subif
snmp-server enable traps hsrp
snmp-server enable traps ipmulticast
snmp-server enable traps msdp
snmp-server enable traps mvpn
snmp-server enable traps pim neighbor-change rp-mapping-change invalid-pim-message
snmp-server enable traps pppoe
snmp-server enable traps cpu threshold
snmp-server enable traps rsvp
snmp-server enable traps ipsla
snmp-server enable traps l2tun session
snmp-server enable traps bulkstat collection transfer
snmp-server enable traps flash insertion remova
snmp-server enable traps cnpd
snmp ifmib ifindex persist
 Troubleshoot BGP routing issue on Router

Scope:  This SOP is used trouble shooting BGP routing issue.
Roles:  User
 Requestor
 IT support team
 AD credentials.
 Router details.
Support team.

Outputs:  BGP routing issue resolved.
Troubleshooting steps in BGP:
Step 1:
We need to confirm that BGP neighbor relationshIP is successfully formed between CE and PE routers
using command:
Show IP bgp summary
The output should look like
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.254.220.62 4 21326 1599736 1555106 111567 0 0 3d08h 665
Note 1: This is a sample output from Pune MPLS router and the above Neighbor IP address and AS
number and others will differ from region to region and location to location.
1. But what we have to see here is that the above “State” should NOT be “Idle / Active”…it should
always be a “numeric number”…..In the above case it is 665.
2. The State/prefix 665 means Neighbor relationshIP is established successfully between CE and PE
routers.
3. If state/prefix is “Active” then in BGP consider it as a bad state, as the CE router is actively trying
to form a neighbor relationshIP with its neighbor PE router. In reality it means trying but not
succeeding.
4. If state/prefix is “Idle” then BGP consider it as a down state, where CE router totally failed to
form a neighbor relationshIP with its neighbor PE router.
5. We need to understand that BGP takes 60 seconds to establish a neighbor relationshIP…because
it is a slow routing protocol.
6. So if there is any issue in forming neighbor relationshIP, then CE router state/prefix will always
change from “Idle” to “Active”.
7. BGP router will wait for 180 seconds (3 keepalive messages) before declaring the neighbor dead.
8. BGP router uses 30 seconds to send updates to its neighbor, whenever changes occurs in BGP.
9. BGP uses 60 seconds to scan the network and learn the network.
10. In above case 665 means prefix number, which says Neighbor relationshIP is established and this
can also be confirmed by using command
Show IP bgp neighbor
This command will display state as “Established” in clear test.
Step 2:
Once you see that neighbor relationshIP is established, then you need to confirm whether
updates are receiving from PE router.
This can be confirmed by using command:
Show tcp brief all
The output should look like
TCB Local Address Foreign Address (state)
63FC9B78 10.254.220.61.179 10.254.220.62.26490 ESTAB
64116024 *.179 10.254.220.62.* LISTEN
11. In the above output TCB means session number. Local address means your CE router IP address
and port number (In our case CE Router). Foreign address means PE router IP address and global
port number.
12. BGP uses TCP port number 179.
13. Session 1 (63FC9B78) say BGP neighbor relationshIP is established between CE and PE router on
your local CE router at TCP port 179.
14. Session 2 (64116024) says BGP in CE router is listening or waiting to receive updates from its PE
router (10.254.220.62).
15. This indicates that your BGP neighbor relationshIP is formed and it is listening to receive
updates.
Step 3:
If above steps fails then clear the BGP neighbor relationshIP whenever you make any changes in
the BGP protocol, during configuring BGP, advertising network, forming neighbor relationshIP
etc. the command is
Clear IP bgp *
16. This clear command will clear the bgp neighbor relationshIP and will reset the BGP process and
establish a fresh neighbor relationshIP.
Step 4:
After BGP neighbor relationshIP is formed, then check the routing table whether you are
receiving the updates from the PE router using the command.
Show IP bgp
The output will look like
Network Next Hop Metric LocPrf Weight Path
*> 10.2.2.0/24 10.254.220.62 0 21326 8034 65100 ?
*> 10.3.0.0/16 10.254.220.62 0 21326 8034 65100 ?
*> 10.3.1.0/24 10.254.220.62 0 21326 21302 64710 ?
17. In above output, ”*”(asterisk) means valid route and > means best routes.
18. If we do not see > sign, that means the router have learned the routes (* sign) but not believed
the routes.
19. If you see > sign, then the routing table will automatically get updated as BGP will send the best
routes to its neighbor routing table and this can be checked using command
Show IP route
Step 5:
Ping and Traceroute
20. Ping IP of any other Colliers MPLS network / internal network
21. Traceroute to any other Colliers MPLS network / internal network.
Step 6:
If you still fail to establish the BGP neighbor relationshIP or if you are not receiving any BGP
updates then run the below debug command.
Debug IP bgp updates

22. This debug command will display if there is any issue while establishing neighbor relationshIP
between BGP routers (CE and PE routers).
Debug IP bgp events

23. This command will display if there is any issue, why we are not receiving the updates from peer
(PE router) or why CE router is not believing the routes as best routes (> sign).
Colliers Firewall details
Introduction:
This document covers the Colliers Firewall architecture and its explanation, and also tried to cover the
maximum knowledge about the Colliers Firewall architecture and its functionality and its end-to-end
communication.
This document has tried to explain using sample configuration examples that are picked from the
production devices in order to make the document explanation much easier way and better to
understand for any new team member.
Design:
Colliers Firewall support covers firewalls in North America sites connecting to Internet and VPN
connections. The updated connectivity LAN diagram for the available locations are uploaded in the
SharePoint. Colliers infrastructure includes Checkpoint firewalls, Palo Alto and ASA firewalls.
Colliers Network includes maximum number of checkpoint firewall deployed at US and CANADA offices.
Palo Alto firewalls are placed in Toronto IDC. There are few ASA firewalls installed in Chicago remote
offices and Mexico locations.
There are also NA sites, where internet traffic goes out via Network Based Firewall. Network based
firewall is deployed in MPLS cloud and managed by AT&T. If any access needs to be enabled on NBFW,
request need to be raised with AT&T. You can find the procedure to raise request with Vendor at the
end of this document.
Toronto IDC Firewalls:

Colliers Toronto IDC has Palo Alto firewall in HA mode and checkpoint Nokia IPSO firewall in cluster
mode. Palo Alto firewall in IDC filters the internet traffic going out via IDC and DMZ traffic. While
Checkpoint firewall handles the VPN traffic from remote offices. Primary checkpoint management server
is also placed in Toronto IDC.
 Checkpoint Firewall Architecture:
Checkpoint firewalls in collier’s network are deployed in distributed architecture. Checkpoint firewall
architecture consist of following components.
1. Smart console :
It is a set of GUI applications that allows security administrators to configure and manage the global
security policy for the entire organization. There are quite a few clients available in the smart console,
each for a different purpose. Of all those clients the main client application used is called
SmartDashboard , which is used to configure the security policy of the network. SmartDashboard
connects to the Security Management Server which houses the actual security policy database of rules
and objects.
Ex. SmartDashboard screen from where you can create firewall policy, push policy on firewall gateway,
create firewall and network objects.
2. Checkpoint management server:
The Security Management Server contains the global security policy for an organization. This policy is
defined using the SmartDashboard—however, the policy is actually saved on the Security Management
Server. It contains the following databases: Object database, User database, Security rules and Log
database. The Security Management Server interacts with the Security Gateways by uploading security
rule sets specific to the Security Gateway and by receiving logging information from the Security
Gateways.
Colliers checkpoint infrastructure includes two management server which are in active/passive mode.
One is placed in Toronto IDC (Primary) and another is in Amsterdam IDC (Secondary).
Toronto IDC Management server IP : 192.168.10.30

Amsterdam IDC management server IP : 10.200.20.6
Procedure to access Checkpoint management server

 Remote Login to terminal server corp-ca-net01
 Open SmartDashboad application installed on the terminal server
 Login using your local credentials created on management server

 You will see global policies as below
 File->open: to open policy for particular location. For example Vancouver. You will see what are
rules are created on Vancouver firewall
Standard Rules included under firewall policy:
With pre-assumption that new firewall is connected to internet and LAN network and also added in
checkpoint management with SIC, below standard rules need to be enabled on firewall
1) Firewall access for admin : -

This standard rule is enabled to provide access to firewall only to authorized colliers external
and internal IP range. This rule will ensure that unauthorized access will not have access to login
page.
2) Blocked IP rule:-
This Rule will block IP addresses identified as blacklisted IP’s (unauthorized source). If any IP
need to be block on collier’s network, it need to be added in “Blocked-IPs” group and policy
need to be pushed on all firewalls.
3) SNMP Rule :
This rule enable access to firewall for SNMP monitoring. This rule will allow firewall to
communicate with NetQos server.
4) Stealth Rule :-
The firewall stealth rule is the explicit rule near the top of the policy denying access to the
firewall beyond what is required to manage the device
5) Clean UP rule :-
The firewall cleanup rule is the explicit rule at the bottom of a firewall policy.
Cleanup rule is required to drop all traffic that did not match any of the other rules
6) Browsing Rule :-
Browsing rule is require for office LAN network to browse internet. This rule include set of ports
which Colliers agreed to enable over internet.
7) VSP traffic :-
If the site has VSP server then then this rule need to be configured to allow VSP server to reach
Twinstrata cloud IP’s
8) Guest Wifi Rules :-

Guest rule is for Gues wifi network to access internet and will have Any Any access however it
should be restricted to have access to colliers LAN ip range. Hence colliers LAN range is negated in
destination field.
3. Security gateway’s :
They are nothing but the ‘firewalls’ you have always known. Security Gateways are installed/located
where the security rules must be applied. So, the security rules are created using the SmartDashboard
which is then saved on the Security Management Server and pushed on the intended Security Gateway.
Checkpoint firewall gateway configuration – 1100 models
Scope:  This SOP is used for Implementing Checkpoint firewall gateway model 1180.
 This SOP is not to be used for Implementing other than Checkpoint firewall &
checkpoint other than 1180 model.
This SOP is not to be used for Implementing Checkpoint Security Management
Server.
Roles:  User
 Requestor
 IT support team
 Issue has been analyzed.
 Smart Dashboard Login credential.
 Checkpoint Firewall 1180 Box.
Support team.

 Colliers policy for Device installation on network.
Outputs:  Checkpoint firewall 1180 has been installed on Network.
1) Configure your computer network adapter to DHCP
2) Connect your computer to port 1 of the firewall
3) Open a browser and go to https://192.168.1.1:4434
4) The following screen will appear:

Click “Next”
5) Enter an Admin password at the next screen and click next
6) Set the time and date manually – we will configure NTP during integration
7) Enter the device name in the next screen- replace Bucharest with the appropriate office but keep the
corp-eu-xxxxx-fw01 syntax
Press next when complete

8) Select central management at the next screen and press next:
9) Enter your Internet IP address information:
Press next
The following screen will pop up:

Press “no” and continue
10) Enter the LAN IP address and disable DHCP and press nest
11) Configure Administrative access per below and press next:

12) Press next at license activation:
Press OK at the warning, we will activate the device during integration:

13) Configure the one time SIC password using key XYZ press next
14) Select connect to management server later and press next:

15) Select finish:
16) You can confirm the network settings be reviewing them on the device tab:
17) Any additional internal networks can be added on the routing page:
18) We are done the initial configuration, once completed please contact firewall.support@colliers.co,
to arrange a time for activation.
Also look through the device there is a lot of reporting included.
For 4200 checkpoint Model initial configuration:
Scope:  This SOP is used for Implementing Checkpoint firewall gateway model 4000
series.
 This SOP is not to be used for Implementing other than Checkpoint firewall &
checkpoint other than 4000 series.
This SOP is not to be used for Implementing Checkpoint Security Management
Server.
Roles:  User
 Requestor
 IT support team
 Checkpoint Firewall 4400 Box.
Support team.

 Colliers policy for Device installation on network.
Outputs:  Checkpoint firewall 4000 series as been installed on Network.
Checkpoint 4200 appliance has different GUI. Once you have access to firewall using GUI, first you need
to assign IP address on interfaces. Go to Network Interfaces ->click on interface which you wish to assign
IP address- > Edit
Enable interface and assign IP address :
Configure Host name and DNS entries:
Add Route:
Go to Static Routes ->Add
Colliers checkpoint firewall usually has default route towards ISP gateway. You also need to add route to
IP subnets on LAN network.
Once firewall configuration is done, we need to create Firewall object on

Checkpoint management server and establish SIC with firewall.
Creating new firwall object on smart Dashboard :-
Scope:  This SOP is use to Create new checkpoint firewall gateway object on smart
Dashboard.
 This SOP is not to be used for Non Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
 Login Credentials for Smart Dashboard.
 Firewall IP address & hostname.
Support team.

 Colliers standard for checkpoint firewall policy package.
Outputs:  Created Checkpoint Firewall object on Smart Dashboard.
1) Login to Security Management Server.

2) Under Network Object, right click on check point, selecct Security Gateway/Management
3) Follow Checkpoint Gateway Installation wizard :-

4) Define General Properties :-
Gateway name: While selecting name for firewall use colliers naming standard
corp-us-nova-fw02
Gateway Platform: select model from drop down menu.
Gateway IP address: IP address of Gateway. Generally use Public IP address for firewall.
5) Create Trusted communication :-
Provide one time password in wizard, it must be same as defined on firewall.
6) Firewall Object has been created on SMS

7) Click Finish
8) Now you need to create SIC between Security gateway & SMS.
9) Double Click on firewall object under Network Objects.
10) Click on communication tab
11) Provide one time password, it must be same which you used on firewall.
12) Click Initialize.
13) Trust created between Security Gateway & SMS.
14) Click on Topology TAB, click on Get, Interfaces with topology.
1) 15 ) Click on Accept
15) Click on install policy.
VPN configuration:
Site to Site VPN Configuration between Checkpoint Gateways:
Scope:  This SOP is used for Implementing Site to Site VPN Configuration between
Checkpoint Gateways.
 This SOP is not to be used for Implementing Site to Site VPN Configuration
other than checkpoint firewall.
 This SOP is not to be used for Implementing Remote Access VPN on
Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
 Phase 1, Phase2 parameters, Pre shared Key, Local & remote networks
details.
Support team.

 Colliers VPN policy.
Outputs:  Site to Site VPN Configuration between Checkpoint Gateways been
installed on Network.
1. The first and foremost point to remember that you must have the reachability between both the
Peers. Verify that the peer IP is correct and reachable.
2. Login to Smart dashboard and go to VPN Tab as shown in Diagram :
3. Select the Site-To-Site VPN type as per your requirement :
Meshed : A Mesh is a VPN community in which a VPN site can create a VPN tunnel with any other VPN site in the
community
Star : A star is a VPN community consisting of central Security Gateways (or "hubs") and satellite Security Gateways
(or "spokes"). In this type of community, a satellite can create a tunnel only with other sites whose Security Gateways
are defined as central.
4. As you select the type, you will see the setup. Give the name and Colour of Site as per
requirement :
5. 2nd Tab – Participating gateways, where you will add the gateways which will be part of VPN
Community. Click on Add and select the appropriate gateway.
6. 3rd Tab – Encryption, here you will define the encryption parameter such as Encryption, Integrity
and IPSec Parameter’s.
7.
Select “Custom Encryption…” and define the parameter as required.
NOTE : Encryption parameter on both the Peer’s must be same .
8. 4th Tab – Tunnel Management, Keep the setting as default.

Permanent Tunnels - This feature keeps VPN tunnels active allowing real-time monitoring capabilities.
VPN Tunnel Sharing - This feature provides greater interoperability and scalability between Security Gateways. It also
controls the number of VPN tunnels created between peer Security Gateways.
9. 5th Tab – Advanced, In Advance tab – Go to Shared Secret
Here you will define the shared secret which will be same on both the peers.
NOTE: 1. if the Checkpoint Gateway belongs to same management server, then no need to
Specify the secret key. Peer’s Secret will be based on ICA Certificate.
2. If the Peer is Interoperable device (3rd Party like Cisco ASA., etc) or Checkpoint
gateway is managed externally, then we need to specify the shared secret.
Note : Shared secret is Case-Sensitive, Set it accordingly.

10. VPN Community settings have been completed, press OK to exit from the Wizard.
11. Now we need to define the traffic, which will be passing through the tunnel. For this open
firewall gateway object (at present our gateway is “corp-eu-amstidc-fw01”).
Select “Topology Tab”….

Here you will find the option to define the VPN Domain. Click on (…) and define the network or host
which should be part of VPN.
Click “OK”.
12. Create a VPN Rule and call the VPN Community in VPN Column:
13. Complete the configuration on Peer end as well. Then save and Push the policy to the respected
Gateway:
14. Open Smart view tracker and check the logs for VPN Community, which should be encrypted as
shown in below Snapshot :
Backups:
Checkpoint Backup and Restore Procedure
Scope:  This SOP is used for Taking Backup and restoration of configuration of
 This SOP is not to be used for Taking Backup and restoration of configuration
of other than Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
 Firewall Access.
Support team.

 Colliers Backup schedule.
Outputs:  Backup & restore of Checkpoint firewall configuration.
(1) Backing Up and Restoring - Gaia Portal
(1-A) To create a backup (Gaia Portal)

 In the tree view, click Maintenance > System Backup.
 Click Add Backup.
The New Backup window opens.
 Select the location of the backup file:
 This appliance
 TFTP server. Specify the IP address.
 SCP server. Specify the IP address, user name and password.
 FTP server. Specify the IP address, user name and password.
(1-B) To restore from a backup (Gaia Portal)
 Before restoring from backup, machine need to be configured with previous hostname.
Otherwise, double reboot is needed post the restore to make the machine active.
 In the tree view, click Maintenance > System Backup.
 Select the backup file and click Restore Backup.
 A pop up messages indicates a restore is in progress until it is done.
 Reboot the machine once restore is done.
 Install policy.
(2) Backing Up and Restoring - in Clish
Scope:  This SOP is used for Taking Backup and restoration of configuration of
Checkpoint firewall in clash mode.
 This SOP is not to be used for Taking Backup and restoration of configuration
of other than Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
Support team.

Outputs:  Backup & restore of Checkpoint firewall configuration using command line
(Clish mode).
(2-A) To create a backup (Clish)
Commands:
Use one of the following commands, depending on the backup type:
 To save a backup locally:

o add backup local
 To save a backup on a remote server using FTP:
o add backup ftp ip VALUE path /some/path/ username VALUE password plain
 To save a backup on a remote server using TFTP:
o add backup tftp ip VALUE
 To save a backup on a remote server using SCP:
o add backup scp ip VALUE path /some/path/ username VALUE password plain
Command Parameters:
 ip VALUE - The IP address of the remote server.

 username VALUE - User name required to log in to the remote server.
 password plain - At the prompt, enter the password for the remote server.
 /some/path/ - Path to stored backup
Example:
HostName> add backup local

Creating backup package. Use the command 'show backup status' to monitor creation progress.
Notes:
 Backup configurations on Check Point appliances are stored in /var/log/CPbackup/backups/

 Backup configurations on Open Servers are stored in /var/CPbackup/backups/
To apply Hotfix on checkpoint gateway:
Scope:  This SOP is used for applying hotfix on Checkpoint firewall 4400 series
model.
Roles:  User
 Requestor
 IT support team
 Win SCP software.
 Hotfix which need to installed on firewall.
Support team.

Outputs:  Hotfix installed on 4400 series Checkpoint firewall.
1. Most of the cases hotfix file will not be available on checkpoint site hence need to contact
support center to get hotfix file and md5 file to check hash value.
2. You will require WINSCP to move file from local system to checkpoint gateway.
3. Connect winSCP with firewall IP address and login credentials.
4. If you see error regarding shell incompatibility enter below command on checkpoint firewall in
expert mode
- firewall#chsh -s /bin/bash admin
5. once connected to checkpoint via WINSCP create folder (ex. Hostfix) under /var directory
6. copy file from local system to /var/hotfix

7. Use below command to generate md5 value and compare with value received from checkpoint.
Firewall#md5sum sim_HOTFIX_GYPSY_HF_BASE_126.tgz
8. Extract the package using below command.
Firewall#tar –xf <package name>

Firewall#tar –zxvf <package name>
9. install hotfix using below command :
Firewall#./extracted_filename
Firewall#./UnixInstallScript
( example : sim_HOTFIX_GYPSY_HF_BASE_126_990126001_1)
10. done and verify if you get below output

Checkpoint 1100 Upgrade Process
Scope:  This SOP is used upgrading firmware on Checkpoint firewall 1100 series model.
Roles:  User
 Requestor
 IT support team
 Firmware image.
Support team.

Outputs:  Firmware upgraded on 1180 series Checkpoint firewall.
1) On the home screen go to system operation

2) Select Manual Upgrade
3) Select the image and press Ok
4) Select Upload File
5) Wait while the image loads

6) Press next when the image has loaded
7) Press next to start the upgrade process
8) The appliance will now install the upgrade and reboot when finished
9) The login screen will appear after the upgrade has finished
10) The system information screen should show Version: R75.20.70
Upgrade Checkpoint Management Server
Scope:  This SOP is used for upgrading the Checkpoint Management Server.
Roles:  User
 Requestor
 IT support team
 Checkpoint Management server Access.
 Checkpoint support center access.
 Win SCP software.
 Hotfix which need to installed on management server.
Support team.

Outputs:  Hotfix installed on 4400 series Checkpoint firewall.
Requirements:
Make sure there is enough free disk space to do the upgrade.
Check the space available for images in the Maintenance > Image Management page.
o Using the CLI: In expert mode, run the df -h command and check the available
space in /var/log.
Up-gradation using the WebUI:

1. Download the Gaia upgrade package from the Check Point Support Center to the Gaia
WebUI client computer.
Check_Point_upg_WEBUI_and_SmartUpdate_R77.Gaia.tgz
2. Connect to the Gaia WebUI from a Web browser to
https://<management_IP_address>
3. In the WebUI go to the Maintenance > Upgrade page. (Ensure the View
Mode is Advanced.)
1. Click Upload.
2. Browse to the location of the upgrade package.
3. After the package is uploaded, either click Done to add the package to
the Upgrade Packages repository, or click Upgrade.
If you added the package to the package repository, select the package, and
click Upgrade.
The package is extracted.
4. After the package is extracted, click OK.
A console window opens.
You are asked if you want to save a snapshot of the system before upgrade. It always
recommended to save.
5. The pre-upgrade verifier runs. The output is stored in a text file
at /tmp/pre_upgrade_out.txt.
6. If you see the error: "Pre-upgrade verification failed" it recommend that review
the file, fix the problems, and restart the upgrade. Do not take another system
snapshot.
7. You are asked if you want to start the upgrade. Select Yes.
8. After the upgrade, click Reboot.
Up-gradation using CLI:

You can upload the TGZ to the WebUI, and upgrade Gaia with CLI commands.
1. Download the Gaia upgrade package from the Check Point Support Center.
Check_Point_upg_WEBUI_and_SmartUpdate_R76.Gaia.tgz
2. In the Gaia CLI, enter expert mode.
3. Use FTP, SCP or similar to transfer the upgrade package to the Gaia appliance or
computer. We recommend that you place the package in /var/log/upload.
4. Exit expert mode.
5. In clish, register the file as an upgrade package. Run the command:
add upgrade <version> package file <full path>
6. Run:
upgrade local <version>
For example:
upgrade local R76
You are asked if you want to save a snapshot of the system before upgrade. It always
recommended to save.
If you see the error: "Pre-upgrade verification failed" we recommend that you review
the file, fix the problems, and restart the upgrade. Do not take another system
snapshot.
9. After the upgrade, type OK to reboot.
To upgrade using an ISO image on a DVD:
1. Download the Gaia ISO image from the Check Point Support Center.
Check_Point_Install_and_Upgrade_R76.Gaia.iso
2. Burn the ISO file on a DVD.
3. Connect an external DVD drive to a USB socket on the appliance or computer.
4. Run upgrade cd
5. You are asked if you want to save a snapshot of the system before upgrade. It
always recommend to save first.
7. If you see the error: "Pre-upgrade verification failed" we recommend that you
review the file, fix the problems, and restart the upgrade. Do not take another system
snapshot.
The upgrade takes place.
9. After the upgrade, before rebooting, remove the DVD from the drive.
10. Type OK to reboot.
Changing ISP for checkpoint firewall
Scope:  This SOP is use to change IP address of ISP on Checkpoint firewall.
 This SOP is not to be used for change of IP address for other interface on
Roles:  User
 Requestor
 IT support team
 Access Login Credentials for Checkpoint Firewall & Smart Dashboard.
 ISP IP address.
Support team.

 Colliers standard for checkpoint firewall policy package.
Outputs:  ISP IP address changed on Checkpoint firewall.
When Internet service Provider changes, we need to change the external interface IP address of
checkpoint firewall.
Step to follow to change the ISP IP address on Checkpoint firewall:-
2) Login to Security gateway

3) Click on Device, under Network option click on Internet button,
4) Click on Edit button & fill the IP address, subnet mask & default gateway as per provided by ISP.
5) Click on Apply.
6) Default route will get added automatically under Routing option.
7) Now you need to reestablish SIC between security gateway & Management server. Under Home
Tab click on Security Management, click setup.
8) Click on Next.
9) System will prompt you for set-One Time Password (SIC)
10) Click Next
11) Write your SMS server IP in Management IP field & click on Connect
12) Click Finish
13) Go to Management server & Select policy Package.

14) Select Checkpoint firewall object under Network Object for which we need to change the ISP IP,
Double click on selected firewall object.
15) In General Properties change the IP address of firewall in IPv4 Address section.
16) Click OK
17) Click on Topology TAB, click on Get, Interfaces with topology.
18) Click on Accept
19) Click on communication.
20) Click on Reset.

20 ) Type one time password, it should be same which is set on security gateway. Check Initiate Trusted
communication now & click on Initialize.
21) After getting Trust established message click on OK.
22) Click on Test SIC Status.
23) Click Close.
24) Install the policy on Firewall.

Creating Policy Package on smart Dashboard
Scope:  Creating New Policy Package on Smart Dashboard.
Roles:  User
 Requestor
 IT support team
 Firewall Object should be created on smart dashboard to use in firewall
rule base.
Support team.

 Colliers standard for Firewall Rule base.
Outputs:  Created Policy Package on Smart Dashboard.
1) Login to Checkpoint management server

2) Click on File >>>> New.
3) It prompts you below message
4) Click ok
5) New Policy Package screen appears. Type Name of Package.
6) Create the standard firewall rule as per below.
7) Click on Save button
SNMP configuration on firewall gateways
Scope:  Configuring SNMP configuration on Checkpoint firewall.
 This SOP is not used for configuration of SNP configuration on other devices.
Roles:  User
 Requestor
 IT support team
 Checkpoint firewall & SMS login credentials.
 SNMP server IP, community details.
Support team.

 Colliers standard for Firewall Rule base.
Outputs:  SNMP configuration on security gateway.
1) Login to Checkpoint firewall with standard credentials.

2) From the home screen select the “Logs and Monitoring tab”
3) Go to SNMP under the diagnostics section
4) Turn SNMP on
5) Select configure on the top banner

6) Enter System Location(office) and community string – coll01
7) Press Apply
8) Turn on the “high CPU Utilization trap”
9) Double click High CPU Utilization
10) Select enable trap

11) Press Apply
12) Login to Smart Dashboard
13) Create the below rule on firewall rulebase & push the policy on particular firewall
Troubleshoot routing issue on Checkpoint firewall

Scope:  This SOP is used troubleshooting the routing issue on Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
 Smart dashboard access
Support team.

Outputs:  Routing issue has been resolved.
Smart View Tracker is one of most useful tool in checkpoint console bunch, its provides a very
realistic way to find out the respective communication in current logs.
Smart View Tracker's filtering mechanism allows you to conveniently focus on log data of
interest and hide other data, by defining the appropriate criteria per-log field. Once you have
applied the filtering criteria, only entries matching the selected criteria are displayed.
The filtering options available are a function of the log field in question. For example, while
the Date field is filtered to show data that is after, before or in the range of the specified date,
the Source, Destination and Origin fields are filtered to match (or differ from) the specified
machines.
Smart View Tracker records the Firewall Rule Base rule to which a connection was
matched.
The matching rule is recorded in four columns in Smart View Tracker, as follow:
Entry Description
No. The log entry number.
Date The date the log entry was generated.
Time The time the log entry was generated.
Product The product for which this log entry is relevant.
Interface The interface on which the packet being logged came in or went out (with direction) or
indicates "daemon" if the message came from a FireWall-1 daemon (e.g., the security
servers, fwm).
Origin The firewall module that generated the log entry.

Entry Description
Type Log (normal rulebase logging), Alert (for alert log entries), or Control (changes to policy or
logging on a firewall).
Action The action taken on the packet (Drop, Reject, Accept, and so on).
Service The service of the packet (HTTP, Telnet, and so on), which is usually based on the destination
TCP/UDP port.
Source The source IP address of the connection or packet.
Destination The destination IP address of the connection or packet.
Proto The protocol of the IP packet (TCP, UDP, ICMP, and so on).
Rule No. The rule number that this connection or packet matched in the rulebase.
NAT rule num. The NAT rule number that this connection or packet matched in the rulebase.
Nat add. rule The number of an additional NAT rule if one was applied.
num.
Source Port The source port of the packet if TCP or UDP. For other protocols, this field is gibberish.
User The appropriate username if the action was an authorization or deauthorization or was the
result of an authenticated connection (with or without encryption).
SrcKeyID The source's KeyID if the action was encrypt or decrypt.
DstKeyID The destination's KeyID if the action was encrypt or decrypt.
Elapsed The amount of time the connection was active (Accounting mode only).
Bytes The number of bytes for the connection in question (Accounting mode only).
XlateSrc The source IP address the connection will have after NAT is applied.
XlateDst The destination IP address the connection will have after NAT is applied.
XlateSPort The source port the connection will have after NAT is applied.
XlateDport The destination port the connection will have after NAT is applied.
Partner The name of the partner site making a connection if an extranet is defined.
Community The name of the community to which this log entry relates if a VPN Community is used.
Enc Scheme The encryption scheme used for this connection (usually IKE, but may be others if managing
FireWall-1 4.1 boxes).
Entry Description
VPN Peer The gateway that sent or received the encrypted packet.
Gateway
IKE Initiator Information relating to the IKE communication.

Cookie
IKE Responder Information relating to the IKE communication.

Cookie
IKE Phase 2 The ID of the IKE negotiation.

Msg Id
Encryption Lists all the encryption methods used in both encryption and data integrity.
Methods
Info More information about this log entry. For most packets or connections, it will simply show
"len X," where X is the number of bytes in the packet. This entry also shows useful
information on encrypt/decrypt log entries and drops or rejects on Rule 0.
Smart View Monitor is a tool which helps to verify about any specific traffic which has been
allowed or blocked by pre-defined rules in respective Security gateway in Smart Dashboard, but
in certain cases might possibility that traffic logs not shown on Tracker in that case we can
check directly on Security Gateways with the help of following commands:
1> TCPDUMPS
2> FW MONITOR
 TCPDUMPS:
A common step in troubleshooting is finding out what not to troubleshoot. With a packet
capture you can confirm things such as routing, firewall rules, and remote services.
TCPDUMP is a powerful tool for debugging on checkpoint, tcpdump feeds directly to the
screen packets crossing an interface, if dumped to a file TCPDUMPS can be read by
wireshark. you need to be in expert mode to invoke TCPDUMP.
Eg:
tcpdump -nni <interface> host <ip>
tcpdump -nni <interface> net <ip>/<cidr>
tcpdump -nni <interface> host <ip> and port <port>
tcpdump -nni <interface> vlan <vlan #> and host <ip>
tcpdump -w <file>.cap -s 1514 -nni <interface> host <src> and host <dst> !! captures entire packet into file
tcpdump -r <file>.cap !! Replay the capture from the file
tcpdump -nni <interface> host <ip> & !! & symbol puts capture in the background
tcpdump -nni <interface> $host <ip> or host <ip>$ and $host <ip> or host <ip>$
tcpdump -nni <interface> ip proto 112
Saving a TCP dump in a .pcap file:
tcpdump -w capture.pcap -i eth-s1p2c0 host 10.1.1.1 and host 20.2.2.2

tcpdump -nni any host 10.1.1.1 -w capture.pcap
tcpdump -nni any host 10.1.1.1 and host 20.2.2.2 -w capture.pcap
 FW MONITOR:
Check Point's fw monitor is a powerful built-in tool to assist with inspecting and capturing network traffic at
the packet level. The fw monitor utility captures network packets at multiple capture points along the VPN-
1/FireWall-1 inspection chain. These packets can be inspected using either Wireshark or Check Point's
CPethereal.
HOW FW MONITOR WORKS
There are four inspection points along the passage of a packet through a firewall:
1. Before the virtual machine, in the inbound direction (i or PREIN)

2. After the virtual machine, in the inbound direction (I or POSTIN)
3. Before the virtual machine, in the outbound direction (o or PREOUT)
4. After the virtual machine, in the outbound direction (O or POSTOUT)
Disable SecureXL for capturing proper logs:

 If you have SecureXL enabled, it fast switches the packets and won’t show you all the detailed logs
that you would like to see in your captures. You can disable SecureXL temporarily, if you want to
inspect packets at that granularity (i.e ‘I’ and ‘o’). In my experience, disabling SecureXL hasn’t been a
problem* and I haven’t seen any performance impact as such.
Right before the capture, turn off SecureXL on the gateway:
fwaccel off
Immediately after the capture, turn on SecureXL on the gateway:
fwaccel on
 Routing Issues:
Sometime after all done all troubleshooting steps not able to diagnose the reason behind this
why communication is not happened, then probably the issue will have routing that the reason
traffic will not able to reach at defined destination or not able to get return traffic as well.
e.g.: Briefing here one scenario for ICMP traffic between Security Gateway & Security
Management, let’s if any communication fails, and security policy allows ICMP, then it is most
likely a routing issue on Security Gateway. Check the routing table on Security Gateway - there
has to be a route to Security Management Server's network / Security Management Server's IP
address and added if required.
Command:
netstat –rn
 Services related issue:
All above troubleshooting tools and step are co-related with each other when try to
troubleshoot any issue.
Let’s when we get any issue regards to particular service/s are not working, as concerned about
the access that have already provided on required firewall but still not accessible in that case
first of all we apply filter on Smart Tracker and try to track the specific traffic which having issue
and as above mentioned in Smart Tracker section we can verify the respective traffic is it
allowed or dropped and troubleshoot accordingly.
It means we can determine any access with help of logs on Smart Tracker and treat accordingly
to allow the access and add new rule in rulebase of Security Gateway and installed the policy on
Gateway/s. Such as required services RDP, http, https, DNS, DHCP, and FTP etc.
Troubleshoot Firewall Issue
Scope:  This SOP is used for troubleshooting the various issue’s on Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
Support team.

Outputs:  Issue has been resolved.
Problems with CPU on Security Gateway
There some basic keyword to monitor /check the CPU utilization of any particular
gateway.
Key words:
--- load
--- softirq
--- interrupts
Commands:
(1) cpstat -f cpu os
 Displays internal statistics for OS about CPU as collected by Check Point.
 Helpful in monitoring CPU utilization.

(2) top
 Displays dynamic real-time view of a running gateways.
 Helpful in monitoring different aspects of CPU utilization.
 We can sort the details as per our requirement like as we can sort as per cpu usage as shown in
below screen shot (Ctrl M).
 To exit press combination of Ctrl + C keys (simultaneously).

(3) ps auxwf
 Displays information about the current processes (daemons).
 Helpful in detecting problems in User Space (Memory, CPU).
 Look at the amount of "CPU", "MEM", "VSZ", "RSS", "TIME" consumed by the daemons.
Problems with memory on Security Gateway
There are some keyword we need to keep in mind while check / monitor gateways for memory
usage.
Key words:
--- memory
--- swap
Commands:
(A) cpstat -f memory os
 displays internal statistics for OS about memory as collected by Check Point
 helpful in monitoring memory utilization
(B) fw ctl pstat
 Displays FireWall internal statistics about memory and traffic.
 Helpful in monitoring memory utilization, traffic counters, ClusterXL Sync counters.
 No single field that indicates a problem - need to interpret all counters together.
(C) fw tab -t connections -s
 Displays summary about connections in Connections Table.
 Helpful in monitoring the amount of concurrent connections.
 Collect the output several times to see how fast the #VALS counter changes.
 Calculate the ratio of the #SLINKS counter to the #VALS counter.
 Compare the #PEAK counter to the limit of Connections Table (fw tab -t connections | head -n 3
| grep limit).
Logging related issue:

Troubleshooting Check Point logging issues when Security Management Server / Log Server is not
receiving logs from Security Gateway.
As per our environment Checkpoint Appliances has been deployed in distributed environment. When
troubleshooting logging related issues in a distributed environment, we need to proceed as follows:
1. In Smart Dashboard, go to 'Policy' menu - click on 'Install Database...' - select the Security Management
Server and Log Servers - click 'OK'.
2. Ensure that you have not run out of disk space on the Security Management Server / Log Servers, to
which the logs are being sent:
 On Gaia:
Command - check the "Use%" column
Run df –kh
 On Windows OS:
On Desktop, open 'My Computer' - right-click on the relevant hard disk - click on 'Properties' - check the
"Free space" line.
3. If needed, delete or move the unneeded logs / files to an external storage device.
4. Is Security Gateway configured to send logs to Security Management Server / Log Server?
In Smart Dashboard, open the Security Gateway object - check each setting in the "Logs" section.
If any change was made, install policy.
5. Is Security Management Server able to communicate over SIC with Security Gateway?
In Smart Dashboard, open the Security Gateway object - on 'General Properties' pane, in "Secure
Internal Communication" section - click on "Test SIC Status...".
If this fails, then it might be due to connectivity / routing issues between the Security Gateway and the
Security Management Server.
6. Is Security Gateway able to communicate (other than SIC) with Security Management Server?
Test by sending pings from the Security Gateway to the Security Management Server.
Test by sending pings from the Security Management Server to the Security Gateway.
Note: Security policy must allow ICMP between the Security Gateway and the Security Management
Server.
If this fails, and security policy allows ICMP, then it is most likely a routing issue on Security Gateway.
Check the routing table on Security Gateway - there has to be a route to Security Management Server's
network / Security Management Server's IP address:
netstat –rn
7. Is Security Management Server listening on TCP port 257?
 On Gaia:
# netstat -anp | grep ":257"
 On Windows OS:
netstat -abno | findstr ":257"
8. Check the Log Policy settings in log_policy.C file on Security Management Server:
Note: Settings in this file have to match the settings in Smart Dashboard in Security Management Server
object.
 On Gaia:
$FWDIR/conf/log_policy.C
 On Windows OS:
%FWDIR%\conf\log_policy.C
Configure OSPF routing on Checkpoint Firewall
Scope:  This SOP is used to configure OSPF routing on Checkpoint firewall.
Roles:  User
 Requestor
 IT support team
Support team.

Outputs:  OSPF Routing has been configured.
Open Shortest Path First (OSPF) is an interior gateway protocol (IGP) used to exchange routing
information between routers within a single autonomous system (AS). OSPF calculates the best path
based on true costs using assigned metric number. OSPF has a quicker convergence, and provides equal-
cost multipath routing where packets to a single destination can be sent using more than one interface.
OSPF is suitable for complex networks with a large number of routers.
Gaia supports OSPFv2, which supports ;IPv4 addressing.
(NOTE: We can run OSPF over a route-based VPN by enabling OSPF on a virtual tunnel interface)
To Configuring OSPF - WebUI

1. Login to Security Gateway through WebUI(browser).
2. In the Network Management > Network interfaces, configure Ethernet Interfaces and
assign an IP address to the interface.
3. Open the Advanced Routing > OSPF option (Left tab).
4. Define other Global settings, including the Router ID.
5. Optional: Define additional OSPF areas (in addition to the backbone area).
6. Optional: For each area, you can add one or more address ranges if you want to reduce
the number of routing entries that the area advertises into the backbone.
Note - To prevent an address range from being advertised into the backbone,
selectRestrict for the address range
7. Configure OSPF Interfaces.

8. Configure virtual links for any area that does not connect directly to the backbone area.
To configure an OSPF interface:
1. In the Edit Interface window, assign the appropriate Area to each interface by selecting
the OSPF area that this interface participates in.
The OSPF interface configuration parameters are displayed showing the default settings. If you
want to accept the default settings for the interface, no further action is necessary.
2. (Optional) Change any configuration parameters for the interface.
Note - The hello interval, dead interval, and authentication method must be the same
for all routers on the link.
SMS Logs movement:-
Scope:  This SOP is used to Logs from Security Management server to FTP server.
Roles:  User
 Requestor
 IT support team
 Access to FTP server.
 WinScp software running on FTP server
 Login credentials for SMS
Support team.

Outputs:  Logs moved from SMS to FTP server.
Steps to move logs from SMS to FTP server.
1) Login to FTP server 192.168.10.14 with below credentials

192.168.10.14(For RDP) administrator Pr0perty
2) Login to SMS using winscp.

3) find the Logs from SMS as per below path and select the folder where you want to move the logs on
FTP server.
4) Select the Logs from SMS to move on FTP server.

5) Right click on selected logs and click on copy / move.
Palo Alto Firewall:

 Getting started with Palo alto firewall :
Palo Alto firewalls are in HA mode and managed by Panorama. You can access Panorama using below
link. Use credentials for individual admin created on Panorama.
https://10.168.176.31/php/login.php
How to add rules on Palo Alto firewall:
Scope:  Adding Rule on Palo Alto Firewall that user need to access destination from
Source on Specific Ports for business purpose.
 This SOP is not to be used for Adding Rules on other than Palo Alto firewall.
This SOP is not to be used for adding rule that have been explicitly denied
access in accordance with company policy.
Roles:  User
 Requestor
 IT support team
 Palo Alto firewall Access & Login credentials.
 Source, Destination & Service port details.
Support team.

 Colliers Standard Rules policy.
Outputs:  Rule has been added on Palo Alto firewall
Login to Panorama with your credentials.
1) Click on Policies TAB on Panorama Dashboard.

2) Click on Add tab at bottom of screen.
3) Below screen will appear
4) In General tab you can give Name to Firewall rule.
5) Under source Tab you need to specify the Source Zone & source address as per requirement.
6) Under Destination Tab you need to specify the destination Zone & destination address as per
requirement.
7) Under service tab as per requirement add services which needs to be open for this rule
8) Select action as per requirement whether to allow or block the traffic for source to destination
on particular port.
9) Select Target where you want to install firewall rule.

10) Click OK.
11) Click on commit button.
12) Now you need to commit the changes on Panorama & Device Group. First commit the changes
on Panorama then on Device group.
How to add routes on Palo Alto firewall:
Scope:  Adding Route on Palo Alto Firewall that user need to reach to destination.
 This SOP is not to be used for Adding route on other than Palo Alto firewall.
This SOP is not to be used for adding route that have not been permitted in
accordance with company policy.
Roles:  User
 Requestor
 IT support team
 Destination IP/Subnet & Next hop details.
Support team.

Outputs:  Route has been added on Palo Alto firewall
1) Click on drop down menu & select the firewall object where you want to add routes.
2) Click on Network tab on dashboard.
3) Click on Virtual Routers under Network Tab.
4) Click on Virtual router where you want to add routes on firewall
5) Under static routes option click on Add button to add static route on particular virtual router.
6) Write name in name field, define destination address / subnet in Destination field, select
interface according to exit interface of firewall for that subnet or select Next Hop, define Admin
Distance if any then click on OK button.
7) Click on commit button to apply changes on firewall.

8) Click OK to commit the changes.
URL Filtering on Palo Alto Firewall:
Scope:  Whitelisting of URLs on Palo Alto Firewall that user need to access for business
purposes & which has not been explicitly banned.
 This SOP is not to be used for whitelisting of URLs on other than Palo Alto
firewall.
This SOP is not to be used for whitelisting of URLs that have been explicitly
banned in accordance with company policy e.g. Porn site etc .
Roles:  User
 Requestor
 IT support team
 Categorization of URL has been checked.
Support team.

 Colliers Whitelisting policy.
Outputs:  URL has been added in Whitelist on Palo Alto firewall
1) Login to Panorama.
2) Click on Object on Dashboard
3) Click on URL filtering under Security Profiles

4) Colliers URL filtering rule is present on firewall, select that rule.
5) Click on Colliers URL filtering rule & add URL under Block List or Allow List as per your
requirement.
6) Click OK.
7) Click on commit to save & apply changes on firewall.
How to schedule back up in Panorama:
Scope:  This SOP is used for Scheduling Back up On Panorama.

Roles:  User
 Requestor
 IT support team
 Categorization of URL has been checked.
Support team.

 Colliers Whitelisting policy.
Outputs:  URL has been added in Whitelist on Palo Alto firewall
1) Go to Panorama -> scheduled config export :
2) Click on Add and configure below setting. We need to specify FTP server IP address on which
backup will be stored and FTP server FTP credentials :
3) Commit the changes.
Network Based Firewall:
There are few NA sites, where internet traffic is going out via Network based firewall managed by AT&T.
Network based firewall is situated in MPLS cloud and managed by At&T. If we need to allow any IP/Ports
or URL, we need to raise request with AT&T to enable the access.
 ATT CONTACT INFORMATION
Kindly find below contact information to log ticket with AT&T to do live troubleshooting for NBFW –
18007272222
Option 8
Option 2
Option 2
Provide your business direct ID (colliexxx) if they ask.
 LOGIN INFO
You can login to ATT Network Based Firewall using below mentioned link –
https://www.e-access.att.com/iprotent/
If you wanted to raise a change then click on Manage my Network Security (MACD) and it will redirect to
https://srvc.mss.att.com/
 REPORT’s
You can generate customized reports depends on requirement by clicking on Report Section on left
hand side as below –
HOSTNAME NBFCIL02F
IP ADDRESS 134.24.19.28
USERNAME collieXX (Where XX is initial of your first name and last name)
NOTE: To create Username / Password you need to contact Beninger, Kevin

<Kevin.Beninger@colliers.com>
Service-Now ticketing tool:
Service-now ticketing tool is a tool that covers both Incident and Change tickets.
Colliers-Zensar onsite service desk team raises the incident ticket and coordinates between the end user
and the technician from the start of the ticket raised and till it is resolved.
While change tickets are raised by technician himself who implements the actual change in to the
device.
https://colliers.service-now.com
Incident Ticket flow will be:

End user issue addressed to ---- > ISA of the site, calls ----- > service desk team, raises ----- > incident
ticket in Service-Now ticketing tool and assign it to ------ > network group ----- > technician from the
network group has to pick the incident ticket and assign it to his own name and start working ------- >
Technician responsibility is to keep updating the ticket within the SLA period ----- > once issue is fixed,
technician responsibility is to resolve the ticket with the resolution ----- > Service-desk team will be
responsible to carry out follow-ups till the ticket is resolved.
Process:
All incident tickets will be raised by Zensar Service desk team via Service-now ticketing tool and assign it
to their respective support team. The end-to-end incident ticket will be followed through proper SLA
matrix.
While change tickets will be raised by the Colliers network team technician himself and get the change
ticket approved via CAB meeting that is scheduled on every Wednesday at 10 am PST.
Contact details:
CHECKPOINT
1) Firewall team can raise request with Herjavec team for any checkpoint firewall issues.
2) If we call up Herjavec, we just need to tell them that we are speaking from colliers and our name
/ email ID and they will help us with the issue. In case, if they need to go to checkpoint for help
then they would need the account id first then the serial no or the MAC address if Checkpoint to
open the case.
3) If we directly call checkpoint they would need the account id and then if they want to open a
case they would need the serial no or the certificate key of the box.
Account Id: 0005261497
MAC / Certificate Key: Depends for which box we logging the request.
PALO ALTO
1) Firewall team can raise request with Herjavec team for any checkpoint firewall issues.
2) If we call up Herjavec, we just need to tell them that we are speaking from colliers and our name
and they will help us with the issue. In case, if they need to go to Palo Alto for help then they
would need the serial no of the box to open the case.
3) If we directly call Palo Alto, they would need the serial no of the box to open a case.
Primary Box Serial No: 0002C101388

Secondary Box Serial No: 0002C101381
CISCO
Contact Information –
Customers and Partners: 1-800-553-6387
Technical Support: 1-800-553-2447 or 1-408-526-7209
Email – tac@cisco.com
Open TAC Case - https://tools.cisco.com/ServiceRequestTool/scm/mgmt/case?referring_site=tsmodel
RMA - http://www.cisco.com/c/en/us/support/rma_portal.html
CONTACT INFO
Herjavec: e-mail: support@herjavecgroup.com
Tel No: 1-866-575-4909
Checkpoint:
Tel No: +1 (972) 444 6600 / +1 (888) 361 5030 / +1 (613)271-7950
Online chat on checkpoint.com - Web Service Request via supportcenter.checkpoint.com (but we would
require a checkpoint id to login and raise a request)
Palo Altos:
Phone: US: (866) 898-9087
Int'l: +1 (408) 738-7799
Email: support@paloaltonetworks.com
Web Service Request via support.paloaltonetworks.com (but we would require a Palo Alto ID to login
and raise a request)
ASA Firewall
Basic ASA configuration
Scope:  This SOP is used for Basic ASA configuration.
Roles:  User
 Requestor
 IT support team
 ASA device access & login credentials.
 Hostname, IP address, subnet details
Support team.

Outputs:  ASA configured & implemented on network.
1) Configure the hostname, IP address, and zone and security level to interface.
2) Verify the ip address configured on interface.
3) Configure the access list and apply it on interface with direction
4) Create object group
5) Create service object

6)Make an access-list and apply it on interface.
Configure site to site VPN on cisco ASA
Scope: This SOP is used to create site to site VPN on cisco ASA using WebUI.
Roles:  User
 Requestor
 IT support team
 ASA Login credentials. ASDM access.
 Phase1, phase2 parameters, Preshared key, Peer IP address, Local &
remote subnet details.
Support team.

Outputs:  Site to site VPN configured on Cisco ASA firewall.
This section describes how to configure the site-to-site VPN tunnel via the Adaptive Security Device
Manager (ASDM) VPN wizard.
Network Diagram
This is the topology that is used for the examples throughout this document:
Configure Via the ASDM VPN Wizard

Complete these steps in order to set up the site-to-site VPN tunnel via the ASDM wizard:
1. Open the ASDM and navigate to Wizards > VPN Wizards > Site-to-site VPN Wizard:
2. Click Next once you reach the wizard home page:
3. Configure the peer IP address. In this example, the peer IP address is set to 192.168.1.1 on Site B. If
you configure the peer IP address on Site A, it must be changed to 172.16.1.1. The interface through
which the remote end can be reached is also specified. Click Next once complete.
4. Configure the local and remote networks (traffic source and destination). This image shows the
configuration for Site B (the reverse applies for Site A):
5. On the Security page, configure the pre-shared key (it must match on both of the ends).
Click Next once complete.
6. Configure the source interface for the traffic on the ASA. The ASDM automatically creates the
Network Address Translation (NAT) rule based on the ASA version and pushes it with the rest of the
configuration in the final step.
Note: For the example that is used in this document, inside is the source of the traffic.
7. The wizard now provides a summary of the configuration that will be pushed to the ASA. Review and
verify the configuration settings, and then click Finish.
Remote Access VPN on Cisco ASA firewall
Scope: This SOP is used to create Remote Access VPN on cisco ASA using WebUI.
Roles:  User
 Requestor
 IT support team
 ASA Login credentials. ASDM access.
 Phase1, phase2 parameters, IP pool.
Support team.

Outputs:  Remote Access VPN configured on Cisco ASA firewall.
Check Cisco firewall ASA version
Make sure you have ASA 8.2.2 and up.
Start Cisco firewall IPsec VPN Wizard
Login to your Cisco firewall ASA5500 ASDM and go to Wizard > IPsec VPN Wizard ... and follow up the
screens.
In "VPN Tunnel Type", choose "Remote Access"
From the drop-down list, choose "Outside" as the enabled interface for the incoming VPN tunnels. Keep
the box checked,"Enable inbound IPSec sessions to bypass interface access lists. Group policy and per-
user authorization access lists still apply to the traffic."
In Remote Access Client, Check "Microsoft Windows client using L2TP over IPSec"
Check "MS-CHAP-V1" and "MS-CHAP-V2" as PPP authentication protocol.

Choose "Pre-shared Key" for VPN Client Authentication Method
Pre-shared key must be the same for the firewall and client side.
Authenticate remote users using local device user database
Add new user into the user authentication database
You will use this username and password to connect in the client side.
Add address pool
Create a pool of local addresses to be used for assigning dynamic IP addresses to remote VPN clients.
You can use 10.10.20.240 to 10.10.20.249 (may depends on your internal network).
Leave empty for attributes pushed to the client
Default for IKE Policy
3DES encryption & SHA authentication and Diffie Hellman Group 2.

Default for IPSec Settings
Uncheck "Enable split channeling ..." and uncheck "Perfect Forwarding Secrecy(PFS)"
Verify the summary information and click "Finish" button
3. Add Transform Set
Go to Configuration > Remote Access VPN > Network (Client) Access > Advanced > IPSec > Crypto
Maps. Edit the IPSec rules and add "TRANS_ESP_3DES_SHA" and click "Ok" button.
Save the running configuration to flash and all done.
Troubleshoot Site to Site VPN
Scope:  This SOP is used to trouble shoot site to site VPN config on ASA
Roles:  User
 Requestor
 IT support team
 ASA Login credentials.
 CLI access of ASA firewall.
Support team.

Outputs:  Site to Site VPN issue resolved.
Use the information that is provided in this section in order to troubleshoot configuration issues.
ASA Versions 8.4 and Later
Enter these debug commands in order to determine the location of the tunnel failure:
 debug crypto ikev1 127 (Phase 1)
 debug crypto ipsec 127 (Phase 2)

Here is a complete example debug output:
IPSEC(crypto_map_check)-3: Looking for crypto map matching 5-tuple: Prot=1,
saddr=10.2.2.1, sport=19038, daddr=10.1.1.1, dport=19038
IPSEC(crypto_map_check)-3: Checking crypto map outside_map 20: matched.
Feb 13 23:48:56 [IKEv1 DEBUG]Pitcher: received a key acquire message, spi 0x0
IPSEC(crypto_map_check)-3: Looking for crypto map matching 5-tuple: Prot=1,
saddr=10.2.2.1, sport=19038, daddr=10.1.1.1, dport=19038
IPSEC(crypto_map_check)-3: Checking crypto map outside_map 20: matched.
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, IKE Initiator: New Phase 1, Intf NP
Identity Ifc, IKE Peer 192.168.1.1 local Proxy Address 10.2.2.0, remote Proxy
Address 10.1.1.0, Crypto map (outside_map) Feb 13 23:48:56 [IKEv1 DEBUG]IP =
192.168.1.1, constructing ISAKMP SA payload Feb 13 23:48:56 [IKEv1 DEBUG]IP =
192.168.1.1, constructing NAT-Traversal VID ver 02 payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing NAT-Traversal VID
ver 03 payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing NAT-Traversal VID
ver RFC payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing Fragmentation VID +
extended capabilities payload
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, IKE_DECODE SENDING Message (msgid=0)
with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR
(13) + NONE (0) total length : 172
Feb 13 23:48:56 [IKEv1]IKE Receiver: Packet received on 172.16.1.1:500
from 192.168.1.1:500
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, IKE_DECODE RECEIVED Message (msgid=0)
with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total
length : 132
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing SA payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Oakley proposal is acceptable
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing VID payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Received NAT-Traversal ver 02 VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Received Fragmentation VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, IKE Peer included IKE
fragmentation capability flags: Main Mode: True Aggressive Mode: True
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing ke payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing nonce payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing Cisco Unity
VID payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing xauth V6
VID payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Send IOS VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Constructing ASA spoofing IOS
Vendor ID payload (version: 1.0.0, capabilities: 20000001)
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing VID payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Send Altiga/Cisco VPN3000/Cisco
ASA GW VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing NAT-Discovery payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, computing NAT Discovery hash
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, constructing NAT-Discovery payload
with payloads : HDR + KE (4) + NONCE (10) + VENDOR (13) + VENDOR (13) + VENDOR
(13) + VENDOR (13) + NAT-D (130) + NAT-D (130) + NONE (0) total length : 304
from 192.168.1.1:500
with payloads : HDR + KE (4) + NONCE (10) + VENDOR (13) + VENDOR (13) + VENDOR
(13) + VENDOR (13) + NAT-D (130) + NAT-D (130) + NONE (0) total length : 304
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing ke payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing ISA_KE payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing nonce payload
Feb 13 23:48:56 [IKEv1 DEBUG]?IP = 192.168.1.1, processing VID payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Received Cisco Unity client VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Received xauth V6 VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Processing VPN3000/ASA spoofing
IOS Vendor ID payload (version: 1.0.0, capabilities: 20000001)
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Received Altiga/Cisco
VPN3000/Cisco ASA GW VID
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing NAT-Discovery payload
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, processing NAT-Discovery payload
!
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, Connection landed on tunnel_group
192.168.1.1
Feb 13 23:48:56 [IKEv1 DEBUG]!Group = 192.168.1.1, IP = 192.168.1.1, Generating
keys for Initiator...
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, constructing
ID payload
Feb 13 23:48:56 [IKEv1 DEBUG]!Group = 192.168.1.1, IP = 192.168.1.1, constructing
hash payload
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Computing
hash for ISAKMP
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Constructing IOS keep alive
payload: proposal=32767/32767 sec.
!
Success rate is 80 percent (4/5), round-trip min/avg/max = 1/3/10 ms
ciscoasa# Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1,
constructing dpd vid payload
with payloads : HDR + ID (5) + HASH (8) + IOS KEEPALIVE (128) + VENDOR (13) +
NONE (0) total length : 96
Feb 13 23:48:56 [IKEv1]Group = 192.168.1.1, IP = 192.168.1.1, Automatic NAT
Detection Status: Remote end is NOT behind a NAT device This end is NOT behind
a NAT device
from 192.168.1.1:500
with payloads : HDR + ID (5) + HASH (8) + IOS KEEPALIVE (128) + VENDOR (13) +
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, processing
ID payload
Feb 13 23:48:56 [IKEv1 DECODE]Group = 192.168.1.1, IP = 192.168.1.1,
ID_IPV4_ADDR ID received 192.168.1.1
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1,
processing hash payload
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Computing
hash for ISAKMP
Feb 13 23:48:56 [IKEv1 DEBUG]IP = 192.168.1.1, Processing IOS keep alive payload:
proposal=32767/32767 sec.
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, processing
VID payload
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Received
DPD VID
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, Connection landed on tunnel_group
192.168.1.1
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Oakley
begin quick mode
Feb 13 23:48:56 [IKEv1 DECODE]Group = 192.168.1.1, IP = 192.168.1.1, IKE
Initiator starting QM: msg id = 4c073b21
Feb 13 23:48:56 [IKEv1]Group = 192.168.1.1, IP = 192.168.1.1, PHASE 1 COMPLETED
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, Keep-alive type for this connection: DPD
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Starting P1
rekey timer: 73440 seconds.
IPSEC: New embryonic SA created @ 0x75298588,
SCB: 0x75C34F18,
Direction: inbound
SPI : 0x03FC9DB7
Session ID: 0x00004000
VPIF num : 0x00000002
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
IKE got SPI from key engine: SPI = 0x03fc9db7
oakley constucting quick mode
constructing blank hash payload
constructing IPSec SA payload
constructing IPSec nonce payload
constructing proxy ID
Transmitting Proxy Id:
Local subnet: 10.2.2.0 mask 255.255.255.0 Protocol 0 Port 0
Remote subnet: 10.1.1.0 Mask 255.255.255.0 Protocol 0 Port 0
IKE Initiator sending Initial Contact
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1,
IP = 192.168.1.1, constructing qm hash payload
Feb 13 23:48:56 [IKEv1 DECODE]Group = 192.168.1.1,
IP = 192.168.1.1, IKE Initiator sending 1st QM pkt: msg id = 4c073b21
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, IKE_DECODE SENDING Message (msgid=4c073b21)
with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + ID (5) + ID (5) +
NOTIFY (11) + NONE (0) total length : 200
from 192.168.1.1:500
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, IKE_DECODE RECEIVED Message (msgid=4c073b21)
with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + ID (5) + ID (5) + NONE (0)
total length : 172
processing SA payload
processing nonce payload
processing ID payload
ID_IPV4_ADDR_SUBNET ID received--10.2.2.0--255.255.255.0
loading all IPSEC SAs
Generating Quick Mode Key!
NP encrypt rule look up for crypto map outside_map 20 matching ACL
100: returned cs_id=6ef246d0; encrypt_rule=752972d0;
tunnelFlow_rule=75ac8020
Generating Quick Mode Key!
IPSEC: New embryonic SA created @ 0x6f0e03f0,
SCB: 0x75B6DD00,
Direction: outbound
SPI : 0x1BA0C55C
VPIF num : 0x00000002
Tunnel type: l2l
Protocol : esp
IPSEC: Completed host OBSA update, SPI 0x1BA0C55C
IPSEC: Creating outbound VPN context, SPI 0x1BA0C55C
Flags: 0x00000005
SA : 0x6f0e03f0
SPI : 0x1BA0C55C
MTU : 1500 bytes
VCID : 0x00000000
Peer : 0x00000000
SCB : 0x0B47D387
Channel: 0x6ef0a5c0
IPSEC: Completed outbound VPN context, SPI 0x1BA0C55C
VPN handle: 0x0000f614
IPSEC: New outbound encrypt rule, SPI 0x1BA0C55C
Src addr: 10.2.2.0
Src mask: 255.255.255.0
Dst addr: 10.1.1.0
Dst mask: 255.255.255.0
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed outbound encrypt rule, SPI 0x1BA0C55C
Rule ID: 0x74e1c558
IPSEC: New outbound permit rule, SPI 0x1BA0C55C
Src addr: 172.16.1.1
Src mask: 255.255.255.255
Dst addr: 192.168.1.1
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x1BA0C55C
Use SPI: true
IPSEC: Completed outbound permit rule, SPI 0x1BA0C55C
Rule ID: 0x6f0dec80
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, NP encrypt rule
look up for crypto map outside_map 20 matching ACL 100: returned cs_id=6ef246d0;
encrypt_rule=752972d0; tunnelFlow_rule=75ac8020
Feb 13 23:48:56 [IKEv1]Group = 192.168.1.1, IP = 192.168.1.1, Security negotiation
complete for LAN-to-LAN Group (192.168.1.1) Initiator, Inbound SPI = 0x03fc9db7,
Outbound SPI = 0x1ba0c55c
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, oakley
constructing final quick mode
Feb 13 23:48:56 [IKEv1 DECODE]Group = 192.168.1.1, IP = 192.168.1.1, IKE Initiator
sending 3rd QM pkt: msg id = 4c073b21
Feb 13 23:48:56 [IKEv1]IP = 192.168.1.1, IKE_DECODE SENDING Message (msgid=4c073b21)
with payloads : HDR + HASH (8) + NONE (0) total length : 76
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, IKE got a KEY_ADD
msg for SA: SPI = 0x1ba0c55c
IPSEC: New embryonic SA created @ 0x75298588,
SCB: 0x75C34F18,
Direction: inbound
SPI : 0x03FC9DB7
VPIF num : 0x00000002
Tunnel type: l2l
Protocol : esp
IPSEC: Completed host IBSA update, SPI 0x03FC9DB7
IPSEC: Creating inbound VPN context, SPI 0x03FC9DB7
Flags: 0x00000006
SA : 0x75298588
SPI : 0x03FC9DB7
MTU : 0 bytes
VCID : 0x00000000
Peer : 0x0000F614
SCB : 0x0B4707C7
Channel: 0x6ef0a5c0
IPSEC: Completed inbound VPN context, SPI 0x03FC9DB7
VPN handle: 0x00011f6c
IPSEC: Updating outbound VPN context 0x0000F614, SPI 0x1BA0C55C
Flags: 0x00000005
SA : 0x6f0e03f0
SPI : 0x1BA0C55C
MTU : 1500 bytes
VCID : 0x00000000
Peer : 0x00011F6C
SCB : 0x0B47D387
Channel: 0x6ef0a5c0
IPSEC: Completed outbound VPN context, SPI 0x1BA0C55C
VPN handle: 0x0000f614
IPSEC: Completed outbound inner rule, SPI 0x1BA0C55C
Rule ID: 0x74e1c558
IPSEC: Completed outbound outer SPD rule, SPI 0x1BA0C55C
Rule ID: 0x6f0dec80
IPSEC: New inbound tunnel flow rule, SPI 0x03FC9DB7
Src addr: 10.1.1.0
Src mask: 255.255.255.0
Dst addr: 10.2.2.0
Dst mask: 255.255.255.0
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed inbound tunnel flow rule, SPI 0x03FC9DB7
Rule ID: 0x74e1b4a0
IPSEC: New inbound decrypt rule, SPI 0x03FC9DB7
Src addr: 192.168.1.1
Src mask: 255.255.255.255
Dst addr: 172.16.1.1
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x03FC9DB7
Use SPI: true
IPSEC: Completed inbound decrypt rule, SPI 0x03FC9DB7
Rule ID: 0x6f0de830
IPSEC: New inbound permit rule, SPI 0x03FC9DB7
Src addr: 192.168.1.1
Src mask: 255.255.255.255
Dst addr: 172.16.1.1
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x03FC9DB7
Use SPI: true
IPSEC: Completed inbound permit rule, SPI 0x03FC9DB7
Rule ID: 0x6f0de8d8
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Pitcher:
received KEY_UPDATE, spi 0x3fc9db7
Feb 13 23:48:56 [IKEv1 DEBUG]Group = 192.168.1.1, IP = 192.168.1.1, Starting
P2 rekey timer: 24480 seconds.
Feb 13 23:48:56 [IKEv1]Group = 192.168.1.1, IP = 192.168.1.1, PHASE 2
COMPLETED (msgid=4c073b21)
ASA Versions 8.3 and Earlier
Enter these debug commands in order to determine the location of the tunnel failure:
 debug crypto isakmp 127 (Phase 1)
 debug crypto ipsec 127 (Phase 2)

Here is a complete example debug output:
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, IKE_DECODE RECEIVED Message (msgid=0) with
payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) +
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing SA payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Oakley proposal is acceptable
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing VID payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received NAT-Traversal ver 02 VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received NAT-Traversal ver 03 VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received NAT-Traversal RFC VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received Fragmentation VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, IKE Peer included IKE fragmentation
capability flags: Main Mode: True Aggressive Mode: True
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing IKE SA payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, IKE SA Proposal # 1, Transform # 1
acceptable Matches global IKE entry # 1
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing ISAKMP SA payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing NAT-Traversal VID ver
02 payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing Fragmentation VID +
extended capabilities payload
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, IKE_DECODE SENDING Message (msgid=0) with
payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 132
payloads : HDR + KE (4) + NONCE (10) + VENDOR (13) + VENDOR (13) + VENDOR (13) +
VENDOR (13) + NAT-D (130) + NAT-D (130) + NONE (0) total length : 304
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing ke payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing ISA_KE payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing nonce payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received Cisco Unity client VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received xauth V6 VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Processing VPN3000/ASA spoofing IOS
Vendor ID payload (version: 1.0.0, capabilities: 20000001)
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Received Altiga/Cisco VPN3000/Cisco
ASA GW VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing NAT-Discovery payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, computing NAT Discovery hash
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, processing NAT-Discovery payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing ke payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing nonce payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing Cisco Unity VID payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing xauth V6 VID payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Send IOS VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Constructing ASA spoofing IOS Vendor
ID payload (version: 1.0.0, capabilities: 20000001)
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing VID payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Send Altiga/Cisco VPN3000/Cisco
ASA GW VID
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing NAT-Discovery payload
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, constructing NAT-Discovery payload
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, Connection landed on tunnel_group 172.16.1.1
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Generating keys
for Responder...
payloads : HDR + KE (4) + NONCE (10) + VENDOR (13) + VENDOR (13) + VENDOR (13) +
VENDOR (13) + NAT-D (130) + NAT-D (130) + NONE (0) total length : 304
payloads : HDR + ID (5) + HASH (8) + IOS KEEPALIVE (128) + VENDOR (13) + NONE (0)
total length : 96
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, processing
ID payload
Feb 13 04:19:53 [IKEv1 DECODE]: Group = 172.16.1.1, IP = 172.16.1.1, ID_IPV4_ADDR
ID received 172.16.1.1
hash payload
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Computing
hash for ISAKMP
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Processing IOS keep alive payload:
VID payload
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Received DPD VID
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, Automatic NAT Detection
Status: Remote end is NOT behind a NAT device This end is NOT behind
a NAT device
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, Connection landed on tunnel_group 172.16.1.1
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1,
constructing ID payload
constructing hash payload
Computing hash for ISAKMP
Feb 13 04:19:53 [IKEv1 DEBUG]: IP = 172.16.1.1, Constructing IOS keep alive payload:
constructing dpd vid payload
payloads : HDR + ID (5) + HASH (8) + IOS KEEPALIVE (128) + VENDOR (13) + NONE (0)
total length : 96
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, PHASE 1 COMPLETED
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, Keep-alive type for this connection: DPD
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Starting P1
rekey timer: 82080 seconds.
Feb 13 04:19:53 [IKEv1 DECODE]: IP = 172.16.1.1, IKE Responder starting QM: msg id =
4c073b21
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, IKE_DECODE RECEIVED Message
(msgid=4c073b21) with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + ID (5) +
ID (5) + NOTIFY (11) + NONE (0) total length : 200
processing SA payload
processing nonce payload
Feb 13 04:19:53 [IKEv1 DECODE]: Group = 172.16.1.1, IP = 172.16.1.1,
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, Received remote IP
Proxy Subnet data in ID Payload: Address 10.2.2.0, Mask 255.255.255.0,
Protocol 0, Port 0
Feb 13 04:19:53 [IKEv1 DECODE]: Group = 172.16.1.1, IP = 172.16.1.1,
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, Received local IP
Proxy Subnet data in ID Payload: Address 10.1.1.0, Mask 255.255.255.0,
Protocol 0, Port 0
notify payload
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, QM IsRekeyed old sa
not found by addr
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, Static Crypto Map
check, checking map = outside_map, seq = 20...
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, Static Crypto Map
check, map outside_map, seq = 20 is a successful match
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, IKE Remote Peer
configured for crypto map: outside_map
IPSec SA payload
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, IPSec SA
Proposal # 1, Transform # 1 acceptable Matches global IPSec SA entry # 20
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, IKE: requesting SPI!
IPSEC: New embryonic SA created @ 0xAB5C63A8,
SCB: 0xABD54E98,
Direction: inbound
SPI : 0x1BA0C55C
VPIF num : 0x00000001
Tunnel type: l2l
Protocol : esp
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, IKE got SPI
from key engine: SPI = 0x1ba0c55c
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, oakley
constucting quick mode
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, constructing
blank hash payload
IPSec SA payload
IPSec nonce payload
proxy ID
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Transmitting
Proxy Id:
Remote subnet: 10.2.2.0 Mask 255.255.255.0 Protocol 0 Port 0
Local subnet: 10.1.1.0 mask 255.255.255.0 Protocol 0 Port 0
qm hash payload
Feb 13 04:19:53 [IKEv1 DECODE]: Group = 172.16.1.1, IP = 172.16.1.1, IKE Responder
sending 2nd QM pkt: msg id = 4c073b21
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, IKE_DECODE SENDING Message
(msgid=4c073b21) with payloads : HDR + HASH (8) + SA (1) + NONCE (10) + ID (5) +
ID (5) + NONE (0) total length : 172
Feb 13 04:19:53 [IKEv1]: IP = 172.16.1.1, IKE_DECODE RECEIVED Message
(msgid=4c073b21) with payloads : HDR + HASH (8) + NONE (0) total length : 52
hash payload
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, loading all
IPSEC SAs
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Generating
Quick Mode Key!
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, NP encrypt
rule look up for crypto map outside_map 20 matching ACL 100: returned
cs_id=ab9302f0; rule=ab9309b0
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Generating
Quick Mode Key!
IPSEC: New embryonic SA created @ 0xAB570B58,
SCB: 0xABD55378,
Direction: outbound
SPI : 0x03FC9DB7
VPIF num : 0x00000001
Tunnel type: l2l
Protocol : esp
IPSEC: Completed host OBSA update, SPI 0x03FC9DB7
IPSEC: Creating outbound VPN context, SPI 0x03FC9DB7
Flags: 0x00000005
SA : 0xAB570B58
SPI : 0x03FC9DB7
MTU : 1500 bytes
VCID : 0x00000000
Peer : 0x00000000
SCB : 0x01512E71
Channel: 0xA7A98400
IPSEC: Completed outbound VPN context, SPI 0x03FC9DB7
VPN handle: 0x0000F99C
IPSEC: New outbound encrypt rule, SPI 0x03FC9DB7
Src addr: 10.1.1.0
Src mask: 255.255.255.0
Dst addr: 10.2.2.0
Dst mask: 255.255.255.0
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed outbound encrypt rule, SPI 0x03FC9DB7
Rule ID: 0xABD557B0
IPSEC: New outbound permit rule, SPI 0x03FC9DB7
Src addr: 192.168.1.1
Src mask: 255.255.255.255
Dst addr: 172.16.1.1
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x03FC9DB7
Use SPI: true
IPSEC: Completed outbound permit rule, SPI 0x03FC9DB7
Rule ID: 0xABD55848
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, NP encrypt rule
look up for crypto map outside_map 20 matching ACL 100: returned cs_id=ab9302f0;
rule=ab9309b0
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, Security negotiation
complete for LAN-to-LAN Group (172.16.1.1) Responder, Inbound SPI = 0x1ba0c55c,
Outbound SPI = 0x03fc9db7
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, IKE got a
KEY_ADD msg for SA: SPI = 0x03fc9db7
IPSEC: Completed host IBSA update, SPI 0x1BA0C55C
IPSEC: Creating inbound VPN context, SPI 0x1BA0C55C
Flags: 0x00000006
SA : 0xAB5C63A8
SPI : 0x1BA0C55C
MTU : 0 bytes
VCID : 0x00000000
Peer : 0x0000F99C
SCB : 0x0150B419
Channel: 0xA7A98400
IPSEC: Completed inbound VPN context, SPI 0x1BA0C55C
VPN handle: 0x0001169C
IPSEC: Updating outbound VPN context 0x0000F99C, SPI 0x03FC9DB7
Flags: 0x00000005
SA : 0xAB570B58
SPI : 0x03FC9DB7
MTU : 1500 bytes
VCID : 0x00000000
Peer : 0x0001169C
SCB : 0x01512E71
Channel: 0xA7A98400
IPSEC: Completed outbound VPN context, SPI 0x03FC9DB7
VPN handle: 0x0000F99C
IPSEC: Completed outbound inner rule, SPI 0x03FC9DB7
Rule ID: 0xABD557B0
IPSEC: Completed outbound outer SPD rule, SPI 0x03FC9DB7
Rule ID: 0xABD55848
IPSEC: New inbound tunnel flow rule, SPI 0x1BA0C55C
Src addr: 10.2.2.0
Src mask: 255.255.255.0
Dst addr: 10.1.1.0
Dst mask: 255.255.255.0
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed inbound tunnel flow rule, SPI 0x1BA0C55C
Rule ID: 0xAB8D98A8
IPSEC: New inbound decrypt rule, SPI 0x1BA0C55C
Src addr: 172.16.1.1
Src mask: 255.255.255.255
Dst addr: 192.168.1.1
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x1BA0C55C
Use SPI: true
IPSEC: Completed inbound decrypt rule, SPI 0x1BA0C55C
Rule ID: 0xABD55CB0
IPSEC: New inbound permit rule, SPI 0x1BA0C55C
Src addr: 172.16.1.1
Src mask: 255.255.255.255
Dst addr: 192.168.1.1
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0x1BA0C55C
Use SPI: true
IPSEC: Completed inbound permit rule, SPI 0x1BA0C55C
Rule ID: 0xABD55D48
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Pitcher: received
KEY_UPDATE, spi 0x1ba0c55c
Feb 13 04:19:53 [IKEv1 DEBUG]: Group = 172.16.1.1, IP = 172.16.1.1, Starting P2 rekey
timer: 27360 seconds.
Feb 13 04:19:53 [IKEv1]: Group = 172.16.1.1, IP = 172.16.1.1, PHASE 2 COMPLETED
(msgid=4c073b21)
Trouble shoot Remote Access VPN on ASA Firewall
Scope:  This SOP is used to trouble shoot Remote Access VPN config on ASA
Roles:  User
 Requestor
 IT support team
 Remote user details.
Support team.

Outputs:  Remote Access VPN issue resolved.
VPN Client Drops Connection Frequently on First Attempt or "Security VPN Connection
terminated by peer. Reason 433." or "Secure VPN Connection terminated by Peer Reason
433:(Reason Not Specified by Peer)"
Problem
Cisco VPN client users might receive this error when they attempt the connection with the head end VPN device.
"VPN client drops connection frequently on first attempt" or "Security VPN Connection terminated by peer. Reason 433." or
"Secure VPN Connection terminated by Peer Reason 433:(Reason Not Specified by Peer)" or "Attempted to assign network or
broadcast IP address, removing (x.x.x.x) from pool"
Solution 1
The problem might be with the IP pool assignment either through ASA/PIX, Radius server, DHCP server or through Radius server
acting as DHCP server. Use the debug crypto command in order to verify that the netmask and IP addresses are correct. Also,
verify that the pool does not include the network address and the broadcast address. Radius servers must be able to assign the
proper IP addresses to the clients.
Solution 2
This issue also occurs due to the failure of extended authentication. You must check the AAA server to troubleshoot this error.
Checking the server authentication password on Server and client and reloading the AAA server might resolve this issue.
Solution 3
Another workaround for this issue is to disable the threat detection feature. At times when there are multiple re-transmissions
for different incomplete Security Associations (SAs), the ASA with the threat-detection feature enabled thinks that a scanning
attack is occuring and the VPN ports are marked as the main offender. Try to disable the threat-detection feature as this can
cause a lot of overhead on the processing of ASA. Use these commands in order to disable the threat detection:
no threat-detection basic-threat
no threat-detection scanning-threat shun
no threat-detection statistics
no threat-detection rate
For more information about this feature, refer to Threat Detection.
Note: This can be used as a workaround to verify if this fixes the actual problem. Make sure that disabling the threat detection
on the Cisco ASA actually compromises several security features such as mitigating the Scanning Attempts, DoS with Invalid SPI,
packets that fail Application Inspection and Incomplete Sessions.
Solution 4
This issue also occurs when a transform set is not properly configured. A proper configuration of the transform set resolves the
issue
Unable to Access the Servers

in DMZ
Once the VPN client is established the IPsec tunnel with the VPN head-end device (PIX/ASA/IOS Router), the VPN client users
are able to access the INSIDE network (10.10.10.0/24) resources, but they are unable to access the DMZ network (10.1.1.0/24).
Diagram
Check that the Split Tunnel, NO NAT configuration is added in the head-end device to access the resources in the DMZ network.
Unable to Connect More Than Three VPN Client Users
Problem
Only three VPN clients can connect to ASA/PIX; connection for the fourth client fails. Upon failure, this error message is
displayed:
Secure VPN Connection terminated locally by the client.
Reason 413: User Authentication failed.
tunnel rejected; the maximum tunnel count has been reached
Solutions
In most cases, this issue is related to a simultaneous login setting within group policy and the maximum session-limit.
Try these solutions in order to resolve this issue:
 Configure Simultaneous Logins

 Configure the ASA/PIX with CLI
 Configure Concentrator Configure Concentrator
For more information, refer to the Configuring Group Policies section of Selected ASDM VPN Configuration Procedures for the
Cisco ASA 5500 Series, Version 5.2.
Configure Simultaneous
Logins
If the Inherit check box in ASDM is checked, only the default number of simultaneous logins is allowed for the user. The default
value for simultaneous logins is three.
In order to resolve this issue, increase the value for simultaneous logins.
1. Launch ASDM and then navigate to Configuration > VPN > Group Policy.
2. Choose the appropriate Group and click the Edit button.
3. Once in the General tab, undo the Inherit check box for Simultaneous Logins under Connection Settings. Choose an
appropriate value in the field.
Note: The minimum value for this field is 0, which disables login and prevents user access.
Note: When you log in using the same user account from a different PC, the current session (the connection
established from another PC using the same user account) is terminated, and the new session is established. This is
the default behaviour and is independent to VPN simultaneous logins.
Configure the ASA/PIX with

CLI
Complete these steps in order to configure the desired number of simultaneous logins. In this example, 20 was chosen as the
desired value.
ciscoasa(config)#group-policy Bryan attributes
ciscoasa(config-group-policy)#vpn-simultaneous-logins 20
In order to learn more about this command, refer to Cisco Security Appliance Command Reference, Version 7.2.
Use the vpn-sessiondb max-session-limit command in global configuration mode in order to limit VPN sessions to a lower value
than the security appliance allows. Use the no version of this command in order to remove the session limit. Use the command
again in order to overwrite the current setting.
vpn-sessiondb max-session-limit {session-limit}
This example shows how to set a maximum VPN session limit of 450:
hostname#vpn-sessiondb max-session-limit 450
Configure Concentrator
Error Message
20932 10/26/2007 14:37:45.430 SEV=3 AUTH/5 RPT=1863 10.19.187.229
Authentication rejected: Reason = Simultaneous logins exceeded for user
handle = 623, server = (none), user = 10.19.187.229, domain = <not
specified>
Solution
Complete these steps in order to configure the desired number of simultaneous logins. You can also try to set the Simultaneous
Logins to 5 for this SA:
Choose Configuration > User Management > Groups > Modify 10.19.187.229 > General > Simultaneous Logins, and change the
number of logins to 5.
Troubleshoot Performance issue of ASA firewall
Scope:  This SOP is used to trouble shoot performance issue of Cisco ASA firewall.
Roles:  User
 Requestor
 IT support team
Support team.

Outputs:  ASA performance issue resolved.
CPU Utlilization
If you noticed the CPU utlization is high, complete these steps in order to troubleshoot:
1. Verify that the connection count in show xlate count is low.
2. Verify that the memory block is normal.
3. Verify that the number of ACLs is higher.
4. Issue the show memory detail command, and verify that the memory used by the ASA is normal utilization.
5. Verify that the counts in show processes cpu-hog and show processes memory are normal.
6. Any host present inside or outside the security appliance can generate the malicious or mass traffic that can be a
broadcast/multicast traffic and cause the high CPU utilization. In order to resolve this issue, configure an access
list to deny the traffic between the hosts (end to end) and check the usage.
7. Check the duplex and speed settings in ASA interfaces. The mismatch setting with the remote infterfaces can
increase the CPU utilization.
This example shows the higher number in input error and overruns due to the speed mismatch. Use the show
interface command in order to verify the errors:
8. Ciscoasa#sh int GigabitEthernet0/1

9. Interface GigabitEthernet0/1 "inside", is up, line protocol is up
10. Hardware is i82546GB rev03, BW 1000 Mbps, DLY 10 usec
11. Auto-Duplex(Full-duplex), Auto-Speed(100 Mbps)
12. Input flow control is unsupported, output flow control is unsupported
13. MAC address 0013.c480.b2b8, MTU 1500
14. IP address 192.168.17.4, subnet mask 255.255.255.0
15. 311981 packets input, 20497296 bytes, 0 no buffer
16. Received 311981 broadcasts, 157 runts, 0 giants
17. 7186 input errors, 0 CRC, 0 frame, 7186 overrun, 0 ignored, 0 abort
18. 0 pause input, 0 resume input
19. 0 L2 decode drops
20. 121 packets output, 7744 bytes, 0 underruns
21. 0 pause output, 0 resume output
22. 0 output errors, 0 collisions, 1 interface resets
23. 0 late collisions, 0 deferred
24. 0 input reset drops, 0 output reset drops, 0 tx hangs
25. input queue (blocks free curr/low): hardware (255/249)
output queue (blocks free curr/low): hardware (255/254)
In order to resolve this issue, set speed as auto to the corresponding interface.
Note: Cisco recommends that you enable the ip verify reverse-path interface command on all the interfaces as
it will drop packets that do not have a valid source address, which results in less CPU usage. This applies to
FWSM facing high CPU issues.
26. Another reason for high CPU usage can be due to too many multicast routes. Issue the show mroute command in
order to check if ASA receives too many multicast routes.
27. Use the show local-host command in order to see if the network experiences a denial-of-service attack, which
can indicate a virus attack in the network.
28. High CPU might occur due to Cisco bug ID CSCsq48636 . Refer to Cisco bug
ID CSCsq48636 (registeredcustomers only) for more information.
Note: If the solution provided above does not resolve the issue, upgrade the ASA platform according to
the requirements. Refer to Cisco ASA 5500 Series Adaptive Security Appliances Data Sheet for more
information on Adaptive Security Appliance Platform capabilities and capacities. Contact
TAC (registered customers only) for further information.
High Memory Utilization
Here are some possible causes and resolutions for high memory utilization:
 Event logging: Event logging can consume large amounts of memory. In order to resolve this issue, install and
log all events to an external server, such as a syslog server.
 Memory Leakage: A known issue in the security appliance software can lead to high memory consumption. In
order to resolve this issue, upgrade the security appliance software.
 Debugging Enabled: Debugging can consume large amounts of memory. In order to resolve this issue, disable
debugging with the undebug all command.
 Blocking Ports: Blocking ports on the outside interface of a security appliance cause the security appliance to
consume high amounts of memory to block the packets through the specified ports.In order to resolve this issue,
block the offending traffic at the ISP end.
 Threat-Detection: The threat detection feature consists of different levels of statistics gathering for various
threats, as well as scanning threat detection, which determines when a host is performing a scan. Turn off this
feature to consume less memory.
Viewing CPU Usage on ASDM
Complete these steps in order to view the CPU usage on the ASDM:
1. Go to Monitoring > Properties > System Resources Graphics > CPU in ASDM and choose the Graph
Window Title. Then, choose the required graphs from the list of Available Graphs and click Add as shown.
2. Once the required graph name is added under the Selected Graphs section, click Show Graphs.
The next image shows the CPU Usage graph on the ASDM. Different views of this graph are available and can
be changed by selecting the view from the View drop-down list. This output can be printed or saved to the
computer as required.
Description of Output
This table describes the fields in the show cpu usage output.
Field Description
CPU utilization for 5 seconds CPU utilization for the last five seconds
Average of 5 second samples of CPU utilization over the

1 minute last minute
Average of 5 second samples of CPU utilization over the

5 minutes last five minutes
show traffic
The show traffic command shows how much traffic that passes through the ASA over a given period of
time. The results are based on the time interval since the command was last issued. For accurate results,
issue the clear traffic command first and then wait 1-10 minutes before you issue the show
traffic command. You could also issue the show traffic command and wait 1-10 minutes before you issue
the command again, but only the output from the second instance is valid.
You can use the show traffic command in order to determine how much traffic passes through your ASA.
If you have multiple interfaces, the command can help you determine which interfaces send and receive
the most data. For ASA appliances with two interfaces, the sum of the inbound and outbound traffic on
the outside interface should equal the sum of the inbound and outbound traffic on the inside interface.
Example
Ciscoasa#show traffic
outside:
received (in 124.650 secs):
295468 packets 167218253 bytes
2370 pkts/sec 1341502 bytes/sec
transmitted (in 124.650 secs):
inside:
received (in 124.650 secs):
transmitted (in 124.650 secs):
If you come close to or reach the rated throughput on one of your interfaces, you need to upgrade to a
faster interface or limit the amount of traffic that goes into or out of that interface. Failure to do so can
result in dropped packets. As explained in the show interface section, you can examine the interface
counters in order to find out about throughput.
show perfmon
The show perfmon command is used to monitor the amount and types of traffic that the ASA inspects.
This command is the only way to determine the number of translations (xlates) and connections (conn)
per second. Connections are further broken down into TCP and User Datagram Protocol (UDP)
connections. See Description of Output for descriptions of the output that this command generates.
Example
PERFMON STATS Current Average
Xlates 18/s 19/s
Connections 75/s 79/s
TCP Conns 44/s 49/s
UDP Conns 31/s 30/s
URL Access 27/s 30/s
URL Server Req 0/s 0/s
TCP Fixup 1323/s 1413/s
TCPIntercept 0/s 0/s
HTTP Fixup 923/s 935/s
FTP Fixup 4/s 2/s
AAA Authen 0/s 0/s
AAA Author 0/s 0/s
AAA Account 0/s 0/s
Description of Output
This table describes the fields in the show perfmon output.
Field Description
Xlates Translations built up per second
Connections Connections established per second
TCP Conns TCP connections per second

UDP Conns UDP connections per second
URL Access URLs (websites) accessed per second
Requests sent to Websense and N2H2 per second

URL Server Req (requires filter command)
TCP Fixup Number of TCP packets that the ASA forwards per second
Number of SYN packets per second that have exceeded the embryonic
TCPIntercept limit set on a static
Number of packets destined to port 80 per second (requires fixup

HTTP Fixup protocol http command)
FTP Fixup FTP commands inspected per second
AAA Authen Authentication requests per second
AAA Author Authorization requests per second
AAA Account Accounting requests per second
show blocks
Along with the show cpu usage command, you can use the show blocks command in order to determine
whether the ASA is overloaded.
Packet-Processing Blocks (1550 and 16384 Bytes)
When it comes into the ASA interface, a packet is placed on the input interface queue, passed up to the
OS, and placed in a block. For Ethernet packets, the 1550-byte blocks are used; if the packet comes in
on a 66 MHz Gigabit Ethernet card, the 16384-byte blocks are used. The ASA determines whether the
packet is permitted or denied based on the Adaptive Security Algorithm (ASA) and processes the packet
through to the output queue on the outbound interface. If the ASA cannot support the traffic load, the
number of available 1550-byte blocks (or 16384-byte blocks for 66 MHz GE) hovers close to 0 (as shown
in the CNT column of the command output). When the CNT column hits zero, the ASA attempts to
allocate more blocks, up to a maximum of 8192. If no more blocks are available, the ASA drops the
packet.
show memory
The show memory command displays the total physical memory (or RAM) for the ASA, along with the
number of bytes currently available. In order to use this information, you must first understand how the
ASA uses memory. When the ASA boots, it copies the OS from Flash into RAM and runs the OS from
RAM (just like routers). Next, the ASA copies the startup configuration from Flash and places it into RAM.
Finally, the ASA allocates RAM in order to create the block pools discussed in the show blocks section.
Once this allocation is complete, the ASA needs additional RAM only if the configuration increases in
size. In addition, the ASA stores the translation and connection entries in RAM.
During normal operation, the free memory on the ASA should change very little, if at all. Typically, the
only time you should run low on memory is if you are under attack and hundreds of thousands of
connections go through the ASA. In order to check the connections, issue the show conn count command,
which displays the current and maximum number of connections through the ASA. If the ASA runs out of
memory, it eventually crashes. Prior to the crash, you might notice memory allocation failure messages in
the syslog (%ASA-3-211001). If you run out of memory because you are under attack, contact the Cisco
Technical Assistance Center (TAC).
Example
Ciscoasa#
show memory
Free memory: 845044716 bytes (79%)
Used memory: 228697108 bytes (21%)
------------- ----------------
Total memory: 1073741824 bytes (100%)
show conn count
The show conn count command shows the current and maximum number of connections through the ASA.
A connection is a mapping of Layer 4 information from an internal address to an external address.
Connections are built up when the ASA receives a SYN packet for TCP sessions or when the first packet
in a UDP session arrives. Connections are torn down when the ASA receives the final ACK packet, which
occurs when the TCP session handshake closes or when the timeout expires in the UDP session.
Extremely high connection counts (50-100 times normal) might indicate that you are under attack. Issue
the show memory command in order to ensure that the high connection count does not cause the ASA to
run out of memory. If you are under attack, you can limit the maximum number of connections per static
entry and also limit the maximum number of embryonic connections. This action protects your internal
servers, so they do not become overwhelmed. Refer to Cisco ASA 5500 Series Adaptive Security Appliances
Command References for more information.
Example
Ciscoasa#show conn count

2289 in use, 44729 most used
show processes
The show processes command on the ASA displays all the active processes that run on the ASA at the
time the command is executed. This information is useful in order to determine which processes receive
too much CPU time and which processes do not receive any CPU time. In order to get this information,
issue the show processescommand twice; wait about 1 minute between each instance. For the process in
question, subtract the Runtime value displayed in the second output from the Runtime value displayed in
the first output. This result shows you how much CPU time (in milliseconds) the process received in that
interval of time. Note that some processes are scheduled to run at particular intervals, and some
processes only run when they have information to process. The 577poll process most likely has the
largest Runtime value of all your processes. This is normal because the 577poll process polls the
Ethernet interfaces in order to see if they have any data that needs to be processed.
Note: An examination of each ASA process is out of the scope of this document, but is mentioned briefly
for completeness. Refer to The ASA show processes Command for more information about the ASA
processes.
Command Summary
In summary, use the show cpu usage command in order to identify the load that the ASA is under.
Remember that the output is a running average; the ASA can have higher spikes of CPU usage that are
masked by the running average. Once the ASA reaches 80% CPU usage, the latency through the ASA
slowly increases to about 90% CPU. When CPU usage is more than 90%, the ASA starts to drop
packets.
If the CPU usage is high, use the show processes command in order to identify the processes that use the
most CPU time. Use this information in order to reduce some of the time that is consumed by the
intensive processes (such as logging).
If the CPU does not run hot, but you believe packets are still dropped, use the show interface command in
order to check the ASA interface for no buffers and collisions, possibly caused by a duplex mismatch. If
the no buffer count increments, but the CPU usage is not low, the interface cannot support the traffic that
flows through it.
If the buffers are fine, check the blocks. If the current CNT column in the show blocks output is close to 0
on the 1550-byte blocks (16384-byte blocks for 66 MHz Gig cards), the ASA most likely drops Ethernet
packets because it is too busy. In this instance, the CPU spikes high.
If you experience trouble when you make new connections through the ASA, use the show conn
count command in order to check the current count of connections through the ASA.
If the current count is high, check the show memory output in order to ensure that the ASA does not run
out of memory. If memory is low, investigate the source of the connections with the show conn or show
local-hostcommand in order to verify that your network has not experienced a denial-of-service attack.
You can use other commands in order to measure the amount of traffic that passes through the ASA.
The show traffic command displays the aggregate packets and bytes per interface, and the show
perfmon breaks the traffic down into different types that the ASA inspects.
F5 Load Balancer
To create a virtual server
Scope:  This SOP is used to create Virtual Server on F5 Load Balancer using WebUI.
 This SOP is not used to create Virtual Server on F5 Load Balancer using CLI
mode.
Roles:  User
 Requestor
 IT support team
 F5 Load Balancer Login credentials.
 Virtual server IP address & FQDN details, Virtual service pool, Pool
members details
Support team.

Outputs:  Virtual server (VIP) has been created on F5 Load balancer.
1) Login to F5 Big IP load balancer with credential.
a) On the Main tab of the navigation pane, expand Local Traffic, and click Virtual
Servers. The Virtual Servers screen opens.
. b) On the upper right portion of the screen, click the Create button.
The New Virtual Server screen opens Configure the below settings
Name: - Name of the Virtual server
Destination: - The destination Host or Network
Service Port: - Select the service port as per requirement
Protocol: - As per Requirement select one of the Protocol and available form Drop down menu
In Basic configuration under Resource section add the Service pool

and Click Finished.
To create server load balancing pool
Scope:  Creating Server pool on F5 Load Balancer.
Roles:  User
 Requestor
 IT support team
 Virtual service pool, Pool member’s details.
Support team.

Outputs:  Created Server Pool on F5 Load balancer.
a) On the Main tab of the navigation pane, expand Local Traffic, and click Pools.
The Pools screen opens.
b ) Under Basic configuration of service pool define the parameter as below

Name:- Pool Name
Load balancing method :- Select the required Load balancing method from drop down menu
c) New member: - In this add server members, specify the Node Name, pool address and
service port no.
d) Repeat the step c to add more members in pool.
e) Click Finished.
Modifying (Add, Remove, )existing Server pool members
a) On the Main tab of the navigation pane, expand Local Traffic, and click Pools.
The Pools screen opens.
b) In the Members column, click the number shown. This lists the existing members of the
pool.
c) Click on Add/Remove/Enable and disable as per action required.
d) Click on update
To create an iRule
Scope:  Creating I Rule on F5 to modify the traffic behavior of Load Balancer.
Roles:  User
 Requestor
 IT support team
 Name, condition, Event & Action details to create I Rule
Support team.

Outputs:  Created I Rule on F5 Load balancer.
a) On the Main tab of the navigation pane, expand Local Traffic, and click iRules.
The iRules screen opens.
b) In the upper right corner, click Create.
c) In the Name box Type the I rule Name and In the Definition box type the syntax
for your iRule.
d) Click Finished.
To Install SSL Certificates
Scope:  Installing SSL certificate on F5 Load Balancer for secure communication.
Roles:  User
 Requestor
 IT support team
 Certificate created.
Support team.

Outputs:  SSL certificate installed on F5 Load balancer.
a) Download the SSL Certificate & Intermediate CA Certificate

b) Install the SSL Certificate
Log in to the Configuration utility.
Navigate to System > File Management > SSL Certificates List and click Import.
From the Import Type list, select Certificate.
In the Certificate Name section, click Create New.
In the Certificate Name box, type a name for the SSL certificate.
In the Certificate Source section, click either Upload File or Paste Text the SSL
certificate (i.e. ssl_certificate.crt, as described in Step 1).
Click Import.
: Configure an SSL Client Profile to use the Intermediate CA Certificate
1. Log in to the Configuration utility.

2. Click Local Traffic.
3. Click Profiles.
4. Click SSL.
5. Click Client or Server.
6. Click the name of the Client SSL or Server SSL profile.

7. From the Configuration box, select Advanced.
8. If it is not already selected, select the Chain check box.
9. From the Chain box, select the appropriate chain certificate.
10. To save the modifications, Click Finished.
Troubleshooting F5 Load Balancer
Scope:  Trouble shooting techniques for F5 Load Balancer.
Roles:  User
 Requestor
 IT support team
Support team.

Outputs:  Resolution provided to user & issue resolved.
Troubleshooting health monitors
The following table lists Configuration utility pages where you can check the status of pools,
pool members, and nodes:
Configuration utility Description Location

page
Network map Summary of pools, pool members, Local Traffic > Network Map >
and nodes Show Map
Pools Current status of pool/members Local Traffic > Pools > Statistics
Pool members Current status of pool/members Local Traffic > Pools > Statistics
Nodes Current status of nodes Local Traffic > Nodes > Statistics
You can use below commands for trouble shooting various F5 Load Balancer issues.
Show sys version view system version and hotfix summary information
show sys – General system configuration
show sys license - View license information
show net arp – shows the F5s arp table
show sys cpu - CPU statistics of system overall performance and on management hosts.
show net route – routing tables and configuration

NSS Final SOP

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

NSS Final SOP

Загружено:

Авторское право:

Доступные форматы

 Configure Layer 2 switch –

Scope: This SOP is used to Configure Cisco Layer 2 Switch.

Controls:  Tickets SLAs

1. Once we received new switch we need to login it by default credentials – (cisco/cisco).

4. Configure the Interface according to Requirements.

Scope: Configuration of VSP static routing on Core switch.

Controls:  Tickets SLAs

1. Login to core switch with the help of AD credentials.

Troubleshoot VLAN issue on switch –

Core Switch – (Server Mode)

Access Switch – (Client Mode)

6. To configure VLAN on switch follow below steps.

7. To check vlans on switch.

9. To check that interface on switch.

 IOS version upgrade on Router Or Switch –

Controls:  Tickets SLAs

1. To Download IOS Image.

2. Saving Backup Copies of Your Old System Image and Configuration.

Router# copy nvram:startup-config ftp:

Router# copy flash0: ftp:

Router# copy nvram:startup-config tftp:

Remote host[]? 192.0.0.1

Router# copy flash0: tftp:

3. Copying the System Image into Flash Memory

Router# copy tftp flash0:

When prompted, enter the IP address of the TFTP

Address or name of remote host []? 10.10.10.2

Source filename []? c2900-universalk9-mz.bin

Destination filename []? c2900-universalk9-mz.bin

Router# copy run start

 Printer/Scanners not accessible on network

Scope: This SOP is used of Printer/Scanner issue trouble shooting.

Controls:  Tickets SLAs

1. Login to the switch with AD credentials.

 Troubleshoot Meraki wireless issue

Scope: This SOP is used of Meraki Wireless issue trouble shooting.

Controls:  Tickets SLAs

How to Login to Meraki Dashboard

Meraki is Switch from Gateway to Repeater Mode.

Follow the step above problems.

Users complain slowness for one of SSID.

Login to meraki Dashboard, check SSID presents on that AP or not.

Check SSID is OK and present on the AP.

If any of SSID is not ON then check all configuration on all AP.

If we want to check Access Points Details and Health.

1. Login to the Meraki Dashborad .

Go to Network-wide Tab and Select Clients.

 Troubleshooting DHCP & DNS issues on Switch.

Controls:  Tickets SLAs

1. Login to the core Switch.

Controls:  Tickets SLAs

 This screenshot shows the traffic generated TO a particular local host.

 The below screenshot says host edmokeatingm2.na.corp.local (207.61.65.67) is receiving the

 DHCP configuration on Switch –

Controls:  Tickets SLAs

Scope:  This SOP is used for up gradation of MPLS link on CE Router.

Controls:  Tickets SLAs

After complete the Phase 1 we will run the BGP session

IP route 0.0.0.0 0.0.0.0 10.254.160.26

Remove the existing BGP process using command

col-att-torrance-ce01(config)# router bgp 65246

col-att-torrance-ce01(config)# router eigrp 10

Check the routing table in the CE router using command

 Create Network diagram in VISIO and update time to time