Method of Procedure

Cut-Off Link

Initial Drafted by: Mahardika Nila Date: 12 November 2019

Reviewed by: Nugraha Achmad Date: 12 November 2019

Reviewed by: Date:

Approved by: Date:

 MoP Cut Off Link
_________________________________________________________________________________________________
Revision Record

Date Revised Version Change description Author

2
 MoP Cut Off Link
_________________________________________________________________________________________________

Table of Contents

1 EXECUTIVE SUMMARY............................................................................................................4
1.1 Background.....................................................................................................................4
1.2 Work Information............................................................................................................4
1.2.1 Site Address............................................................................................................4
1.2.1 Work Schedule........................................................................................................4
1.3 Outage Information.........................................................................................................4
1.4 Prerequisites...................................................................................................................4
2 BACKGROUND AND OBJECTIVES.............................................................................................5
3 SCOPE OF WORK.....................................................................................................................6
4 DETAIL TIME TABLE OF OVERALL ACTIVITIES...........................................................................7
5 DETAIL OPERATION AND EXECUTION......................................................................................8
5.1 Procedure Pre-Activity at All Router (23:00 – 00:00)......................................................8
(Call i-NOC Fault Handling IP (0816100320 / 021-30441862) & i-SOC Operation (0816-
100330) or NOC Coordinator: 0816801603)...........................................................................8
5.2 Procedure During Activity (00:00 - 01:41).....................................................................11
5.2.1 Procedure Create Rollback File..............................................................................11
5.2.2 Procedure Cut Off Existing Link SBY-WDRI-EN1-SASSx-OPT >< SBY-SSU-EN1-SASR6
12
5.3 Procedure After Activity (01:41 - 02:36)........................................................................17
(Call i-NOC Fault Handling IP (0816100320 / 021-30441862) & i-SOC Operation (0816-
100330) or NOC Coordinator: 0816801603).........................................................................17
6 SERVICE AND AREA IMPACT..................................................................................................19
7 TESTING AND VERIFICATION.................................................................................................20
8 ROLLBACK OPERATION (02:36 – End)....................................................................................21
9 PERSON IN CHARGE..............................................................................................................22
10 SUPPORTING DOCUMENTS...............................................................................................23

3
 MoP Cut Off Link
_________________________________________________________________________________________________

1 EXECUTIVE SUMMARY

1.1 Background
This document is the procedure for cut off link on Nokia router.

1.2 Work Information

1.2.1 Site Address
Site Name : SBY-WDRI-EN1-SASSX-OPT (114.0.173.246/32)
SBY-SSU-EN1-SASR6 (114.0.146.6/32)

Address : TAMBAK SAWAH 03/II_ TAMBAK SAWAH_ WARU_

SIDOARJO

PIC During NIW & No HP PIC Monitoring Pasca NIW & No HP

1.2.1 Work Schedule

Date : TBI
Working Time : 23:00 – 05.00
Work Description : Cut Off Link

1.3 Outage Information

This kind of work will cause an outage/interrupt on existing equipment for 1
hour.

1.4 Prerequisites
Before the cut off link process, please make sure that requirement can be fulfill.
Below are the list of prerequisites:
- Laptop and Console for cut off process.

4
 MoP Cut Off Link
_________________________________________________________________________________________________

2 BACKGROUND AND OBJECTIVES

This document is the procedure for cut off link on Nokia router.

<show router interface exclude-services>

5
 MoP Cut Off Link
_________________________________________________________________________________________________

3 SCOPE OF WORK

For this activity, our scope of works are:

 Cut Off Link

 Monitoring during and after activities

6
 MoP Cut Off Link
_________________________________________________________________________________________________

4 DETAIL TIME TABLE OF OVERALL ACTIVITIES

Time Operation Detail Service Impact

Power, Module and Service Verification on Metro-E
23:00 – 23:30
as pre-activity by DTC team
23:30 – 00:00 Verify the responsibility of each team (DTC, Indosat,
and another Vendor)
00:00 - 00:35 During activity Re-engineering & Upgrade link Expected downtime 30
minutes
00:35 - 01:30 Power, Module and Service Verification on Metro-E
as post-activity by DTC team

Rollback operation

01:30 - 01:40 Rollback escalation

01:40 - 02:00 Perform Rollback Expected downtime 30

minutes
02:00 - 02:10 Perform testing services (if any)

7
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

5 DETAIL OPERATION AND EXECUTION

5.1 Procedure Pre-Activity at All Router (23:00 – 00:00)

(Call i-NOC Fault Handling IP (0816100320 / 021-30441862) & i-SOC Operation (0816-100330) or NOC Coordinator: 0816801603)

Remarks/Expected
No Steps Config Status Duration
Result
1 Check all module status show time  Check 5 minutes Log using SecureCrt
and existing configuration show uptime
show bof
show system information
show system cpu
show chassis
show chassis detail
show card state
show card detail
show mda detail
show log log-id 99
show log log-id 99 application snmp
show log log-id 99 application chassis
show log log-id 100
show system alarm
show port
show port detail | match expression
"Speed|Change|Rx Optical"
show port | match "Yes"
show router status
show router route-table summary
show router interface
show router ospf <All Instance ID>
interface
show router ospf <instance ID> neighbour
show router bgp summary
show router mpls status

8
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

show router mpls lsp

show router mpls lsp terminate
show router ldp status
show router ldp session
show service service-using
show service service-using | match Down
show service service-using | match Up
show service sdp
show service sdp-using
show service sap-using
show service fdb-mac
show service fdb-mac | match No.
show system sync-if-timing
2 Check Reachability NE show system security authentication  Check 2 minutes TACACS Status : Up
( SAM & Tacacs) ping 10.137.32.209 (exclude .195)
ping 10.137.32.210
ping 10.137.32.230 Ping result OK
ping 10.34.169.101
ping 10.34.169.107
3 Create TS File Before admin tech-support  Check 15 minutes create 2 TS file
Activity cf3:/pre_CutOffLink_1.tec interval 15 Minutes
admin tech-support
cf3:/pre_CutOffLink_2.tec
4 Monitor and Check status monitor port x/y/[a..z] interval 3 repeat  Check 5 minutes
ALL port 1 | match %
show port x/y/[a..z] | match expression
"Oper Speed|Oper Duplex"
5 Backup all existing admin display-config  Check 2 minutes capture to the
configuration by display separate file
the configuration and
capture it.
6 Save configuration admin save  Check 1 minutes
7 Verify the responsibility Check 25 minutes
of each team (DTC,

9
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

Indosat, and another

Vendor)

10
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

5.2 Procedure During Activity (00:00 - 00:30)

5.2.1 Procedure Create Rollback File

Remarks/Expected
No Steps Config Status Duration
Result
1 Verification existing Rollback show system rollback Check 1 minutes

2 Create rollback file location configure system rollback Check 3 minutes Make sure system
rollback-location cf3:\rollback- rollback can be
CutOffLink created

3 Verification Rollback show system rollback Check 2 minutes Make sure rollback
configuration location on
cf3:\rollback-
CutOffLink

4 Create first checkpoint admin rollback save comment Check 3 minutes Make sure rollback
"Before CutOffLink" can be created on
cf3:\rollback-
/show system rollback CutOffLink
file dir | match rollback-
CutOffLink

11
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

5.2.2 Procedure Cut Off Existing Link SBY-WDRI-EN1-SASSx-OPT >< SBY-SSU-EN1-SASR6

Remarks/Expected
No Steps Config Status Duration
Result
1 Verification before cut off At SBY-WDRI-EN1-SASSx-OPT Chec 5 minutes
link k
show port 1/1/27

show router interface to_SBY-SSU-EN1-SASR6

show router ospf interface to_SBY-SSU-EN1-
SASR6 detail | match Metric
show router ospf neighbor to_SBY-SSU-EN1-
SASR6
show router ldp interface to_SBY-SSU-EN1-
SASR6
show router mpls interface to_SBY-SSU-EN1-
SASR6
show router rsvp interface to_SBY-SSU-EN1-
SASR6

show router mpls lsp transit | match 1/1/27

show router mpls lsp terminate | match
1/1/27
show router mpls lsp path | match 1/1/27

ping 114.0.204.103 rapid count 1000 do-not-

fragment

monitor port 1/1/27 rate interval 3 repeat 3

| match %

At SBY-SSU-EN1-SASR6

12
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

show port 3/1/1

show router interface to_SBY-WDRI-EN1-SASSx-

OPT
show router ospf interface to_SBY-WDRI-EN1-
SASSx-OPT detail | match Metric
show router ospf neighbor to_SBY-WDRI-EN1-
SASSx-OPT
show router ldp interface to_SBY-WDRI-EN1-
SASSx-OPT
show router mpls interface to_SBY-WDRI-EN1-
SASSx-OPT
show router rsvp interface to_SBY-WDRI-EN1-
SASSx-OPT

show router mpls lsp transit | match 3/1/1

show router mpls lsp terminate | match 3/1/1
show router mpls lsp path | match 3/1/1

ping 114.0.204.102 rapid count 1000 do-not-

fragment

monitor port 3/1/1 rate interval 3 repeat 3

| match %

2 Reroute LSP Traffic #Check LSP transit and terminate Chec 5 minutes

show router mpls lsp transit

k
show router mpls lsp terminate
3 Apply configuration at /configure router ospf area 19 interface Chec 5 minutes Do this step on both
SBY-WDRI-EN1-SASSx- to_SBY-SSU-EN1-SASR6 sides at the same
info
k
OPT metric 65000 time
exit all

13
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

/configure router interface to_SBY-SSU-EN1-

SASR6
no port
shutdown
exit all

/configure router no interface to_SBY-SSU-

EN1-SASR6

/configure port 1/1/27 no description

/configure port 1/1/27 shutdown

4 Apply configuration at /configure router ospf area 19 interface Chec 5 minutes Do this step on both
SBY-SSU-EN1-SASR6 to_SBY-WDRI-EN1-SASSx-OPT sides at the same
info
k
metric 65000 time
exit all

/configure router interface to_SBY-WDRI-EN1-

SASSx-OPT
no port
exit all

/configure router no interface to_SBY-WDRI-

EN1-SASSx-OPT

/configure port 3/1/1 no description

/configure port 3/1/1 shutdown

5 Verification after cut off At SBY-WDRI-EN1-SASSx-OPT Chec 5 minutes Make sure interface
link k is down.
show port 1/1/27

show router interface to_SBY-SSU-EN1-SASR6 Make sure IP

14
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

show router ospf interface to_SBY-SSU-EN1- unreachable.

SASR6 detail | match Metric
show router ospf neighbor to_SBY-SSU-EN1-
SASR6 Make sure traffic is
show router ldp interface to_SBY-SSU-EN1- empty.
SASR6
show router mpls interface to_SBY-SSU-EN1-
SASR6
show router rsvp interface to_SBY-SSU-EN1-
SASR6

show router mpls lsp transit | match 1/1/27

show router mpls lsp terminate | match
1/1/27
show router mpls lsp path | match 1/1/27

ping 114.0.204.103 rapid count 1000 do-not-

fragment

monitor port 1/1/27 rate interval 3 repeat 3

| match %

At SBY-SSU-EN1-SASR6

show port 3/1/1

show router interface to_SBY-WDRI-EN1-SASSx-

OPT
show router ospf interface to_SBY-WDRI-EN1-
SASSx-OPT detail | match Metric
show router ospf neighbor to_SBY-WDRI-EN1-
SASSx-OPT
show router ldp interface to_SBY-WDRI-EN1-
SASSx-OPT

15
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

show router mpls interface to_SBY-WDRI-EN1-

SASSx-OPT
show router rsvp interface to_SBY-WDRI-EN1-
SASSx-OPT

show router mpls lsp transit | match 3/1/1

show router mpls lsp terminate | match 3/1/1
show router mpls lsp path | match 3/1/1

ping 114.0.204.102 rapid count 1000 do-not-

fragment

monitor port 3/1/1 rate interval 3 repeat 3

| match %

6 Save configuration admin save Chec 1 minutes

k

16
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

5.3 Procedure After Activity (00:30 - 01:25)

(Call i-NOC Fault Handling IP (0816100320 / 021-30441862) & i-SOC Operation (0816-100330) or NOC Coordinator: 0816801603)

Remarks/Expected
No Steps Config Status Duration
Result
1 Check all module status show time  5 minutes Log using SecureCrt
and existing configuration show uptime Check
show bof
show system information
show system cpu
show chassis
show chassis detail
show card state
show card detail
show mda detail
show log log-id 99
show log log-id 99 application snmp
show log log-id 99 application chassis
show log log-id 100
show system alarm
show port
show port detail | match expression "Speed|
Change|Rx Optical"
show port | match "Yes"
show router status
show router route-table summary
show router interface
show router ospf <All Instance ID>
interface
show router ospf <instance ID> neighbour
show router bgp summary
show router mpls status
show router mpls lsp

17
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

show router mpls lsp terminate

show router ldp status
show router ldp session
show service service-using
show service service-using | match Down
show service service-using | match Up
show service sdp
show service sdp-using
show service sap-using
show service fdb-mac
show service fdb-mac | match No.
show system sync-if-timing
2 Check Reachability NE show system security authentication  2 minutes TACACS Status : Up
(NFM-P/SAM & Tacacs) ping 10.137.32.209 Check (exclude .195)
ping 10.137.32.210
ping 10.137.32.230 Ping result OK
ping 10.34.169.101
ping 10.34.169.107
3 Create TS File Before admin tech-support  15 minutes create 2 TS file
Activity cf3:/after_CutOffLink_1.tec Check interval 15 Minutes
admin tech-support
cf3:/after_CutOffLink_2.tec
4 Monitor and Check status monitor port x/y/[a..z] interval 3 repeat 1  5 minutes
ALL port | match % Check
show port x/y/[a..z] | match expression
"Oper Speed|Oper Duplex"
5 Backup all existing admin display-config  2 minutes capture to the
configuration by display Check separate file
the configuration and
capture it.
6 Save configuration admin save  1 minutes
Check
7 Verify the responsibility Check 25 minutes
of each team (DTC,

18
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

Indosat, and another

Vendor)

19
 MoP Cut Off Link
_________________________________________________________________________________________________

6 SERVICE AND AREA IMPACT

Network Impact
SBY-WDRI-EN1-SASSx-OPT# show router interface exclude-services

SBY-SSU-EN1-SASR6# show router interface exclude-services

20
 MoP Cut Off Link
_________________________________________________________________________________________________

7 TESTING AND VERIFICATION

NO Commands Duration Expected Result

Make sure SNMP Index status
1. show system information
PERSISTENT
All card Admin State = Up, Oper State
2. show card state
= Up
All card Admin State = Up, Oper State
3. show router interface 10 minutes
= Up
Make sure :
4 show router ospf neighbor
State : FULL at all interface
All card Admin State = Up, Oper State
5 show mda
= Up

21
 MoP Cut Off Link
______________________________________________________________________________________________________________________________

8 ROLLBACK OPERATION (01:25 – End)

Duration Remarks/Expected
No Steps Config Status
Result
1 Escalate to monitoring team Check 10 minutes
2 Create TS file admin tech-support Check 10 minutes Interval 15 minutes
cf3:\rollback_CutOffLink_1.tec
admin tech-support
cf3:\rollback_CutOffLink_2.tec
3 Check rollback file show system rollback Check 1 minutes Make sure rollback
location on
cf3:\rollback-
ReengeneringLink
4 Rollback configuration admin rollback compare <Index file Check 4 minutes Use rollback index
rollback> to active-cfg file with comment
admin rollback revert <Index file
"Before CutOffLink"
rollback> now
Make sure there is no
other configuration
except config in this
MoP
5 Verification after rollback Check 4 minutes Compare results with
previous condition
6 Save configuration admin save Check 1 minutes

22
 MoP Cut Off Link
_________________________________________________________________________________________________

9 PERSON IN CHARGE

Name Company Phone Number Roles and Responsibilities

TBI DTC TBI Onsite Engineer

TBI DTC TBI Remote Monitoring – During NIW

TBI DTC TBI Remote Monitoring – After NIW

TBI Indosat TBI Customer Supervisor – During NIW

23
 MoP Cut Off Link
_________________________________________________________________________________________________

10 SUPPORTING DOCUMENTS

No Mandatory document Status

1. Link budget (SFP/XFP) No
2. Form Data Port No
3. List Impact No
4. Form Utilization No
5. Form NDB No

24