Вы находитесь на странице: 1из 50

SGNDO3001.00 Nokia 3G SGSN Rel.

3, Product Documentation

Troubleshooting Nokia 3G SGSN

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

1 (50)

Troubleshooting Nokia 3G SGSN

The information in this document is subject to change without notice and describes only the product defined in the introduction of this documentation. This document is intended for the use of Nokia's customers only for the purposes of the agreement under which the document is submitted, and no part of it may be reproduced or transmitted in any form or means without the prior written permission of Nokia. The document has been prepared to be used by professional and properly trained personnel, and the customer assumes full responsibility when using it. Nokia welcomes customer comments as part of the process of continuous development and improvement of the documentation. The information or statements given in this document concerning the suitability, capacity, or performance of the mentioned hardware or software products cannot be considered binding but shall be defined in the agreement made between Nokia and the customer. However, Nokia has made all reasonable efforts to ensure that the instructions contained in the document are adequate and free of material errors and omissions. Nokia will, if necessary, explain issues which may not be covered by the document. Nokia's liability for any errors in the document is limited to the documentary correction of errors. NOKIA WILL NOT BE RESPONSIBLE IN ANY EVENT FOR ERRORS IN THIS DOCUMENT OR FOR ANY DAMAGES, INCIDENTAL OR CONSEQUENTIAL (INCLUDING MONETARY LOSSES), that might arise from the use of this document or the information in it. This document and the product it describes are considered protected by copyright according to the applicable laws. NOKIA logo is a registered trademark of Nokia Corporation. Other product names mentioned in this document may be trademarks of their respective companies, and they are mentioned for identification purposes only. Copyright Nokia Corporation 2005. All rights reserved.

2 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Contents

Contents
Contents 3 1 2 3 3.1 3.2 3.3 3.4 3.5 3.6 4 4.1 4.2 4.3 4.4 4.5 5 6 6.1 6.2 7 8 9 9.1 9.2 9.3 9.4 9.5 10 10.1 10.2 10.3 11 12 Changes in troubleshooting 5 Troubleshooting overview 7 Gathering information about 3G SGSN 9 Alarms 10 Logs 10 Statistics 11 Traffic monitoring 11 System Level Trace 12 Subscriber information 12 3G SGSN units and processes 13 Central Routing Processor 13 Forwarding unit 14 Session and Mobility Management unit SS7 unit 14 Tunnelling unit 15 Checking the state of a process 17 Switchovers and reset methods 21 Switchovers 21 Process reset methods 22 Resetting processes 25 Fault monitoring and observation 27 Hardware faults 29 Recommended spare units 29 Faulty GPLC and CRP cards 30 Faulty CPCI interface cards 32 Faulty fan tray 32 Faulty power supply 33 Software faults 35 Checking system integrity Charging fails 36 SMM procedure fails 39 35

14

Reporting faults to Nokia 47 Glossary 49

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

3 (50)

Troubleshooting Nokia 3G SGSN

4 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Changes in troubleshooting

Changes in troubleshooting
This is the first issue of the Nokia 3G SGSN troubleshooting documentation. It includes information previously covered by the Recovery Guide for Nokia 3G SGSN.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

5 (50)

Troubleshooting Nokia 3G SGSN

6 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Troubleshooting overview

Troubleshooting overview
This troubleshooting documentation provides instructions and information on tracing a problem to its origin and resolve the issue if possible. It provides basic information and instructions for both software and hardware issues. The Nokia 3G SGSN is designed to be fault tolerant. It recovers from hardware failures and take actions in case of software faults. Monitor and react to faults as documented. If you encounter problems not covered here, contact Nokia. Use the form in Nokia Online Services (NOLS) to provide feedback about problem situations that should be added to this documentation.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

7 (50)

Troubleshooting Nokia 3G SGSN

8 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Gathering information about 3G SGSN

Gathering information about 3G SGSN


The troubleshooting may consist of one or several stages from the systems fault management process.

Detection is the process of discovering that an error exists. Detection is defined as the time from a failure causing a loss of service, to the system becoming aware of it. Location is the process of narrowing down the failure to the defective component. Isolation takes the defective portion of the system out of service. The region that is isolated must be bounded at a point where it can be removed from all interaction with the system. Recovery is the process of reassigning the necessary resources to restore the system to an operating state. Recovery also requires restoring any portions of the system that were adversely affected by the failing component. Recovery is the final step in the process that contributes to outage time. Once the system is providing complete service again, the remainder of the process does not directly contribute to outage time. Reporting is the process that notifies the outside world that an event has taken place. This is the first step in the repair process. The repair process is indirectly related to availability. In systems employing redundancy, a statistical possibility exists that a second failure can occur in the component covering for this failure. This would result in a complete system outage. While this probability is low, the severity is high enough to make this a factor in the availability equation. It is important, even in redundant systems, to keep repair times low. Repair is the replacement of the defective component. This phase is generally designated for the operator-assisted (human) portion of the process. The repair process is broken into these phases because, firstly, the repair step is usually the most time consuming portion of the process. Secondly, this is a point in the process where mistakes can account for system outages.
The following tools and features are available in the Nokia 3G SGSN:

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

9 (50)

Troubleshooting Nokia 3G SGSN

software alarms - visual warning or fault indicators from the user interface or in a network management system hardware alarms - visual warning or fault indicators on the IP3400 hardware SNMP, standard protocol for informing about problems in a system. log files - listing of error and info messages from the software statistics - packet counters from processes, interfaces, linecards, or the whole system monitoring - additional indicators in the user interface that provide system status and hardware information Traffic monitoring System Level Trace

3.1

Alarms
In troubleshooting situations it is good to start by viewing the current list of active alarms. The fault management can either be enabled or disabled in 3G SGSN. If it is enabled, you can set up the 3G SGSN to act upon alarms by logging the alarms. For a complete collection of alarm descriptions, see Nokia 3G SGSN Alarms. For instructions on configuring fault management, see Fault management and SNMP management in Operating and Maintaining Nokia 3G SGSN.

3.2

Logs
The alarm history can be followed from the alarm log, which also shows the cleared alarms. The following platform log files in /var/log can contain additional information:

fmTraps.log

Contains information about raised alarms and cancellations.

10 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Gathering information about 3G SGSN

httpd_access_log

Contains information about each transaction in the Voyager interface. Contains information about each error in the Voyager interface. Whenever the Voyager interface is enabled or disabled, it is logged to this file. General log file of the system (syslog). This log contains the error logs that 3G SGSN produces. This is one of the key sources of information when troubleshooting.

httpd_error_log

messages

3.3

Statistics
Statistics information provided by the 3G SGSN can be used in troubleshooting situations. The following types of statistics are available:
.

general statistics (including CDR, GTP, Online Service Controller, and Nokia Subscription Manager statistics) PDP session management statistics QoS statistics IP statistics

For more information, see Introduction to Nokia 3G SGSN Performance Indicators and Statistics in Nokia 3G SGSN Performance Indicators.

3.4

Traffic monitoring
You can use sgndump to monitor traffic on 3G SGSN interfaces. It is similar to the UNIX utility tcpdump, but with an extension for decoding GTP messages. Refer to any common UNIX man page of tcpdump for description of the command options. They are the same with sgndump. When using sgndump, it is important to keep the following in mind:
.

You need to make sure you are monitoring traffic on the right interface. If you want to store the traffic information, you need to first create a file where the traffic information can be dumped.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

11 (50)

Troubleshooting Nokia 3G SGSN

The sgndump command needs to be executed from the command line on the active CRP. When you execute a traffic dump, make sure there is enough free space on the harddrive of the CRP. You should monitor disk usage during the dump. An alarm (32805) is raised if you use too much memory.

Note
The sgndump utility collects, but does not decode the SIGTRAN traffic.

3.5

System Level Trace


System Level Trace (SLT) helps in detecting problems in the network that affect service level. When tracing is enabled, each invoking event (GPRS attach/detach, PDP context activation/deactivation/modification) activates the trace automatically. The trace record is generated and transferred to the network management system. For more information on how to carry out traces, see Trace handling in Operating and Maintaining Nokia 3G SGSN.

3.6

Subscriber information
3G SGSN has a utility smmdump that can be used to gather subscriber information from the SMM database. This information is also valuable when trying to isolate problems in 3G SGSN. For more information on handling subscriber information, see Fault traceability in Operating and Maintaining Nokia 3G SGSN.

12 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

3G SGSN units and processes

4
4.1

3G SGSN units and processes


This is an overview of 3G SGSN units (linecards) and their tasks. For more information, refer also to the 3G SGSN product description.

Central Routing Processor


The Central Routing Processor (CRP) is the operation and management unit of the system and has a hard disk for permanent storage. The switch fabric is physically located on this linecard. Each IP3400 has two CRPs, one acting as the active unit and the second one in standby mode, ready to take over in case the active CRP fails. The active CRP manages the installed software, interfaces, collects statistics, and distributes the routing information. The two CRPs synchronise the system configuration files in real-time and synchronise other critical files like application packages every 5 minutes. The following processes run on this unit:
.

dbd - handles subscriber database filtering, dumping, and deletion requests issued from the Voyager management console O&M - This is not a process, but a collection of services. It provides configuration services through Voyager with xpand, for example, or an external interface with daemons. It also offers performance management and other counters with a pmd process that is in every linecard. pmd - collects and saves statistics to a fail-safe disk system. This process also collects the performance indicators and sends them to NetAct on request via the NE3S interface. sctptrn - provides the SCTP translation layer. (SCTP itself is part of the platform functionality.) sltd - system level trace daemon. This process performs the system level trace functionality of the 3G SGSN. sysman - performs the same tasks as the signalling system manager, but for non-signalling processes and the sigsm process.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

13 (50)

Troubleshooting Nokia 3G SGSN

4.2

Forwarding unit
The Forwarding units in the Nokia 3G SGSN which provide Gn and Ga interface connectivity using Ethernet interfaces. Routing protocols are use on these interfaces to provide redundancy. All FUs and their interfaces can be used at all times when configured.

4.3

Session and Mobility Management unit


Similair to the CRP, the Session and Mobility Management Units manage the connections in the IP3400 system. SMM units control all the signalling traffic and maintain a database of attached subscribers. A Nokia 3G SGSN has two SMM units, one active and one standby.The SMM database is synchronised to the standby SMM unit. The following processes run on this unit:
.

smm - provides the SMM application, including CDR creation (M, SMS, and LCS CDRs). This process handles the protocol users for RANAP, MAP and CAP, in addition to the full GTP-C implementation (protocol and user). cdrsender - sends M-, S-SMO-, S-SMT-, LCS-MT, and LCS-MOCDRs to Nokia Charging Gateway

4.4

SS7 unit
The Signalling System No. 7 (SS7) units provide connectivity to the Gr interface to reach, for example, the HLR. Even though only one SS7 at a time has the control over the connections, all SS7 interfaces are active, meaning the active unit may use an interface in the standby unit. The following processes run on this unit:
.

bbsig - provides MTP3, M3UA distributor (M3D), SCCP and RANAP distributor (RD) in the broadband stack m3uasigbb - provides the M3UA protocol layer including the layer manager (m3ualm) for the broadband stack m3uasignb - provides the M3UA protocol layer including the layer manager (m3ualm) for the narrowband stack

14 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

3G SGSN units and processes

nbasig - provides MAP and CAP stacks (and distributor functions) and layer manager to MAP and CAP. It also takes care of the interface to smm. nbsig - provides the M3UA, MTP-3, SCCP, and TCAP stacks, and the layer manager to these stacks. pmcdrv1, pmcdrv2 - manage the E1 card and the MTP2 driver. There is one per MTP2 card. This process also initialises the actual SS7 interface cards. sigsm - signalling system manager process. This process manages fault tolerance operations for signalling related processes, for example, performing monitoring and switchover of fault tolerant and redundant signalling applications in the 3G SGSN.

4.5

Tunnelling unit
There can be two to nine Tunnelling units (GPLCs) in the 3G SGSN, depending on the system configuration. The Tunnelling unit provides the Iu interface and handles the user plane. The following processes run on this unit:
.

atmsig - provides SSCOP and SSCF layers, the adaption layers between AAL and MTP3. AAL and ATM layers themselves run on the ATM card. cdrsender - sends S-CDRs to Nokia Charging Gateway kGTP - loadable kernel module that provides the user data handling part of the tunnelling, Quality of Service, Charging and Lawful Interception functions. It also handles part of the GTP-U signalling and statistics gathering. This process starts automatically and cannot be manually stopped or restarted. There is still a log for it, which can be seen from the fault traceability pages in Voyager. tunnel - provides tunnelling, quality of service, charging, PDP context database, and lawful interception

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

15 (50)

Troubleshooting Nokia 3G SGSN

16 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Checking the state of a process

Checking the state of a process


Summary

There are three alternatives for checking process status: 1. Check the process states with the Voyager interface http://<sgsn_ip_address>/opt/cgi-bin/sgsn/processconf.tcl 2. If you do not have a graphical user interface available, browse to the System Supervision page using lynx (a text-based browser). To start lynx, first open a telnet connection to the active CRP, then enter lynx at the command line prompt. If neither Voyager nor lynx work, open a console connection to the active CRP and use the command line interface to check process states as instructed below.

3.

Steps

1.

Check if the process is enabled or disabled. To check the process state (enable/disable) for CRP, enter:
dbget package:sgsn:process:sysman

To check the process state (enable/disable) for GPLCs, enter:


dbget node:<slot no.>:package:sgsn:process:\ <process name>

Where <process name> is one of the following:


. . . . .

atmsig bbsig cdrsender nbasig m3uasigbb

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

17 (50)

Troubleshooting Nokia 3G SGSN

. . . . . . .

m3uasignb nbsig pmcdrv1 pmcdrv2 sigsm smm tunnel

Expected outcome

Return value 't' means that the process is enabled. No return value means that the process is disabled. 2. Check process states on CRP. Use the following commands to check the CRP processes:
dbget sgsn:sgsnCrp:sgsnCrpStat:\ sgsnProcessStateTable:17:sgsnSysmanState dbget sgsn:sgsnCrp:sgsnCrpStat:\ sgsnProcessStateTable:17:sgsnDbdState dbget sgsn:sgsnCrp:sgsnCrpStat:\ sgsnProcessStateTable:17:sgsnSltdState dbget sgsn:sgsnCrp:sgsnCrpStat:\ sgsnProcessStateTable:17:sgsnPmdState

The slot number is 17 if CRP-A is the active CRP, or 18 if CRP-B is active. 3. Check process states on the GPLCs. Use the following command to check the sate (active, standby, out-ofservice, and so on) of processes running on GPLCs.
dbget sgsn:sgsnCrp:sgsnCrpStat:\ sgsnProcessStateTable:<slot no>:<process name>

Where <process name> is one of the following: sgsnSmmState sgsnCdrSenderState sgsnNbsigState

18 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Checking the state of a process

sgsnBbsigState sgsnTunnelState sgsnAtmsigState sgsnPmcdrv1State sgsnPmcdrv2State sgsnSigsmState sgsnM3uasignbState sgsnM3uasigbbState sgsnNbasigState


Expected outcome

The possible return values of the dbget commands are:


.

0 = unknown 1 = disabled 2 = init 3 = ready 4 = active 5 = standby 6 = oos 7 = activeStandalone 8 = activeUpdating 9 = activeSynchronized 11 = standbyUpdating 12 = standbySynchronized

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

19 (50)

Troubleshooting Nokia 3G SGSN

Process state definitions:

unknown (0)

The state of the process is unknown. Note that when the pmcdrv process is started, it is at first temporarily in this state. Process is disabled. A process is init state when it is started. Process is ready when it is started and has read the configuration. It is 'ready' to be the active or standy process. Process is actively handling protocol messages. A process is in a standby state when the software is acting as a backup to the corresponding active process on the Active Node. When in this state, the process is not actively involved in protocol message processing. A process is in an out of service (oos) state when the software is unable to process protocol messages. Either the software on the node has not been configured or it has crashed due to a fault.

disabled (1) init (2) ready (3)

active (4) standby (5)

oos (6)

activeStandalone (7) Fault tolerant process is activeStandalone when the process is actively handling protocol messages. activeUpdating (8) Active fault tolerant process (SMM) is activeUpdating when the process is updating databases with the active process. activeSynchronized (9) Active fault tolerant process (SMM) is activeSynchronized when the software is actively processing protocol messages. standbyUpdating (11) Standby fault tolerant process (SMM) is standbyUpdating when the process is updating databases with the standby process. standbySynchronized (12) SMM process is in a standbySynchronized state when the software is acting as a backup to the corresponding active process on the active SMM unit. When in this state, the process is not actively involved in handling protocol message.

20 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Switchovers and reset methods

6
6.1

Switchovers and reset methods


Switchovers
Central Routing Processor

A switchover to the second CRP occurs when the active CRP has a critical software or hardware fault, or when it is manually initiated by the user. In addition, the user may defined in Voyager any of the following events to trigger a switchover:
.

Disk failure Active CRPs fabric down Overheating High/low voltage

The system is interrupted for a short period of time during a CRP switchover. This is due to the handover of the control to the switch fabric in the second CRP, and the linecards reconnecting and exchanging information with the new, active CRP.
SMM Unit

If the SMM unit fails, a switchover is performed to the standby unit. This should not affect the end users, if the status returns to active after the failure. If there is an SMM switchover, check the following to ensure that everything continues smoothly: 1. 2. Check the states of the processes running on the SMM unit. Check the Session Management statistics. Ensure that they are increasing by reloading the page a few times on the web browser.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

21 (50)

Troubleshooting Nokia 3G SGSN

3. 4.

Check the Mobility Management statistics. Ensure that they are increasing by reloading the page a few times on the web browser. Check that the broadband and narrowband signalling links are up.

Also check from the Voyager interface that the state of the crashed SMM process is now standby. Ensure that the SMM process backs up all MM and PDP contexts from the active unit by checking the start and end times of the backup process. These times correspond to the activation and cancellation times of the alarm 32385 STANDBY COPYING DB FROM ACTIVE.

6.2

Process reset methods


atmsig

This is a standalone process, and no switchover is performed. When disabled, all broadband links configured via the affected TU are dropped down and the SCCP connections are released.
bbsig

This is a fault tolerant process, and a switchover is performed if an active process is reset. Wait for the switchover to finish (PROCESS FAILED alarms are cancelled and the process states are as expected in the process status table in Voyager) before performing another reset.

Note
If both processes are disabled at the same time, all links to the RNCs are dropped and the SCCP connections released.

cdrsender

This is a standalone process, and no switchover is performed. When disabled, all CDRs in the send queue are lost. For this reason, you should never manually restart the process. However, the cdrsender process needs to be restarted if the Ga address for the given linecard is changed.

22 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Switchovers and reset methods

m3uasigbb and m3uasignb

This is a fault tolerant process, and a switchover is performed if an active process is reset. Wait for the switchover to finish (PROCESS FAILED alarms are cancelled and the process states are as expected in the process status table in Voyager) before performing another reset.
nbasig

All instances run as active, without standby. They operate in a load sharing relationship. If one process instance fails, the signals handled by process at the time are lost, but the 3G SGSN maintains the MAP and CAP signalling functionality through the other process.
nbsig

This is a fault tolerant process, and a switchover is performed if an active process is reset. Wait for the switchover to finish (PROCESS FAILED alarms are cancelled and the process states are as expected in the process status table in Voyager) before performing another reset.
pmcdrv1 and pmcdrv2

All instances of this process run as active without standby processes. When disabled, all narrowband links configured via the affected process are dropped.
sctptrn

This is a standalone process within the CRP, and a switchover is performed only during a CRP switchover.
sigsm

This is a fault tolerant process, and a switchover is performed if an active process is reset. Wait for the switchover to finish (PROCESS FAILED alarms are cancelled and the process states are as expected in the process status table in Voyager) before performing another reset.
smm

This is a fault tolerant process, and a switchover is performed if an active process is reset. Wait for the switchover to finish (PROCESS FAILED alarms are cancelled and the process states are as expected in the process status table in Voyager) before performing another reset.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

23 (50)

Troubleshooting Nokia 3G SGSN

After the SMM switchover or a standby SMM reset, a backup for the MM and PDP contexts is started. Another switchover may not be performed before the backup is finished and the alarm 32385 STANDBY COPYING DB FROM ACTIVE is cancelled. This can take several minutes, depending on the number of contexts in the database. Another switchover during the backup results in lost contexts in the SMM unit.
sysman

This process should always be running on the CRP, and it cannot be controlled from Voyager. However, if a restart is required, it can be done from the command line. To disable the process, use the command: dbset package:sgsn:process:sysman To enable the process, use the command: dbset package:sgsn:process:sysman t
tunnel

This is a standalone process, and no switchover is performed. When disabled, all PDP contexts on the affected TU are released. Manual CDR generation must be performed first (Charging basic configuration), otherwise all charging data is lost.

Note
Never restart several TUs or tunnel processes simultaneously if there are many active PDP contexts. This could overload the entire network because the 3G SGSN starts to deactivate the related PDP contexts from the GGSN and page idle subscribers to deliver the Delete PDP Context Request.

24 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Resetting processes

Resetting processes
Purpose

Controlled process resets and switchovers should always be performed by disabling/enabling the process from the process configuration page in the Voyager. Processes can also be terminated from the command line in the active CRP, but this method should only be used if disabling/enabling through Voyager is not successful.
Steps

1.

Disable the process.


dbset sgsn:sgsnCrp:sgsnCrpConf:sgsnProcessTable:\ <slot no>:sgsnProcess<process name> 2

Example 1. The process name must start with a capital letter!


dbset sgsn:sgsnCrp:sgsnCrpConf:sgsnProcessTable:1:\ sgsnProcessSmm 2

2.

Enable the process.


dbset sgsn:sgsnCrp:sgsnCrpConf:sgsnProcessTable:\ <slot no>:sgsnProcess<process name> 1

Example 2. The process name must start with a capital letter!


dbset sgsn:sgsnCrp:sgsnCrpConf:sgsnProcessTable:1:\ sgsnProcessSmm 1

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

25 (50)

Troubleshooting Nokia 3G SGSN

26 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Fault monitoring and observation

Fault monitoring and observation


The critical resources of the 3G SGSN system can be monitored via the Voyager and CLI interfaces. Failure situations are also indicated with alarms. For more detailed information see, Operating and Maintaining Nokia 3G SGSN, Nokia 3G SGSN Alarms and CLI Reference Guide for IPSO 3.9NET.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

27 (50)

Troubleshooting Nokia 3G SGSN

28 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Hardware faults

9
9.1

Hardware faults
Recommended spare units
Reserve the following units as spares for each IP3400 system:
.

1 x CPCIs (one of each type used in the system) 1 x CRP 1 x fan tray 1 x GPLC 1 x harddisk 1 x power supply

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

29 (50)

Troubleshooting Nokia 3G SGSN

9.2
Description

Faulty GPLC and CRP cards

FABRIC

POWER

ALERT RESET

S1 S2

FAULT

LINK 10Base-T 100Base-Tx ACTIVITY

FABRIC

POWER

CONSOLE (COM 1)

ALERT

FAULT

CRP status Link LED Ethernet Port Activity LED Reset Switch
RESET
SLOT 2

LINK 10Base-T 100Base-Tx ACTIVITY

SLOT 1

Figure 1.

CRP card front panel details

In normal operation the POWER LED is lit. It is also normal that the FABRIC led is lit or blinking. Also, blinking power LED indicates that the card is being booted, and a blinking fabric led indicates SW download via the fabric. The fabric LED is also off on the CRP if there are no fabric connections to GPLCs. This is always the case for the standby CRP. The conditions when GPLC or CRP LEDs are lit are shown in the following table.

30 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Hardware faults

LED
POWER FABRIC ALERT FAULT

Condition when lit


The unit is in normal operation. Traffic indicator. Indicates alert condition. There is a fault condition.

If a switchover has occurred between two CRP cards one of the CRP cards could be faulty. In this case check the status indicated by the LEDs on the CRP front panel. If no front panel LEDs are illuminated it could indicate a faulty CRP or GPLC card. If the FAULT LED is illuminated it could indicate a faulty CRP or GPLC card.
Symptoms

ALERT or FAULT LEDs are lit, or no LEDs are lit, on the front panel of the CRP or the GPLC card.
Recovery procedures
.

Check the system status pages and alarm list in Voyager. Reset the faulty unit from the Voyager interface and check if the card has recovered from the fault situation. Reset the faulty unit using the Reset switch located on the front panel of the GPLC or CRP card and check if the unit has recovered from the fault situation. Remove the faulty GPLC or CRP card from the system and insert it again. Always use the halt command to halt the operating system on the CRP before removing a CRP from the system. If you cannot halt the CRP for some reason, press the reset button on the CRP before removing it. Wait until it boots and check if it has recovered from the fault situation. Replace the faulty GPLC or CRP card with a spare card. Contact the Nokia Care organisation and report the problem. Report the version information and serial number of the faulty card to Nokia.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

31 (50)

Troubleshooting Nokia 3G SGSN

For information on the location of the card version information and serial number see Section Module identification.

9.3
Description

Faulty CPCI interface cards

This section describes what to do if a CPCI interface card is faulty.


Symptoms

One of the following alarms is sent: Unit Missing, Unit Mismatch, or Interface Failure.
Recovery procedures
.

Check if the connectors on the CPCI interface card are damaged. Replace the faulty card with a spare CPCI interface card. It is possible to hotswap the faulty card and add a new card. Before inserting CPCI interface cards into the GPLC card, inspect the GPLC connectors and the connectors on the CPCI interface card for damage (such as slightly bent pins). In you observe any damage, do not inset the card. After you partially insert a CPCI interface card into the slot, verify that it is properly seated in the guide slots. If a card is not properly seated and you push it all the way in, you can damage the CPCI interface card and the GPLC card. Do not force the card in.

Contact Nokia Care organisation and report the problem. Report version information and the serial number of the faulty card to Nokia.

For information on the location of the card version information and serial number see Section Module identification.

9.4
Description

Faulty fan tray

This section describes what to do if there is a hardware fault on the fan tray.

32 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Hardware faults

Symptoms

The system reports a faulty fan on the primary fan tray.


Recovery procedures
.

Replace the whole fan tray with a spare unit.

Note
Do not operate the system for longer than one (1) minute under a full traffic load without the primary fan tray installed.

Contact Nokia Care organisation and report the problem. Report version information and the serial number of the faulty fan tray to Nokia.

For information on the location of the version information and serial number see Section Module identification.

9.5
Description

Faulty power supply

The power supply unit has three LEDs located on the front panel.

Figure 2.

Power Supply LEDs

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

33 (50)

Troubleshooting Nokia 3G SGSN

The power supply LEDs are lit in the following conditions:

LED
AC OK: AC power OK

Condition when lit


The input power is within the specified operating range. There is a detectable fault condition. The power supply output exceeds the specified operating range (48 volts). The power supply temperature exceeds the specified operating range.

PS FLT: Power supply fault CL LED: Current limit

CT LED: Over temperature

In normal operation the AC OK LED is lit.


Symptoms

The PS FLT LED is lit.


Recovery procedures
.

Replace the faulty power supply unit with a spare unit. The IP3400-3 system is able to operate with three power supplies during the service period. Contact Nokia Care organisation and report the problem. Report version information and the serial number of the power supply to Nokia.

For information on the location of the version information and serial number see Section Module identification.

34 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Software faults

10
10.1

Software faults
Checking system integrity
Purpose

Configuration errors are the most common source of problems in 3G SGSN. A typical configuration error is using inconsistent parameters (and software builds) for linecards that have the same function in the 3G SGSN unit (for example, tunnelling units). 3G SGSN has a built-in feature that allows you to verify your system configuration. This should be done as the first step when troubleshooting software-related problems.
Steps

1.

Verify the software package status. a. In Voyager, enter the Configuration Integrity Tools dialogue by selecting the link Configuration Integrity Tools under the Basic Services category of the 3G SGSN configuration. View the SGSN Packages Status table and verify that all bullets are green.

b.

2.

Verify configuration consistency. a. In Voyager, enter the Configuration Integrity Tools dialogue by selecting the link Configuration Integrity Tools under the Basic Services category of the 3G SGSN configuration. In the Consistency Check Status table, click the OK button to run a system check. View the Consistency Check Status table and verify that all bullets are green.

b. c.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

35 (50)

Troubleshooting Nokia 3G SGSN

10.2
Description

Charging fails

There are three basic fault types concerning charging:


.

CDRs are not created CDRs are not sent CDR fields are missing or incorrect

You main sources of information concerning these problems are the chargingrelated counters and alarms.
Symptoms CDR queue gets full

This is indicated by alarms, so you need to follow up by first checking the active alarms.
CDRs are not created

CDR types are not being created as expected.


CDR fields are missing or incorrect

CDR field are empty (fixed format) or missing (TLV format), or they contain unexpected values. This is usually indicated by a validation failure at the CG end, or errors in the Customer Care and Billing System (CCBS). The value 'abnormal' the Cause for Record Closing field also can indicate that fields are missing.
Recovery procedures

In most cases, charging problems arise due to configuration errors, either in the 3G SGSN or in some part of the network. Charging can otherwise handle most other error situations. If a problem does occur, you may need to investigate what happened in the 3G SGSN or network several hours or days before the problem arose.

36 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Software faults

Isolating the source of a CDR queue overflow Purpose

If the CDR queue of a cdrsender is getting too full, it usually means that the messages from the cdrsender process are getting blocked at some point between the SMM or Tunnelling Unit and the target Charging Gateway (CG), or the acknowledgment messages from the CG are not getting back to the cdrsender. If the connection to the Charging Gateway (CG) is lost, this also causes the queue to fill up. note that each GPLC can send this alarm, and that the alarm is generated per CG.
Summary

It is unlikely that the problem is caused by the cdrsender process itself, but by one of the interfaces or elements between cdrsender and the CG. In most cases, an alarm is raised that gives you details on what caused the problem. When trying to locate the problem, it is best to work 'backwards', from the CG back to the cdrsender process. Check the traffic at each point to locate where the messages are being lost, then check the related network configuration.
Steps

1.

Check active alarms. Look for any charging-related alarms, in particular:


. . .

67016 NO CONNECTION TO CG 67015 CONNECTION LOST TO CDR SENDER 32333 CANNOT INITIALISE CHARGING

2.

Check the GTP message log of the CG to see which messages are being received from the 3G SGSN and which responses CG is sending back. Check the traffic on any routers you are using between the 3G SGSN and CG to see if any message are being lost/blocked there. Check the traffic on the Ethernet interface of the Forwarding Unit in question using sgndump. Check the traffic on the VPP interface between the SMM or Tunnelling Unit and Forwarding Unit in question using sgndump.

3.

4.

5.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

37 (50)

Troubleshooting Nokia 3G SGSN

If the acknowledgement messages are getting to the Forwarding Unit but not to the SMM or Tunnelling Unit in question, then the problem is a missing static route. A static route must be configured between these units for the response messages to get all the way back to the cdrsender process.
Further information

If the queue is full because the cdrsender process itself is not working, this is probably a software bug that needs to be handled by Nokia. When information Nokia of such a case, you need to include the following information in your report:
.

messages log file charging configuration parameter values alarm trap list

When dealing with connection problems to CG, keep in mind how the 3G SGSN selects CG addresses. For S-CDRs, the CG address is received from GGSN. The received address is used if the address is in 3G SGSN list of allowed CGs (Charging Gateway configuration in Voyager). If the 3G SGSN checks the address against its own list and does not find it, the 3G SGSN selects an address from the list based on configured priority levels. All other CDR types are automatically sent to the CG address with the highest priority according to the list in 3G SGSN.
Isolating the source of CDR creation failures Steps

1.

Check that the CDR type creation is enabled. In Voyager you can find this information from the Basic Charging Configuration page. The Tunnelling Unit creates S-CDRs, and the SMM Unit generates M-, SSMO-, S-SMT-, LCS-MT, and LCS-MO-CDR types.

2.

Check the current list of alarms. Look for any charging-related alarms, in particular:
. . .

32335 CANNOT CREATE CDR 67012 CANNOT INIT SMM CHARGING 67013 CANNOT INIT TUN CHARGING

38 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Software faults

3.

Monitor the charging counters. There are CDR creation counters for each CDR type. Refresh the page in Voyager to see which counters are increasing. Any counter that does not increase tells you exactly which CDR type is not being created.

Isolating the cause of corrupted CDRs Summary

In most cases, if a CDR field is missing or has the wrong information, it usually concerns information not generated by the 3G SGSN. So the configuration of the source of the information should be checked first. The information usually is lost due to either a failed inter-SGSN routing area update or a relocation procedure fails.
Steps

1.

If the charging ID is missing from the CDR, check the GGSN configuration. A missing charging ID usually indicates that the GGSN rejected the PDP context after the routing area update or relocation procedure.

2.

If the MSISDN is missing from the CDR, check the HLR configuration. A missing charging ID usually indicates that the HLR rejected the PDP context after the routing area update or relocation procedure.

10.3
Description

SMM procedure fails

Signalling sequences in Session and Mobility Management (SMM) procedures are a potential source of problems in 3G SGSN due to the complex nature of these procedures in 3G networks. There are several points within each procedure that can cause the entire sequence to fail. These troubleshooting instructions deal specifically with the Attach and PDP Context Activation procedures.
Symptoms

SMM performance indicators recording failure instances increase rapidly.

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

39 (50)

Troubleshooting Nokia 3G SGSN

Recovery procedures Interpreting the messages log file Purpose

To locate the exact point of failure when an you have SMM problems, you need to carefully analyse the messages log file. Each SMM sequence generates certain log entries that you can use as checkpoints, the objective being to find which checkpoints has an error message as the log entry.
Steps

1.

Determine which SMM procedure failed. Search the messages log file for the follow type of entry :
May 30 14:04:56 fe80::5501 [LOG_INFO] smm: (1896:270284) EXT_PROC_END: [MODULE]: [PROCEDURE NAME]: NOK: cause=0x0000: [INFO] ([FILE]: [LINENRO])

The PROCEDURE NAME value indicates which procedure failed. 2. Analyse the messages log file. See the examples below for details on which log entries to look for. The entries correspond to certain checkpoints in the SMM procedure. Use these to determine which phase (which interface) of the procedure failed. 3. Take corrective actions to fix the identified error. The correct action of course depends on which phase of the SMM procedure failed. However, the general guideline is to: a. b. First check that the relevant parameters in 3G SGSN are configured correctly. If the 3G SGSN configuration is correct, check for configuration errors elsewhere in the network. Also look for recent changes in your network configuration that require 3G SGSN changes (that were not recognized or implemented at the time). If you cannot locate any configuration problems, contact Nokia. Be sure to provide all relevant error messages from the messages log.

c.

Example 3. Attach sequence

40 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Software faults

The following figure illustrates the signalling sequence for the Attach procedure in 3G SGSN:

MS
1

UTRAN

New SGSN

Old SGSN

EIR

HLR

Attach message Ue_state_Mm_Idl, Ue_state_Mm_con Ue_state_Mm_det,Attach_proc, After collision / OLC Reuse cache hits, cache misses Ue contexts in use Identification request Identity request Identification response 2-3

4-5

Identity response Authentication IMEI Check Identity Check procedure 6-7 Auth. procedures, Security failures 12-13 Common Id Security Mode Identity Check procedure Update location Insert subscriber data Insert subscriber data ack Update Location Ack Camel_GPRS_ attach Attach TDPs, Currently allocated GPRS sessions, CAP dialogue failures, GPRS sessions continued after CAP failure (def. handl. = cont) Attach failures 17-20

8-9 10-11 14 15-16

21 22

Attach Accept / Reject Attach Complete

Figure 3.

Attach sequence

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

41 (50)

Troubleshooting Nokia 3G SGSN

Each numbered step in the sequence produces an entry in the messages log file. The following are log entries that are produced at the corresponding checkpoint in the Attach procedure. 1.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:703218) EXT_MSG_IN: L3MM: Attach Request: trid=0x????????: buffer=08 01 02 E1 61 09 32 6B 08 29 44 50 01 10 00 80 50 42 F0 80 00 69 69 05 23 67 80 07 E0 (l3mm_msg_handler.cc:301) //191139

2.
May 30 13:55:04 fe80::5501 [LOG_INFO] smm: (1304:199054) EXT_MSG_OUT: GTP-C: Identification Request: teid=0x????????: seq_num=0x????: ipv4_addr=10.20.82.131:2123: (gtpc_main_handler.cc:820) //146640a

3.
May 30 13:55:04 fe80::5501 [LOG_INFO] smm: (1304:207240) EXT_MSG_IN: GTPC: Identification Response: teid=0x????????: seq_num=0x????: ipv4_addr=10.20.82.131:2123: (gtpc_main_handler.cc:816) //146663

4.
May 30 14:01:53 fe80::5501 [LOG_INFO] smm: (1713:734376) EXT_MSG_OUT: L3MM: Identity Request: trid=0x????????: buffer=08 15 01 (l3mm_msg_handler.cc:131) //211222

5.
May 30 14:01:53 fe80::5501 [LOG_INFO] smm: (1713:866875) EXT_MSG_IN: L3MM: Identity Response: trid=0x????????: buffer=08 16 08 29 44 50 01 10 00 80 50 (l3mm_msg_handler.cc:487) //211255

6.
May 30 14:01:53 fe80::5501 [LOG_INFO] smm: (1713:875294) EXT_MSG_OUT: MAP: sendAuthenticationInfo: trid=0x????????: muid=0x????????: imsi=244051001000805f (mapu_proc_send_authentication_info.c:158)

7.
May 30 14:01:53 fe80::5501 [LOG_INFO] smm: (1713:904369) EXT_MSG_IN: MAP: sendAuthenticationInfoRes: trid=0x????????: muid=0x????????: error= (mapu_proc_send_authentication_info.c:217) //211408

8.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:711748) EXT_MSG_OUT: L3MM: Authentication And Ciphering Request: trid=0x????????: buffer=08 12 10 00 21 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 81 28 10 48 9A 4D 9B FA 9D 24 17 00 01 48 9A 4D 9B FA 9D (l3mm_msg_handler.cc:131) //191268

9.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:869468) EXT_MSG_IN: L3MM: Authentication And Ciphering Response: trid=0x????????: buffer=08 13 00 22 11 23 31 47 23 09 43 09 31 17 90 25 00 01 F9 29 0C 51 63 71 8F 19 2B 39 4F 59 6B 79 0E (l3mm_msg_handler.cc:487) //191311

10.
May 30 13:55:32 fe80::5501 [LOG_INFO] smm: (1332:268151) EXT_MSG_OUT: L3MM: Identity Request: trid=0x??????: buffer=08 15 03

42 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Software faults

(l3mm_msg_handler.cc:131) //150547

11.
May 30 13:55:32 fe80::5501 [LOG_INFO] smm: (1332:283845) EXT_MSG_IN: L3MM: Identity Response: trid=0x????????: buffer=08 16 09 43 09 31 17 90 25 00 01 F9 (l3mm_msg_handler.cc:487) //150584

12.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:874405) EXT_MSG_OUT: MAP: checkImei: trid=0x????????: muid=0x????????: imei=4901371095200109 (mapu_proc_check_imei.c:116) //191382

13.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:900011) EXT_MSG_IN: MAP: checkImeiRes: trid=0x????????: muid=0x????????: equip_status=0x00: error= (mapu_proc_check_imei.c:145) //191408

14.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:901469) EXT_MSG_OUT: RANAP: Common Id: trid=0x????????: connection_id=3c.ee (ranapu_sif.c:583) //191430

15.
May 30 14:00:16 fe80::5501 [LOG_INFO] smm: (1616:903063) EXT_MSG_OUT: RANAP: Security Mode Command: trid=0x????????: connection_id=3c.ee (ranapu_sif.c:518) //191453

16.
May 30 14:00:17 fe80::5501 [LOG_INFO] smm: (1617:056806) EXT_MSG_IN: RANAP: Security Mode Complete: connection_id=3c.ee (ranapu_successful_outcome_msg_handlers.c:112) //191466

17.
May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:872005) EXT_MSG_OUT: MAP: updateGprsLocation: trid=0x????????: muid=0x????????: imsi=244051001000805f (mapu_proc_update_gprs_location.c:143) //160114

18.
May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:897802) EXT_MSG_IN: MAP: insertSubscriberData: trid=0x????????: muid=0x????????: imsi=244051001000805f (mapu_proc_insert_subscriber_data.c:71) //160139

19.
May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:898914) EXT_MSG_OUT: MAP: insertSubscriberDataRes: trid=0x????????: muid=0x????????: error= (mapu_proc_insert_subscriber_data.c:165) //160156

20.
May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:914070) EXT_MSG_IN: MAP: updateGprsLocationRes: trid=0x????????: muid=0x????????: error= (mapu_proc_update_gprs_location.c:172) //160183

21.
May 30 14:00:17 fe80::5501 [LOG_INFO] smm: (1617:076741) EXT_MSG_OUT: L3MM: Attach Accept: trid=0x????????: buffer=08 02 01 49 00 42 F0 80 00 69 69 19 19 DA 7F 18 05 F4 E5 3E 18 4E B8 (l3mm_msg_handler.cc:131) //191768

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

43 (50)

Troubleshooting Nokia 3G SGSN

22.
May 30 14:00:17 fe80::5501 [LOG_INFO] smm: (1617:090713) EXT_MSG_IN: L3MM: Attach Complete: trid=0x????????: buffer=08 03 (l3mm_msg_handler.cc:487) //191798

Example 4. PDP Context Activation sequence The following figure illustrates the signalling sequence for the PDP Context Activation procedure in 3G SGSN:

MS
1 Activate PDP Context

RAN

SGSN

GGSN

C1
4 Radio Access Setu Invoke 6 8 Update PDP Context Update PDP Context 8 Create PDP Context Create PDP Context 5 4

C2
Activate PDP Context 9

Figure 4.

PDP Context Activation sequence

Each numbered step in the sequence produces an entry in the messages log file. The following are log entries that are produced at the corresponding checkpoint in the PDP Context Activation procedure. 1.
May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:948270) EXT_MSG_IN: L3SM: Activate PDP Context Request: mmid=a, ti=1, trid=0x000100d8 (l3sm_msg.cc:345) //160658

2. 3. 4.

No log entry checkpoint. No log entry checkpoint.

May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:973940) EXT_MSG_OUT: GTP-C: Create PDP Context Request: teid=0x00000000: seq_num=0x0047: ipv4_addr=10.20.82.132:2123: (gtpc_main_handler.cc:820) //161281

44 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Software faults

May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:986746) EXT_MSG_IN: GTP -C: Create PDP Context Response: teid=0x0a000050: seq_num=0x0047: ipv4_addr=10.20.82.132:2123: (gtpc_main_handler.cc:816) //161332

5.
May 30 13:56:14 fe80::5501 [LOG_INFO] smm: (1374:991093) EXT_MSG_OUT: RANAP: Rab Assignment Request: trid=0x000100d8: connection_id=35.d2 (ranapu_sif.c:325) //161508 May 30 13:56:15 fe80::5501 [LOG_INFO] smm: (1375:006121) EXT_MSG_IN: RANAP: RAB Assignment Response: connection_id=35.d2 (ranapu_outcome_msg_handlers.c:133) //161519

6. 7. 8.

No log entry checkpoint. No log entry checkpoint.

May 30 14:03:54 fe80::5501 [LOG_INFO] smm: (1834:915256) EXT_MSG_OUT: GTP-C: Update PDP Context Request: teid=0x0000ff51: seq_num=0x0065: ipv4_addr=10.20.83.133:2123: (gtpc_main_handler.cc:820) //240997  May 30 14:03:54 fe80::5501 [LOG_INFO] smm: (1834:916449) EXT_MSG_IN: GTP -C: Update PDP Context Response: teid=0x0f000050: seq_num=0x0065: ipv4_addr=10.20.83.133:2123: (gtpc_main_handler.cc:816) //241037

9.
May 30 13:56:15 fe80::5501 [LOG_INFO] smm: (1375:026352) EXT_MSG_OUT: L3SM: Activate PDP Context Accept: mmid=10, imsi=244051001000805F, ti=1, BUFFER: 9A 42 05 0B 23 51 1F 91 96 40 48 46 00 00 00 01 27 1B 80 80 21 0A 03 00 00 0A 81 06 0A 10 3E E6 80 21 0A 04 00 00 0A 83 06 00 00 00 00 TRID: 100D8 (l3sm_msg.cc:1522) //161737

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

45 (50)

Troubleshooting Nokia 3G SGSN

46 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Reporting faults to Nokia

11

Reporting faults to Nokia


After the system is up and running and the fault situation has been corrected with recovery instructions, you should report the fault situation to Nokia by using the Problem Reporting Tool (refer to MPC Problem Reporting Guide). The Problem Report (PR) helps analysing the fault situation and solving the problem in the SGSN node. The PR should include a clear PR title, the fault description, and all the necessary file information. Provide the following information to Nokia:
.

A brief and precise problem description Information on how to reproduce the fault situation (for example step-bystep instructions) Impact to the user Severity of the problem for the user Likelihood of problem occurrence

IPSO version used (obtain using the command uname -a) Hardware versions used. Obtain this information with the following commands: ipsctl -a | grep hw:chassis:eeprom ipsctl -a | grep hw:eeprom:serial_number ipsctl -a | grep hw:eeprom:part_number

Log files. To collect all the necessary files from the SGSN system from both CRPs, enter the following command: ipsoinfo -c -a -l The system saves the collected archive in the admin directory /var/ admin. The file name is, for example, ipsoinfo-hostname-date. tar.gz.

Note

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

47 (50)

Troubleshooting Nokia 3G SGSN

The ipsoinfo command requires a lot of processing memory. You should only use it when you really need all system information. If you just need, for example, hardware information, use the ipsctl command.

Occasionally this information is not sufficient and you need to provide additional information to Nokia. Additional information might include the network diagram, route information (enter the iclid, show route, quit command), tcpdump prints, interface information (enter the ifconfig a command), configuration settings (enter the ipsctl -a command) etc. If such information is needed, Nokia requests it separately with the Electra tool.

48 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en

Glossary

12
ATM CLI CPU CRP CPCI FCC FTP GPLC HTTP HTTPS IP IPSO MMF NEBS NIC NTP PCI SGSN SMF SSH SSL

Glossary
Asynchronous Transfer Mode Command Line Interface Central Processing Unit Central Routing Processor Compact PCI Federal Communications Commission File Transfer Protocol General Purpose Line Card Hypertext Transfer Protocol Secure HTTP Internet Protocol Nokia Ipsilon Router Operating System Multi-mode Fibre Network Equipment Building Standards Network Interface Card Network Time Protocol Peripheral Component Interconnect Serving GPRS Support Node Single-mode fibre Secure Shell Secure Sockets Layer

dn04151373 Issue 1 en

# Nokia Corporation Nokia Proprietary and Confidential

49 (50)

Troubleshooting Nokia 3G SGSN

URL

Uniform Resource Locator

50 (50)

# Nokia Corporation Nokia Proprietary and Confidential

dn04151373 Issue 1 en