TS RNC SW 038 I14

Technical Support Note
TS-RNC-SW-038
RNC PROBLEM
REPORT
INSTRUCTIONS
Radio Controllers
WCDMA RNC
RN2.1, RN2.0, RN3.0
This document contains following

type of information
Informative
Corrective
Urgent
Security
Information is classified as
Internal
Public
Customer Specific
Nokia Siemens Networks
Company Confidential
1 (33)
Table of Contents
1.
2.
3.
4.
5.
5.1
5.2
5.3
5.4
6.
6.1
6.2
6.3
6.4
7.
7.1
7.2
7.3
8.
8.1
8.2
8.3
8.4
8.5
8.6
8.7
8.8
8.9
8.10
8.11
8.12
8.13
8.14
8.15
9.
10.
11.
12.
13.
13.1
13.2
14.
15.
16.
17.
Compatibility / Dependencies to other products....................................................................... 4

Keywords ................................................................................................................................. 4
Summary.................................................................................................................................. 4
Detailed description.................................................................................................................. 4
Solution / Instructions............................................................................................................... 7
Using the log collection macro rnclogcol.exe ........................................................................... 7
Basic instructions for displaying logs and for monitoring messages ........................................ 9
Commands for displaying computer logs and system logs from DMX units ............................ 9
Commands for message monitoring in DMX units................................................................... 9
Data to be collected in all abnormal situations....................................................................... 11
SW and HW version information ............................................................................................ 11
Alarm history .......................................................................................................................... 11
Working state of units............................................................................................................. 11
MML command log................................................................................................................. 12
Case specific instructions....................................................................................................... 13
System restart ........................................................................................................................ 13
Spontaneous restart of a DMX unit (OMU, RSMU, RRMU, ICSU, GTPU) ............................ 14
Spontaneous restart of a CHORUS unit (MXU, A2SU, NIP1, NIS1, DMCU, SFU)................ 14
Restart of DSP/DMPG ........................................................................................................... 16
RNC active alarms and alarm history..................................................................................... 16
Plug-in Unit HW information................................................................................................... 16
All Function Unit states and information................................................................................. 16
DSP service statistics information.......................................................................................... 16
Collect computer log in destination DMPG ............................................................................ 16
Collect restarted DSP blackbox information........................................................................... 16
Collect sysdump information for suspected faulty DMPG unit ............................................... 17
Collect bblog information of suspected faulty DMPG unit ...................................................... 17
DMPG communication processor blackbox information ........................................................ 17
Collect DSP service related information................................................................................. 18
Collect PDRACT channel related information ........................................................................ 19
Monitor DMX message in suspected faulty DMPG ................................................................ 19
Monitor DSP message and DMX message to/from suspected faulty DSP ............................ 19
Check SW version of one specific program block.................................................................. 19
Collect diagnostics report history ........................................................................................... 20
BTS initialization case ............................................................................................................ 21
Call case ................................................................................................................................ 24
Synchronization case ............................................................................................................. 27
ESA 12/24 LAN switch case .................................................................................................. 28
OMS case .............................................................................................................................. 29
General case.......................................................................................................................... 29
Measurement case................................................................................................................. 29
NEMU case ............................................................................................................................ 30
Message monitoring from OMU with families for different problem situations ....................... 32
Note ....................................................................................................................................... 32
References............................................................................................................................. 32
2 (33)
Contact:
Contact your local Nokia Siemens Networks support
Summary of changes:
25-02-2008
11
09-04-2008
12
30-05-2008
13
25-06-2008
14
A new template and OMS log instructions

added.
Rnclogcol macro improvement. Amount of MML
sessions are limited.
Rnclogcol macro improvement and chapter 7.3
updated.
Rnclogcol macro improvement. RNC name and
time stamp added to log directory name.
Chapter 9 and chapter 10 ddesk command
added.
3 (33)
Purpose
This document contains generic information about products. These can be
instructions that explain problem situations in the field, instructions on how to prevent
or how to recover from problem situations, announcements about changes or
preliminary information as requirements for new features or releases.
1. COMPATIBILITY / DEPENDENCIES TO OTHER PRODUCTS

No affect
2. KEYWORDS
Problem report, Electra, RNC outage, System restart, Unit restart.
3. SUMMARY
This document contains basic instructions for making a problem report in such a
way that all the necessary information is included in it. Filling in the problem
report carefully and attaching required information to it makes the problem
investigation process faster.
4. DETAILED DESCRIPTION
This document contains instructions for collecting basic data from the network
element (RNC) for fault analyzing purposes. This data shall always be collected if
some abnormal situation has occurred in the system, and Nokia Siemens
Networks (NSN) will be informed of it. An abnormal situation may be, for
example, a decreased traffic handling capacity, or a spontaneous restart of a
computer unit. In the worst case the abnormal situation may lead to a system
outage.
The data to be collected is essential for the further analysis of the problem for
trying to find out the root cause of the fault. The data shall be collected as soon
as possible after the abnormal situation has taken place. This is important
because the information stored about the problem (e.g. the black box of a certain
unit, or alarm history) may get overwritten in the process of time.
The data to be collected, as well as the commands to be used for collecting them,
have been described in the instructions below. The data shall be stored in
descriptively named log files, which shall then be zipped into to file and delivered
to NSN as an attachment of the problem report / outage report.
The log collection macro (rnclogcol.exe) and ini files are available via NOLS. The
macro makes possible to collect automatically the required logs for the following
situations: NE spontaneous restart, unit spontaneous restart, RNC configuration,
call success problems and OAM (BTS) problems. See chapter Using the Log
Collection Macro rnclogcol.exe for more details.
If only few of logs are needed take manually the relevant settings for every
particular case are explained later in this document.
In addition to collecting the data, fill in the problem report by taking into account
the following issues:
4 (33)
Title of the problem report should be a very short description of the fault
situation
Description of the problem in the problem report should be an exact and a
clear description of the fault situation. Note that only one fault shall be
reported in one problem report and at least the following information should
be provided in addition to the actual problem description:
Situation in the beginning, e.g. the first symptoms of the failure
Operations made, which possibly caused the failure
Situation after the failure
Recovery actions made
Name and version of the possibly installed new software modules
Make sure that necessary attachments are included to the problem report
to avoid unnecessary information requests
In a multivendor environment, collect detailed information of other products
to the Description field of the problem report
Write down the SW Release used in the NE.
Write down the SW build of the SW Release and the implemented CD level
in RNC. This information can be obtained with the MML command
ZWQO:CR;
For filling in the severity of the problem, see the definition of the different
severities in Table 1, NSNs Problem Classification:
NSN Problem
Class
A-Critical
Only total or
major outages
that are not
avoidable with a
workaround
solution.
B-Major
The fault affects
traffic randomly
or problem
leads only to
degradation of
network
performance or
Definition of problem report severity

as defined in TL 9000 Quality
Management System, Measurement
Handbook, Release 3
Critical (Emergency duty contacted)
problems severely affect service,
capacity/traffic, billing, and maintenance
capabilities and require immediate
corrective action, regardless of time of
day or day of the week as viewed by a
customer upon discussion with the
collaborator.
Examples
Major problems cause conditions that

seriously affect system operation,
maintenance and administration, etc.
and require immediate attention as
viewed by the customer upon discussion
with the collaborator. The urgency is
less than in critical situations because of
a lesser immediate or impending effect
System restart, all links down

Simultaneous restarts of active
computer units
More than 50% traffic handling capacity
out of use
Subscriber related RNC functionality is
not working
RNC cannot be accessed or monitored
from NetAct
Single restart of computer units
Problems with back-up
Configuration changes (RNW, HW and
SW) are not working
Problems seriously affecting end user
service, but avoidable with workaround
solution
Capacity/quality related functionality is
5 (33)
the fault makes

it difficult for the
customer to
operate the
RNC.
on system performance, customers, and

the customers operation and revenue.
C-Minor
Minor fault not
affecting
operation or
service quality.
Other problems that a customer does

not view as critical or major are
considered minor. Minor problems do
not significantly impair the functioning of
the system and do not significantly affect
service to customers. These problems
are tolerable during system use.
Engineering complaints are classified as
minor unless otherwise negotiated
between the customer and supplier.
Table 1.
not working
Performance measurement or alarm
management is not working
Activation of a new feature fails
A single performance measurement is
not working completely
Configuration changes (RNW, HW and
SW) are not working completely
Subscriber related functions are not
working completely
Alarm management of objects (BTS,
functional units) is not working
completely
Capacity/quality related functionality is
not working completely
Major errors in documentation, for
example, an alarm or description is
missing from documentation
Vital documents are missing from the
documentation library
Failures not seriously affecting traffic
Errors in MML syntax
Minor errors in MML/Statistic output
Minor errors in documentation
NSNs Problem Classification
6 (33)
5. SOLUTION / INSTRUCTIONS
5.1 Using the log collection macro rnclogcol.exe
Most of the RNC configuration data and logs for the call cases can be collected with the
rnclogcol macro. Download the rnclogcol macro from NOLS and extract rnclogcol.zip file
into the PC folder named e.g. C:\Temp\Rnclog\.
1. Create IP address file with *.txt extension and save it to C:\temp\rnclogcol\ folder.
It is possible to define name for your Network Element in NE IP-address file. This is
optional, so you can still use your old IP-address files.
Format:
10.11.12.13 RNC_NAME #comment
This will add RNC_NAME into log directory name.
2. Run rnclogcol.exe on command prompt

Syntax
rnclogcol.exe -c <Configuration file *.ini> -i <IP address file
*.txt>
Example:
c:\temp\rnclogcol\rnclogcol.exe -c config_telecom_rn30.ini -i
rnc.txt
Directory name will contain the current PC time stamp. E.g.
10.11.12.13_logs_20080623_1543
Using new switch "-t" and parameter 0, it is possible to fall back to previous
functionality. This way time stamp will not be added to directory name
Example:
c:\temp\rnclogcol\rnclogcol.exe -c config_telecom_rn30.ini -i
rnc.txt t 0
Directory name would be e.g. 10.11.12.13_logs
Default configuration file names:

config_oam_rn22.ini
config_oam_rn30.ini
config_telecom_rn22.ini
config_telecom_rn30.ini
Cases:
RN2.2 O&M cases
RN3.0 O&M cases
RN2.2 call cases
RN3.0 call cases
7 (33)
If only few logs are need to be collected see the following chapters for more details and
collect logs manually. Rnclogcol *.ini files can be also edited to correspond the need of
required logs. Config *.ini is not configured by default to monitor different kind of call
scenarious. If logs are needed from call case activate message monitoring by changing
the get_monitorings line value from 0 to 1 and give monitoring time in seconds at *.ini
file.
8 (33)
5.2 Basic instructions for displaying logs and for monitoring messages
Displaying computer logs and system logs from Chorus units.
The chorus units are: MXU, A2SU, NIP1, NIS1, DMCU, and SFU
ZDDE:<unit type>,<unit index>:clog sa; chorus unit log
5.3 Commands for displaying computer logs and system logs from DMX units
The DMX units are: OMU, RSMU, RRMU, ICSU, and GTPU
ZDDE:<unit type>,<unit index>:ZGDC; computer log in short format
ZDDE:<unit type>,<unit index>:ZGSC; computer log in long format
ZDDE:<unit type>,<unit index>:ZSLE; system log of other system errors
ZDDE:<unit type>,<unit index>:ZSLP; system log of frozen programs
5.4 Commands for message monitoring in DMX units

Check how much memory can be reserved for monitoring buffer:
ZDDE:<unit type>,<unit index>:ZSB;
In the printout the latter number on the last line tells the amount of the free memory:
TOTAL
FREE
BYTE CNT
BYTE CNT
MEMORY USED FOR BUFFERS:
1C8E9000
08D7BEC0
You may RESERVE MAXIMUM HALF of the free memory for message monitoring
purposes. The bigger the reserved buffer is the longer it naturally takes to print out its
whole contents. It is lot faster if you can use LAN connection to print out the contents
of the buffer.
Example: If the FREE BYTE CNT in ZSB printout is 08D7BEC0 (hex), you may
reserve a buffer of size 08D7BEC0/2= 46BDF60 =74,178MB (ZOEBR)
Start monitoring for each unit:

1. ZDDS:OMU;
2. ZOEQ:<unit physical address>
Reserve buffer
3. ZOEBR:<unit physical address>:302CAAA
4. ZO
5 EC:<unit physical address>:<monitoring condition>
Start monitoring from each unit
6. ZOEM:<unit physical address>:
Stop monitoring from each unit
7. ZOES:<unit physical address>
9 (33)
Collect monitoring from each unit. Use this (ZOEGF) command if there is expected
heavy data from monitoring. After running this command, copy the result file from disk
on shadows directory with FTP. After copying the file delete it from disk to save disk
space.
8. ZOEGF:<unit physical address>:W0-/<unit physical address>.BIN
10 (33)
6. DATA TO BE COLLECTED IN ALL ABNORMAL SITUATIONS

The following data shall be collected in all abnormal fault situations:
6.1 SW and HW version information

Display the version of the SW build, and the version of all software modules that are
run in the computer unit, in which the problem exist. Collect this information with the
following MML commands:
ZWQO:CR;
ZWQV:<unit type>,<unit index>;
These commands are applicable to all DMX units (OMU, RSMU, RRMU, ICSU,
GTPU), as well as to all Chorus units (= other computer units than the ones listed
above).
In case of a Chorus unit, display also the version information of the components
included into the boot image with the MML command:
ZDDE:<unit
type>,<unit
/image/sys_bank/versions.txt;
index>:cat
Display the interchangeability information of the plug-in unit that is attached to the
computer unit, in which the problem exists. This information is needed at least in case
of faults, which have led to a spontaneous restart of a computer unit. It can be obtained
by first finding out the HMS address of the plug-in unit with the MML command
ZWFI:P::::<unit type>,<unit index>;
After this, enter the HMS address (displayed in the first row of the execution printout)
as parameters to the WFL MML command. As an example, if the HMS address
displayed in the execution printout of the WFI command is CHMS: 01 SHMS: 2 PPA:
03, then the command for finding out the interchangeability information of the plug-in
unit would be
ZWFL:P:1,2,3;
6.2 Alarm history

Display the alarm history so that it shows all alarm events from the time period, which
starts one hour before the occurrence of the problem situation, and ends one hour after
the problem situation was over. You should display the alarm history with the MML
command
ZAHP:::<start date>,<start time>,<end date>,<end time>;
For example, if the problem situation emerged on 2005-11-28 at 00:10 and ceased on
2005-11-28 at 03:20, enter the command
ZAHP:::2005-11-27,23-10-00,2005-11-28,04-20-00;
6.3 Working state of units

Display the state and info of the units configured into the system with the MML
command:
ZUSI:::FULL;
11 (33)
6.4 MML command log

Display the MML command log with the MML command
ZIGO:<start date>,,<end date>;
The start date should correspond to the day preceding the problem occurrence, and
the end date should be the day when the problem occurred. For example, if the
problem situation emerged on 2005-11-28 at 00:10 and ceased on 2005-11-28 at
03:20, enter the
ZIGO:2005-11-27,,2005-11-28;
Note that the userid should have enough access rights to output the MML command
log of all users. Also the parameters of ZIGO command should be such that all
information of all commands of all users is output.
12 (33)
7. CASE SPECIFIC INSTRUCTIONS

In addition to collecting the information mentioned in the chapter above, collect the
following information according to the problem case in question:
7.1 System restart

System restart history
If there was a system restart caused by the problem, display the system restart history
by sending the following message from the service terminal connected to the lower
service terminal connector in the front panel of the OMU. The message shall be sent
both in the working OMU and in the spare OMU.
ZOS:FF,*,3,,,,,6034,,,FC;
The lower service terminal connector shall be used because the output is displayed
only via it.
If displaying of the system restart information is supported in the SW build in question,
information on four latest system restarts is displayed on the service terminal in the
following format:
======================= Record number 0x0002
======================
Date and time : 2005-09-16
13:40:45.47
Object unit
: SFU-1
Working state : WO-EX
Recovery task : fault by alarm
Fault observer : 0000 0002 0000 00
Alarm(s)
: 1227 FFFF FFFF
Fault class
: fault_class_t_disturbance_c
======================= End of record number 0x0002
===============
On the basis of the date and time shown in the first data row you can identify the
information, which concerns the system restart that is of interest. The unit displayed as
the object unit is the one that is the origin of the system restart. Note that this
information may reside in the system restart history of the current spare OMU if there
was a switchover of OMU made in connection with the system restart.
Black box(es) of the computer units of interest (store to a file named like
omu_0_bbox.txt):
You should display the black box of both OMUs and the black box (DMX unit) / system
dump and logs (Chorus unit) of the object unit shown in the system restart history. If
the system restart history is not supported in the SW build in question, try to figure out
(e.g. on the basis of the alarm history) the object unit by your own and display the
black box / system dump & logs of that unit as instructed in the section Spontaneous
restart of a DMX unit or Spontaneous restart of a Chorus unit, respectively.
Note that there may be more than one system restart occurring in a row in a certain
fault situation. In that case you also should attach to the outage report all the stored
black box information of the DMX units of interest (i.e., of OMUs and of system restart
object units) from the W0-/var/crash/ directory on the disk. This directory contains
max. 12 latest formed black boxes for each DMX unit. The black box files of a certain
13 (33)
unit, as well as their control file, can be identified on the basis of the physical address
of the unit, which is used as a part of the name of these files. For example, as the
physical address of the OMU-1 is 0001 (obtained from the output of the USI MML
command), the back box files of the OMU-1 are named as 0001_00.bbox
0001_11.bbox, and the control file is named as 0001.seq.
7.2 Spontaneous restart of a DMX unit (OMU, RSMU, RRMU, ICSU, GTPU)
Black Box(es) of the restarted unit (store to a file named like omu_0_bbox.txt):
Display the black box with the following MML commands:
ZDDE:<unit type>,<unit index>:ZLP:1,BOX,Z1U,ZL:1;
ZDDE:<unit type>,<unit index>:ZLP:1,BOX,Z1U,ZL:1;
Note that there may be more than one unit restart occurring in a row in a certain fault
situation. In that case you should attach to the problem report all the stored black box
information of the DMX unit in question from the W0-/var/crash/ directory on the disk.
This directory contains max. 12 latest formed black boxes for each DMX unit. The
black box files of a certain unit, as well as their control file, can be identified on the
basis of the physical address of the unit, which is used as a part of the name of these
files. For example, as the physical address of the OMU-1 is 0001 (obtained from the
output of the USI MML command), the back box files of the OMU-1 are named as
0001_00.bbox 0001_11.bbox, and the control file is named as 0001.seq.
Printout of the MML command:
ZDVL::CONT:::::<unit type>,<unit index>;
For example: ZDVL::CONT:::::OMU,0;
7.3 Spontaneous restart of a CHORUS unit (MXU, A2SU, NIP1, NIS1, DMCU, SFU)
"Tip!
Sysdump and bblog data are stored in RAM memory of a Chorus unit. The data is lost if
the computer unit is powered down."
System dump of the restarted unit:
Display system dump and the stored computer logs and OS logs from the restarted
unit with the following commands
ZDDE:<unit type>,<unit index>:loadext sysdump;
ZDDE:<unit type>,<unit index>:sysdump W;
ZDDE:<unit type>,<unit index>:loadext bblog;
ZDDE:<unit type>,<unit index>:bblog c a <y>; where <y>=
210
14 (33)
ZDDE:<unit type>,<unit index>:bblog o <y>;

210
where <y>=
The parameter value y (where y = 210) in the commands above defines the
order number of the log. With the default parameter value 2, log data written the
previous unit restart is shown, and so on. Please make sure that you define the
value y correctly in order to display the logs that have been written just before
the unit restart(s) of interest.
Printout of the MML command:
ZDVL::CONT:::::<unit type>,<unit index>;
For example: ZDVL::CONT:::::DMPG,34;
15 (33)
8. RESTART OF DSP/DMPG
"Tip!
Sysdump and bblog data are stored in RAM memory of a Chorus unit. The data is lost if
the computer unit is powered down."
8.1 RNC active alarms and alarm history
RNC active alarm:
ZAAP;
RNC active alarm for one specific unit:
ZAAP:<unit type>,<unit index>;
RNC alarm history from start date (default one is current date):
ZAHP:::[start date]:;
RNC alarm history for one specific unit:
ZAHP:<unit type>,<unit index>;
8.2 Plug-in Unit HW information

ZWFL:P;
and
ZWDI;
8.3 All Function Unit states and information

ZUSI:::FULL;
8.4 DSP service statistics information
RN2.1
ZWPS:UNIT,DMCU,<unit index>;
RN2.2 and RN3.0

ZWPS:U:UNIT=<unit type>,INDEX=<unit index>:ALL;
8.5 Collect computer log in destination DMPG

ZDDE:<unit type>,<unit index>:clog sa;
8.6 Collect restarted DSP blackbox information

1.
Start remote session to the DMPG where the restarted DSP belongs to
16 (33)
ZDDS:DMPG,<unit index>;
2.
Load dspbb service terminal extension into DMPG ram disk

dxfload -n CSXDSPBB -d -u 0 -o /bin/dspbb.gz
3.
Dump DSP blackbox by specifying restarted dsp index (0 7)

dspbb c <dsp index>
8.7 Collect sysdump information for suspected faulty DMPG unit

1.
Start a remote session to the suspected DMPG

2.
Load sysdump service terminal extension into DMPG ram disk

loadext sysdump
3.
Print all system dump information from all available restarts (sysdump
can store at max ten latest restarts)
sysdump W
8.8 Collect bblog information of suspected faulty DMPG unit

1.
Start a remote session to the suspected DMPG

2.
Load bblog service terminal extension into DMPG ram disk

loadext bblog
3.
Dump computer logs where file id is between 2 to 10

bblog c a <file_id>
4.
Dump operating system logs where file id is between 2 to 10

bblog o <file_id>
8.9 DMPG communication processor blackbox information

1.
Start a remote session to the suspected faulty DMPG

2.
Load dprdump service terminal extension into DMPG ram disk
17 (33)
loadext -n CSXDPRDU /bin/dprdump.gz

3.
Dump all saved memory regions from the system dump, where nb
indicates the restart number 1 - 10.
dprdump d -4 r <nb>
8.10 Collect DSP service related information

1.

2.
Load rmdext service terminal extension into DMPG ram disk

loadext rmdext
3.
Get current DSP state and other information

rmdext d 1
4.
Get current DSP service related information

rmdext d 2
5.
Get current DSP service instance information

rmdext d 3
6.
Get current connection list

rmdext d 5
7.
Get current channel list

rmdext d 6
8.
Get RMDPRB event log

rmdext l
9.
Dump RMDPRB log under directory /DSP

cat /DSP/rmd_log.txt
10.
Dump DSP fatal error log under directory /DSP

cat /DSP/dsp_fatal_errors.txt
11.
Dump RMDPRB severe error log under directory /DSP

cat /DSP/rmd_serv.txt
18 (33)
8.11 Collect PDRACT channel related information

1.

2.
Dump all client information

pdrext cli
3.
Dump all connection status

pdrext con
4.
Dump detail information from specific connection

pdrext con <gcd index>
5.
Dump channel status

pdrext dsp <dsp index> <channel index>
8.12 Monitor DMX message in suspected faulty DMPG

1.

2.
Load dmxmon service terminal extension into DMPG ram disk

loadext dmxmon;
3.
Monitoring DMX message for specific family IDs

dmxmon [f fam0 fam1 fam9];
8.13 Monitor DSP message and DMX message to/from suspected faulty DSP
1.
to
Start a remote session to the DMPG which suspected faulty DSP belongs
2.
DSP
Monitor DSP message and DMX message in both directions for specific
p2dext monitor [p <dsp index>]
8.14 Check SW version of one specific program block

ZWQV:DMPG,<dmpg index>:<prb name>;
19 (33)
8.15 Collect diagnostics report history

ZUDH;
20 (33)
9. BTS INITIALIZATION CASE

Collect the following information in BTS initialization cases, for example if WBTS cells
remain in blocked state, or if there are problems in changing / editing parameters via
RNW object browser.
Message monitoring from OMU with families REK (050B), REZ (04FA), RAK (04EE), VEX
(04EF) and ENP (0553).
RN21 and RN22 OMU monitoring conditions:
SR:(OFAM=050B,04FA,04EE,04EF,0553)AND(NOT(NUM=0001,0002))
RN22 BTS initialization ICSU monitoring conditions and families: RRC (04FD), NBA
(04FE), and BRM (04FB).
SR:(OFAM=04FE,04FD,04FB)AND(OPRO<141)AND(NOT((CFAM=04FD)AND(C
PRO>140)))AND(NOT((NUM>9000)AND(NUM<9FFF)))AND(NOT(NUM0001,00
02,0A339,0A340,0C465,0C466,0D7A6,0D7A8,0D9DA,0D5B8,0C5DF,0D5B
A,08037,08038,0AB2E))
RN21 BTS initialization ICSU monitoring conditions:
SR:(OFAM=04FE,04FD,04FB)AND(OPRO<58)AND(NOT((NUM=0C465)AND(DA
TB0A=1B,07)))AND(NOT(NUM=0D7A6,0D7A8,0C5DF,0D5BA,0D9DA,0D5B8,
0D49A,0D49B))
Start the monitoring. Produce the fault. Stop the monitoring and print it out. Save log file
with name where units name and number are mentioned.
Other needed info from MXU unit:
Check with MML command ZRRI and ZLJI the affected AAL2 User plane info
(Interface, VPI and VCI)
Check the WO-EX RSMU: ZUSI:RSMU;
Printout RM2PRB error logs from WO-EX RSMU: ZDDE:<unit type>,<unit
index>:ZLP:3,LGU,Z3RIE:*;
Give to the command the affected AAL2 User plane info from WO-EX RSMU:
ZDDE:<unit type>,<unit index>:Z3IT:<interf no>,<vpi no>,<vci
no>;
Check from printout the A2SP number and in MML check which A2SU unit is handling
the A2SP: ZUSI:A2SU,::FULL;
21 (33)
A2SU
Unit
0
A2SPs
SUBRACK
0..3
4..7
8..11
12..15
1
(RNAC)
3
(RNAC)
4
(RNAC)
1
(RNBC)
16..19
2
(RNBC)
20..23
3
(RNBC)
24..27
4
(RNBC)
RN3.0:
After finding out A2SP handling the Iub AAL2 path take ddesk logs from affected
A2SP service terminal:
ZDDE:A2SP,<unit index>:loadext ddesk;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p aal2xc;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p ncid_all;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p ncid_hsdpa;
ZDDE:A2SP,<unit index>:ddesk p msg_counters;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p traffic;
ZDDE:A2SP,<unit index>:ddesk -a pq_alg_ncid p ncid;
22 (33)
Before RN3.0:
Check with MML command the WO-EX MXU that is under the same subrack as the
A2SP: ZUSI:MXU;
Take desk logs from affected MXU service terminal:
ZDDE:MXU,<unit index>:loadext desk;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p aal2xc;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p ncid_all;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p ncid_hsdpa; only
in RN 2.1 and RN2.2 level if HSDPA problems
ZDDE:MXU,<unit index>:desk p msg_counters;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p traffic;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p ncid;
23 (33)
10. CALL CASE

Collect the following information by macro if there are problems related to calls, for
example unsuccessful registrations, unsuccessful calls, unsuccessful handovers,
unacceptable KPI values, or call scenarios not working as expected.
Message monitoring from ICSUs with families NRM (04B7), RRC (04FD), NBA (04FE),
BRM (04FB), HA3 (507), IUR (508), IUV (509) and RBR (54E). IUR is needed in InterRNC handover cases only.
To reduce the size of the monitoring log, following monitoring condition can be used:
SR:(OFAM=4B7,4FD,4FE,4FB,507,508,509,54E)AND(NOT(NUM=0001,0002,0
D5BA,0C5DF,0D7A6,0D7A8,08037,08038,09904,09835,09836,D9DA,0CEC9,
0CECA))
Example RN2.1, RN2.2 calls monitoring conditions:
SR:((OFAM=04FD,509)AND(NOT(NUM=8037,8038,0D7A6,0D5BA,9904,983
5,9836,0AB2E,0D5B8)))OR(NUM=0C5DF,0E47B)
Start the monitoring. Produce the fault. Stop the monitoring and print it out. Save log file
with name where units name and number are mentioned.
Attach the Nethawk trace file(s) with configuration file included (*.gfc) or in text format
(*.txt) if they exist from the fault case. If it is used for Nethawk M5 analyzer take a ( *.rec )
log and Export Online Configuration ( *.wcf ) log.
Other needed info from MXU unit:
Check with MML command ZRRI and ZLJI the affected AAL2 User plane info
(Interface, VPI and VCI)
Check the WO-EX RSMU: ZUSI:RSMU;
Printout RM2PRB error logs from WO-EX RSMU: ZDDE:<unit type>,<unit
index>:ZLP:3,LGU,Z3RIE:*;
Give to the command the affected AAL2 User plane info from WO-EX RSMU:
ZDDE:<unit type>,<unit index>:Z3IT:<interf no>,<vpi no>,<vci
no>;
Check from printout the A2SP number and in MML check which A2SU unit is handling
the A2SP: ZUSI:A2SU,::FULL;
24 (33)
A2SU
Unit
0
A2SPs
SUBRACK
0..3
4..7
8..11
12..15
1
(RNAC)
3
(RNAC)
4
(RNAC)
1
(RNBC)
16..19
2
(RNBC)
20..23
3
(RNBC)
24..27
4
(RNBC)
RN3.0:
After finding out A2SP handling the Iub AAL2 path take ddesk logs from affected
A2SP service terminal:
ZDDE:A2SP,<unit index>:loadext ddesk;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p aal2xc;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p ncid_all;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p ncid_hsdpa;
ZDDE:A2SP,<unit index>:ddesk p msg_counters;
ZDDE:A2SP,<unit index>:ddesk a pq_alg_ncid p traffic;
ZDDE:A2SP,<unit index>:ddesk -a pq_alg_ncid p ncid;
25 (33)
Before RN3.0:
Check with MML command the WO-EX MXU that is under the same subrack as the
A2SP: ZUSI:MXU;
Take desk logs from affected MXU service terminal:
ZDDE:MXU,<unit index>:loadext desk;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p aal2xc;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p ncid_all;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p ncid_hsdpa; only
in RN 2.1 and RN2.2 level if HSDPA problems
ZDDE:MXU,<unit index>:desk p msg_counters;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p traffic;
ZDDE:MXU,<unit index>:desk a pq_alg_ncid p ncid;
26 (33)
11. SYNCHRONIZATION CASE

Collect additional logs in the following way if there is some problem with RNC
synchronization. Store the collected information to a file named e.g. as
synchronization_problem_case.txt.
SYPROC (3DF) message monitoring from WO-OMU when the problem exists (15
minutes period). If the problem is seen in MML command output, message monitoring
must be ongoing during MML command execution.
MML command ZDYI few times and also all other MML command outputs where the
problem is seen.
Following service terminal commands in WO-OMU: ZL:F; ZLP:F,FUT; ZFTN:A1L;
ZFTN:A2P; ZFTN:A2T; ZFTN:GS2;
Active alarms ZAAP and alarm history ZAHP
OMU computer logs ZGSC (both OMUs).
27 (33)
12. ESA 12/24 LAN SWITCH CASE

In case of problems related to ESA 12/24 LAN switch , collect the following information
and store them to a file named e.g. as ESA_problem_case.txt.
For ESA24:
1) Take a console or telnet connection to ESA24 LAN switch. When ESA24 card has
3.5.0 or older software, following commands can not perform. Those commands are
available since 3.14.7 software. More details in Technical Note 86.
2) Give following commands:
ESA24>enable
ESA24#show tech-support
ESA24#show log buffer
ESA24#show configuration-history
For ESA12:
Take a console or telnet connection to ESA12 LAN switch and display all possible
settings under different menus.
28 (33)
13. OMS CASE

13.1 General case
If the problem concerns OMS, collect the following logs and transfer logs to local PC.
Windows environment:
Log in to OMS as Nemuadmin user using puTTY tool. Change to the root user by entering
su command. Copy files first into the /home/Nemuadmin directory and then transfer files
to local PC by WinSCP tool as Nemuadmin user.
cp /var/log/master-syslog /home/Nemuadmin/
Linux environment:
Log in to OMS as Nemuadmin user. Transfer files to local PC with the following
command.
scp /var/log/master-syslog <user>@<local_pc_ip>:/<local_pc_directory>
Print out current OMS software package versions using currentdelivery command.
13.2 Measurement case

If the problem concerns measurement data processing, collect the following logs and
transfer logs to local PC.
Windows environment:
Log in to OMS as Nemuadmin user using puTTY tool, change to the root user by entering
su command. Copy files first into the /home/Nemuadmin directory and then transfer files
to local PC by WinSCP tool as Nemuadmin user.
cp /var/log/master-syslog /home/Nemuadmin/
cp /var/log/master-alarms /home/Nemuadmin/
cp /var/log/auth.log /home/Nemuadmin/
cp /var/opt/Nokia/OMS/OMSRNC/SS_RNCPM/BTSOperationLog.txt /home/Nemuadmin/
cp /var/opt/Nokia/OMS/OMSRNC/SS_RNCPM/performance.log /home/Nemuadmin/
Linux environment:
Log in to OMS as Nemuadmin user. Transfer files to local PC with the following
command.
scp /var/log/master-syslog <user>@<local_pc_ip>:/<local_pc_directory>
scp /var/log/master-alarms <user>@<local_pc_ip>:/<local_pc_directory>
scp /var/log/auth.log <user>@<local_pc_ip>:/<local_pc_directory>
scp /var/opt/Nokia/OMS/OMSRNC/SS_RNCPM/BTSOperationLog.txt \
<user>@<local_pc_ip>:/<local_pc_directory>
scp /var/opt/Nokia/OMS/OMSRNC/SS_RNCPM/performance.log \
<user>@<local_pc_ip>:/<local_pc_directory>
Print out current OMS software package versions using currentdelivery command
29 (33)
Logs from OMU:

ZIFO:OMU:GENE00:;
OMUs computer log
ZGSC;
Collect following files from OMU disk by FTP.
/RUNNING/ASWDIR/FTPSERVR.XML and additionally
/RUNNING/ASWDIR/FTPSERVR.OLD if that exists
14. NEMU CASE

In case of problems related to NEMU, collect the following information to log files and
store them to a zip named e.g. as NEMU_problem_case.zip.
Application and System logs from NEMUs Event Viewer. Open Event Viewer from
NEMU Start-> Programs -> Administrative tools -> Event viewer. Click Log and
check that application is selected and then save it *.evt format, and do the same for
system log.
If the problem occurs in the object browser then save trace.log, which can be usually
found from client computer C:\Program files\APPLAUNCHER_x.xx\. Where x-xx is
version of the applauncher. Trace file has to be taken when error occurs because file is
overwritten every time when application launcher is restarted.
Instructions to activate tracing in NEMU

Concerns after activating the NEMU tracing:
The trace logs files are like ring buffer. The maximum size of one trace log file is
defined in the registry:
[HKEY_LOCAL_MACHINE\SOFTWARE\Nokia\NEMU\InstalledModules\c_core\Nem
uLog\NemuLogCurrentVersion\Settings\TraceConfig\Any]
"dwLogSizeInKB"=dword:00002710 (In this example the maximum size is 10000KB)
The maximum size of trace log file must not be too small, because then there is a risk
that the problem situation wont be saved to the log. That is why in that case it has set
to 10MB.
The background processes, that are always active in the NEMU, have only one trace
log file per process. The user processes, which are started with e.g. Application
Launcher, write a new trace log file every time they are started.
If there have not been any problems, you can eventually clean up the trace log files
from the NEMU (e.g. once in a day or twice in a week), depending on how much the
NEMU is used which means how many time are opening AL and RNC Object
browser in a day. To be safe, clean the files on daily basis until you catch the
problem. You only need to clean up the trace log files of the processes, which are not
non-stop processes. You can identify these files as there are many files with same
beginning in the name.
30 (33)
e.g. EmiAccessInquirerSrv-1-1_trace.txt,
EmiAccessInquirerSrv-2-1_trace.txt,
emialarmservSrv-10-1_trace.txt,
emialarmservSrv-10-3_trace.txt, etc
The trace log file cannot be removed, if it is used by some process. So you cannot
remove all the trace log files when the NEMU PM is running. But cleaning up of trace
log files of the non-stop processes is not needed, because they cannot grow bigger
than defined in dwLogSizeInKB in the registry provided you have plenty of free disk
space in the NEMU.
The following steps are used to activate the tracing in NEMU:

1.
2.
3.
4.
5.
Stop PM from PMUI.

Make changes in registry file according to the values in the table.
Start -->RUN
Type: regedit
Select:
HKEY_LOCAL_MACHINE\SOFTWARE\Nokia\NEMU\InstalledModules\c_core\
NemuLog\NemuLog\CurrentVersion\Settings\TraceConfig\Any
Changes to be made:
dwFlags should be set to value 3 to turn ON tracing. If it is 0 then it means
NO tracing.
dwMode1 should be set to value 3FFFFFFF
dwLogSizeinKB should be set to some appropriate value like 0x00002710.
(The value set the log file size equal to 10000KB=10MB)
"dwFlags"
"dwLogSizeInKB"
"dwMode1"
Original Value
dword:00000000
(0 means that NO
LOGGING at all)
dword:000000c8
dword:38000012
Changed Value
dword:00000003
dword:00002710
dword:3FFFFFFF
Trace files are created into place indicated by szLogPath value.

(Default: E:\nemu\data_area\platform\active\c_core\NemuLog)
6. Start PM from PMUI.
7. After problem occurs, collect all trace files from the directory and send them
for analyzing. These tracings should be ON and at the time of occurrence of
problem then the tracing should be stopped.
The following are the steps for deactivating the tracing:
1. Stop PM from PMUI.
2. Make changes in registry file according to the original values in the table.
3. Start PM from PMUI.
31 (33)
15. MESSAGE MONITORING FROM OMU WITH FAMILIES FOR DIFFERENT

PROBLEM SITUATIONS
In all cases include following families: REK (050B), REZ (04FA), RAK (04EE) and
VEX (04EF): SR:(OFAM=50B,4FA, 4EE, 4EF)
RNW measurement collection: PMI (04ED): SR:(OFAM=4ED)
Transport & HW measurement collection: AMN (0391): SR:(OFAM=0391)
Data saving to disk service: SLY (04C7) and VID (0024): SR:(OFAM=4C7, 0024)
Data transfer communication with NEMU (via EXW): SLY (04C7): SR:(OFAM=4C7)
EMT-interface implementation: EXW (0433): SR:(OFAM=433)
Start the monitoring. Produce the fault. Stop the monitoring and print it out. Save log
file with name where units name and number are mentioned.
If the fault has something to do with COCO, message monitoring from OMU with
families MNM (03C6), ARM (02F7): SR:(OFAM=3C6, 2F7)
Start the monitoring. Produce the fault. Stop the monitoring and print it out. Save log
file with name where the units name and number are mentioned.
16. NOTE
17. REFERENCES
32 (33)
Disclaimer
The information in this document is subject to change without notice and describes only the
product defined in the introduction of this documentation. This documentation is intended for the
use of Nokia Siemens Networks customers only for the purposes of the agreement under which
the document is submitted, and no part of it may be used, reproduced, modified or transmitted in
any form or means without the prior written permission of Nokia Siemens Networks. The
documentation has been prepared to be used by professional and properly trained personnel,
and the customer assumes full responsibility when using it. Nokia Siemens Networks welcomes
customer comments as part of the process of continuous development and improvement of the
documentation.
The information or statements given in this documentation concerning the suitability, capacity, or
performance of the mentioned hardware or software products are given as is and all liability
arising in connection with such hardware or software products shall be defined conclusively and
finally in a separate agreement between Nokia Siemens Networks and the customer. However,
Nokia Siemens Networks has made all reasonable efforts to ensure that the instructions
contained in the document are adequate and free of material errors and omissions. Nokia
Siemens Networks will, if deemed necessary by Nokia Siemens Networks, explain issues which
may not be covered by the document.
Nokia Siemens Networks will correct errors in this documentation as soon as possible. IN NO
EVENT WILL NOKIA SIEMENS NETWORKS BE LIABLE FOR ERRORS IN THIS
DOCUMENTATION OR FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL,
DIRECT, INDIRECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT
NOT LIMITED TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS
OPPORTUNITY OR DATA, THAT MAY ARISE FROM THE USE OF THIS DOCUMENT OR
THE INFORMATION IN IT.
This documentation and the product it describes are considered protected by copyrights and
other intellectual property rights according to the applicable laws.
The wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark
of Nokia Corporation. Siemens is a registered trademark of Siemens AG.
Other product names mentioned in this document may be trademarks of their respective
owners, and they are mentioned for identification purposes only.
Copyright Nokia Siemens Networks 2008. All rights reserved.
33 (33)

TS RNC SW 038 I14

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

TS RNC SW 038 I14

Загружено:

Авторское право:

Доступные форматы

Technical Support Note

This document contains following

Nokia Siemens Networks

Compatibility / Dependencies to other products....................................................................... 4

Nokia Siemens Networks

Nokia Siemens Networks

A new template and OMS log instructions

1. COMPATIBILITY / DEPENDENCIES TO OTHER PRODUCTS

Nokia Siemens Networks

Nokia Siemens Networks

Definition of problem report severity

Major problems cause conditions that

System restart, all links down

the fault makes

on system performance, customers, and

Other problems that a customer does

Nokia Siemens Networks

NSNs Problem Classification

2. Run rnclogcol.exe on command prompt

Default configuration file names:

Nokia Siemens Networks

Nokia Siemens Networks

5.4 Commands for message monitoring in DMX units

Start monitoring for each unit:

Nokia Siemens Networks

Nokia Siemens Networks

6. DATA TO BE COLLECTED IN ALL ABNORMAL SITUATIONS

6.1 SW and HW version information

6.2 Alarm history

6.3 Working state of units

Nokia Siemens Networks

6.4 MML command log

Nokia Siemens Networks

7. CASE SPECIFIC INSTRUCTIONS

7.1 System restart

Nokia Siemens Networks

Nokia Siemens Networks

ZDDE:<unit type>,<unit index>:bblog o <y>;

Nokia Siemens Networks

8.1 RNC active alarms and alarm history

RNC active alarm:

RNC active alarm for one specific unit:

ZAAP:<unit type>,<unit index>;

RNC alarm history for one specific unit:

ZAHP:<unit type>,<unit index>;

8.2 Plug-in Unit HW information

8.3 All Function Unit states and information

8.4 DSP service statistics information

RN2.2 and RN3.0

8.5 Collect computer log in destination DMPG

8.6 Collect restarted DSP blackbox information

Nokia Siemens Networks

Load dspbb service terminal extension into DMPG ram disk

Dump DSP blackbox by specifying restarted dsp index (0 7)

8.7 Collect sysdump information for suspected faulty DMPG unit

Start a remote session to the suspected DMPG

Load sysdump service terminal extension into DMPG ram disk

8.8 Collect bblog information of suspected faulty DMPG unit

Start a remote session to the suspected DMPG

Load bblog service terminal extension into DMPG ram disk

Dump computer logs where file id is between 2 to 10

Dump operating system logs where file id is between 2 to 10

8.9 DMPG communication processor blackbox information