ZXG10 iBSC-High OMP Load Analysis

Product

ZXG10 iBSC/GUBSC/GU Wireless

System/GU

Network
Management
Version

Equipment
Hardware
Version

iBSCV6.20.614fP004

Equipment
Software
Version

iOMCRV6.20.614dP002OMCBV4.00.200mP02

Board
Hardware
Version

Board
Software
Version

Incident Error
Code

Page Views

32

Knowledge
No.

WHD20111221432736

Author

CaoFengJun

Incident Description (Incident Phenomena)

On June 22, the status of the OMP board became abnormal after the version upgrade of Marol iBSC for
XX Project was completed. Although the iBSC version was rolled back at 4:00am, the problem remained.
Networking Environment

Problem Cause Analysis

According to the analysis of the data reported from the engineers on site, during the data transmission
from the OMP board to GIPB board, Site1BTS2 cannot be found in the R_CELL table, as shown below:

82120 14:29:00:165 [LVL2]

File=x:/ibsc_app/Dbs/dbi/source/DBS_DBI_DMAccess.c,Line=2296,DbFace:DM
_GETFRCELLROUTE], SiteId=1, BtsId=2 Can't find in R_CELL!
The processing flow of DM_GETFRCELLROUTE for PS service that adopts E1 on the Gb interface is
as follows: The system first traverses the R_BVC table on the OMP; Finds out a BVC; Searches for the
corresponding record of the BVC in the R_CELL table by the SITEID and BTSID; Confirm the
corresponding UPUDSP according to the cell information; Acquire the IP address of the corresponding
interface through querying the R_SUNIT table. The flow of downloading GIPB data to ZXG10-iBSC is
shown below:
In the R_BVC table, there is a piece of record of Site1BTS2, which does not exist in the R_CELL table
and the R_BTS table. In this case, when the OMP transmits data and queries Site1BTS2 in the R_CELL
table according to the record in the R_BVC table, it finds Site1BTS2 does not exist, and then it returns a
failure message to the GIPB board. When the GIPB board receives the message from the database and
finds the data invalid, it applies for data from the database once again; however, due to the incorrect
data, the application for data from the database fails again and again, which ends up in a vicious circle.

Due to the existence of the illegal data, during the repeated applications of data from the GIPB board,
the processing time of DM_GETFRCELLROUTE is extended, which causes the CPU load of the OMP
board to reach a high level and the GIPI board fails to be booted.
After the problem is replicated in the lab, the comparison between the two (without service) is:
In case of 50 cells: the processing time of DM_GETFRCELLROUTE is around 15ms, and the CPU load
of the OMP board is around 17%;

In case of 450 cells: the processing time of DM_GETFRCELLROUTE is around 150ms, and the
CPU load of the OMP board is around 75%;
Solution

1.check hardware version, normal.

2.put out GIPB card, the issue clear, the issue happen again when put in the GIPB card.
3.the issue continue after changeover the CHUB card.
5.print the OMP log find that
"82120 14:29:00:165 [LVL2]
File=x:/ibsc_app/Dbs/dbi/source/DBS_DBI_DMAccess.c,Line=2296,DbFace:DM_GETFRCELLROUT
E], SiteId=1, BtsId=2 Can't find in R_CELL!"
6.check ZDB, find a wrong data there.
7. use database script to clear the issue date, then the issue clear.

Summary and Notes

When it fails in acquiring the data, GIPB immediately sends the application for data again. The
repeated application causes the OMP load to reach a high level. According to the cause of the
problem, the following measures can be taken to avoid the problem:
Adjusting the data acquisition principle:
When GIPB fails to acquire the necessary data, it should not apply for the data again
immediately, but try it after a delay of time, in order to avoid causing the OMP board to process a
large number of data applications in the same time.

Knowledge Evaluation