Академический Документы
Профессиональный Документы
Культура Документы
Fault Instance
Special Document
How to Expand MSC Single Module to Multi-module........................................................................18
July 2008 Issue 117 Fault Instance
Maintenance Experience
www.zte.com.cn
use mopdb
select * from mop_msccpudata
use mopdb
Figure 1. Abnormal VLR Numbers
select * from mop_msccpudatax
where sdate<'2007-09-05 09:30' and
Analysis and Solution sdate>'2007-09-05 07:00'
It was doubted that the abnormal use mopdb
number of the VLR subscribers had select * from mop_msccpudataxx
a relationship with the loss of the where sdate<'2007-09-05 09:30' and
performance data. When checking the sdate>'2007-09-05 07:00'
serrun.log file on 129, the engineer found use mopdb
that the performance statistics process select * from mop_msctrafdatax
restarted continuously between 5:44 and where sdate<'2007-09-05 09:30' and
6:40, and the last restart time was about sdate>'2007-09-05 05:00' and ntype=0
9:10.
use mopdb
Extract the log according to the
select * from mop_msctrafdataxx
following methods:
where sdate<'2007-09-05 09:30' and
1. Modify ShowAllLog=1 in the file
sdate>'2007-09-05 05:00' and ntype=0
mop_pack.ini under the directory zxg10 of
Maintenance Experience
www.zte.com.cn
The analysis result was as follows: Aiming at the BDE abnormality, the
The foreground buffer could only store 2000 engineer analyzed the operating system
packets. The buffer of the foreground was full, log of the 129 server, located the patch
so the data between 7:35 am and 9:10 am were of SQL Server as SP3 and found that the
lost. After 9:10, the server began to receive the SP4 patch was not installed. After installing
data from the foreground, so these data were the patch SP4 patch program, the problem
recorded. was solved.
Maintenance Experience
www.zte.com.cn
outgoing format of the 0003 in the MT number analysis 3. Subscribe the OSB service for the subscriber in
selector according to the normal flow if the calling the HLR agent, as shown in Figure 3.
subscriber did not subscribe the OSB service. So, it was
required to configure the prefixes 03, 05, 07 and 09 in
the international analyzer entry and specify them to the
corresponding outgoing route links.
Solution
1. Under the Configuration Management >
Supported Service Option Configuration of HLR, check
the checkbox Support OSB Service, as shown in Figure 1.
Maintenance Experience
www.zte.com.cn
10 Maintenance Experience
www.zte.com.cn
00920355XXXX.
ii. The ServedMSISDN number of MOC 1. I t i s a l l o w e d t o c l i c k t h e b u t t o n
was 920355XXXX. The number attribute was 1 Directories to set the path of the original
(international). path. The directory of both QueryResults and
3. When performing the query and analyzer Trace cannot be set through the interface.
tool of the original bill, the engineer did not find any QueryResults is used to save the query result.
TRACE result under the corresponding directory. Trace is used to save the result of the trace bill.
The format of the number in the original bill was These two directories only can be set through
correct, which meant that the foreground module modifying the configuration file obcdrman.ini.
did not have the number problem. The incorrect 2. Save the query result to the directory
codec program of the background resulted in the QueryResults. The file name includes the date
incorrect format of the calling number in the bill. and the time.
Symptom
The client reported that the call failed after a
long time waiting when the MSC was busy.
12 Maintenance Experience
www.zte.com.cn
Experience Summary
From the signaling tracing, it was known that
the wireless side did not give response to the
PAGING message sent by MSC, which resulted
in the call failure. The failure reason needed to
be checked by the wireless side and it had no
relationship with the MSC side.
In additional, the time interval between the first Figure 6. mcaPAGING_M Message(2)
paging message and the second paging message
was three seconds in the old version of MSC. A MSC V3.04.12.P21.B1, and it could be used to
new timer T169 was added in the new version modify the time interval of the paging.
Symptom
When querying the bill of a whole day from the
charging management system on the charging
client in batches, the carrier found that the capacity
of the bill query system was restricted (the default
value was 5M), as shown in Figure 1. At this
time, the bill of a whole day could only be queried
section by section according to the time. The
carrier required to expand the upper limit capacity
Figure 1. Warning
of the bill.
14 Maintenance Experience
www.zte.com.cn
exchange the communication port 1 and the 4 with that of module 3, and check whether
communication port 2 of the adjacent office any error code appears (within fifteen
connected to module 3 and module 4, the problem minutes or more) with the tool Moniproj.
still existed. When replacing the MPMP boards for exe.
module 1 and module 4 with that of module 3, the In the end, the engineer found that
problem still existed. All the above meant that the two DDSN boards on module 4 resulted
problem had no relationship with module 1. When in the alarm. If these two network boards
checking the connection of the cable, the engineer worked as the active board, the alarm
also did not find the problem. would generate on the corresponding
In the physical configuration, when adjusting module. The error code received from the
the HW time delay on module 4, the engineer MPMP board of the module could be seen
found that the problem still existed. Note: it was through the tool Moniproj.exe. When these
only required to adjust the HW occupied by the two network boards worked as the standby
related board, the 4~19 HW and the 68~83 HW board, this alarm would not appear.
could be adjusted. It was not required to adjust the The problem could be solved after the
HW corresponding to the DT; otherwise, the DT fault network board was replaced.
board would not be started.
The DDSN/DSNI/FBI board also had a Note: When replacing the network
relationship with the communication of the board /SDNI/FBI at night, maybe E1
Modules. module 4 was a new expanded module, will lose synchronization, the link will
so it was doubted that there was something wrong be broken, which will further have an
with the board on module 4. influence on the service.
Replace the DDSN/DSNI/FBI boards on module
16 Maintenance Experience
www.zte.com.cn
2 Preparations before
Reconstruction
2.1 Checking running status before the
reconstruction
(1) MSC/VLR system check
1) Check the historical alarm records to see
whether there are some serious faults in the recent
two weeks.
Figure 1. Architecture Instance 2) Check the performance statistics to see
18 Maintenance Experience
www.zte.com.cn
whether the traffic statistics in the recent week is ii. For the original module 2, put
normal. the reconstructed FBI board and the
3) Check whether the parameter setting of the COMM (MP level) board to the slot of the
SQL Server on the O&M server and the charging corresponding shelf, but do not insert them
server meets the requirement. (that is to say, do not power on them).
4) Check the running status of the charging Connect the fiber to the correct location
server. Check whether the charging bills of the and label it clearly.
current day received from the foreground and iii. During the reconstruction, the new
sent to the charging center are normal. Check the HUB should be close to the old HUB
file oblog.log and the file obaecur.log under the in order to facilitate the network cable
directory C:\ZXG10\TRACE on the charging server modification. After the reconstruction,
131 and 132 to see whether they have some the adjustment operation is easy and the
abnormalities. processing flow meets the requirement.
5) Check the running log of each module MP 3) Prepare the clock plan
from the foreground to see whether there are some For the clock adjustment, it is required
abnormal running records in the recent two weeks. to prepare the plan before the expansion
If the device has no essential problems, it is and the reconstruction, such as, the
allowed to arrange the reconstruction. Otherwise, it board adjustment, the cable layout. If this
is required to handle the hidden troubles before the operation is neglected, the expansion and
reconstruction. the reconstruction time will be delayed, the
(2) Check the hardware of the reconstructed clock adjustment will fail or the expansion
device and the cutover will fail.
1) Check hardware device i. For the BITS clock used by the
First, confirm whether the hardware required original single module, it is required to
by the reconstruction is completely prepared. The adjust the CKI board from the original
hardware includes: all boards of the new rack, the single module to the center module during
added FBI board of the old rack, COMM board, the expansion and the cutover, and it
the optic fiber used to connect the center rack and is also required to lay the cables to the
the original single module, the HUB for debugging, center module in advance. At this time, it
one PC for debugging, the version software of the is required to extract the clock from BITS.
current network and the software system to be It should be noted that the imported BITS
installed. clock should have 2-channel input at
On the site, if the MP model is inconsistent (for least. It is allowed to select 2Mb and 2Mhz
example, the original 2# module is PII MP, but the according to the details of the carrier. It
new module is PIII MP), it is required to unify the is required to pay more attention to the
MP model to ensure that the MP model of each impedance match of CKI board. The
module is consistent. unmatched impedance will degrade the
2) Check hardware connection configuration BITS clock of the system and will further
i. For the fiber connection between the affect the steady running of the system.
periphery module and the center module, the fiber ii. For the E8k clock used by the
from each periphery module to the center module original single module, the clock
should be on a pair of CFBI board separately. of the original module 2 is the E8k
clock extracted from E1 by the SYCK (save it as the text file). You can check the file
board. During the expansion and the through the tool OBDBTOOL.
reconstruction, it is required to perform In the basic configuration management system,
the following adjustment in that evening. back up the configuration data (note: adopts the
The adjustment steps are as follows: the BCP method and the SQL normal method).
SYCK board of the center module (the Record the value of the variables in the variable
new module 2) extracts the clock from E1, management system (you can save all the values
and then module 2 distributes the clock or the changed values, and check whether the
through the CFBI board, the FBI board of value in the generated file is consistent with that in
each MPM module and the SYCK board of the variable management system).
each MPM. Record the parameters of the timer. You can
4) MSC system backup save all the values or the changed values.
Back up the original ZXG10 directory Record the data configuration of the link and
of the O&M server to the disk D. the trunk group from MSC system to each office
Back up the directory C:\ZXG10 of direction.
the active/standby charging server to It is required to install the 129 server again
the directory D:\ of the active /standby because of the capacity expansion. At this time,
charging server. the engineer should check whether there is
Back up the important database on the the tone script file msctone.sql used during the
129 server commissioning on the site.
Basic configuration database If the performance statistic has the congestion
ZDB_ZXJ10 setting, it is required to record the statistics
Performance statistics database items that need to be enabled or disabled by the
MOPDB performance statistics. The engineer should pay
Alarm database MOAMSSDB more attention to the collection items that need to
Back up the charging database be enabled by the carrier, such as, each circuit in
MOBMSCVLRDB and record the the circuit group statistics and the office direction in
parameter values in the table SETTING the office direction data, as shown in Figure 2.
Record the setting of the stream control
(execute c:\zxg10\loadctl.exe on OMM server).
20 Maintenance Experience
www.zte.com.cn
be the same as that of the current 129 server. making the MP:
Install the version of the current office. If there is no After the MP is made, put a clear label
new PC and no other PC to replace the temporary on MP in order to facilitate the later
129 server, the MSC O&M agent could be used. debugging.
2) Network connection. The running mode of MP meets the
Module 1, module 2, module 4 of the requirement.
foreground and the new background 129 server 4) Configuring data on the new O&M
comprise a new MSC through a new HUB. Both server.
the network cables and the HUB should be close to The engineer could configure all data
the rack of the server in order to facilitate the wiring again or restore the data to the new O&M
after the normal reconstruction. server and then modify the corresponding
3) Make the MP for module 2, module 2, data. Perform the following steps to modify
module 3 and module 4. the restored data.
During the real reconstruction, module 1, Reserve the exchange configuration,
module 2 and module 4 of the foreground are new, the number analysis, the mobile
and module 3 is reconstructed from the original number analysis, SCCP configuration
module 2. The configuration file omc.cfg under the and LAI configuration.
directory CONFIG used by module 1, module 2 and Delete all trunk configurations and the
module 4 should be consistent with that in module 2. MTP configuration. It is required to
But, the option Version= in the file Version.cfg is record all original configurations before
configured according to the module. the deletion.
Module 1: Version=MSC&&VLR Delete all physical configurations
Module 2: Version=MSC and then perform the corresponding
Module 4: Version=MSC configuration according to the board
Whether the file tcpip.cfg adopts the Ethernet to slot of the real rack, such as slot 1, slot
communicate depends on the setting of the site. 2, slot 3 and slot 4.
The version of the MP under the directory Because a center module and a
VERSION of each new module is the same as that peripheral module are added, so the
of the original module 2. In general, the PP version attribute of the original module 2 is
of module 4 is the same as that of the original changed. At this time, it is required to
module 2, but it is also required to check whether delete the original physical configuration
the PP boards of these two modules are the same. and perform the physical configuration
It is required to check the PP version of module 2 again. It is required to delete all trunk
according to the real board. circuits and the signaling links in all
Use one MP of the module 4 as one MP of configurations before deleting the physical
module 3. During the reconstruction, the single configuration; otherwise, it is not allowed
MP of both module 3 and module 4 run normally. to delete the physical configuration.
After the successful reconstruction, use one MP of Configure the physical data again
module 2 as the standby MP of module 3, and use and connect the networking. During the
the other MP of module 2 as the standby MP of communication configuration among the
module 4. modules, the number of the communication
The following must be paid attention to during module used by the peripheral module is
2. The HW time slot on the board of the new After the debugging, connect the fiber of both
module 3 should be consistent with that of the original module 2 and the new module 2. At this
the current network. time, it is not required to power on the FBI board
Configure the capacity of the office again. on the original module 2. Connect the fiber of both
Configure the capacity for VLR again module 4 and the new module 2.
(apply for the password to modify the 6) Tone adjustment.
capacity of VLR), and put VLR on The temporary 129 server is new, so it is
module 1. required to re-load and adjust the tone script
Configuring the signaling link and the msctone.sql. After performing this script in the
trunk data again. script analyzer, it is required to check whether the
This configuration should be consistent service tone edit item and each KB edit item are
with that of module 2. That is to say, change consistent with that in the tone table maintenance
the number of the module from 2 to 3, but service of the original 129 server; otherwise, the
the other data should be consistent. tone play after the cutover will fail.
In the basic data configuration, the In the tone table management, it is also
number of the module that belongs to required to re-select or add the service tone type
the BSC configuration is 2. After the for the tone board unit; otherwise, the tone cannot
adjustment, it is required to modify the be played after the cutover.
number as 3. 7) Modify the security variables and the timer
5) Debugging. according to the backup security variables and
It is required to debug the data after the the timer. After the modification, it is required to
data configuration. Use the three racks of synchronize them with the foreground (When using
module 1, module 2 and module 3 to perform one MP of module 4 as one MP of the module 3, it
the debugging. is also required to synchronize the security variable
Put the MP of module 3 and module 4 and the timer on the rack 4).
on module 4. At the same time, adjust the 8) Adjust the performance statistics items
location and the fiber connection of the and the alarm filtering items on the temporary 129
corresponding CFBI board. Adjust other server and keep them consistent with the original
boards according to the conditions on the 129 server.
site. Try to simulate the environment of 9) Adjust the stream control parameters and
module 3 and module 4 according to the keep them consistent with that on the original 129
rack of module 4. Synchronize the data of all server (with the tool loadctl.exe).
tables and perform the test.
Use the tool to check whether the 3 Reconstruction process
resource in each module or among the 3.1 Reconstruction steps
modules is normal. Use the tool MONIPROJ 1) Generate the bill from the foreground by
and the diagnostic and test program to manual, and start the 130 charging program from
check whether the communication among dual servers (viz, power off the charging process of
module 1, module 2, module 3 and module 4 the background).
is normal. In the fault management system, 2) Power off the active/standby MP of the
observe whether the board of module 1, original module 2, and unplug the active/standby
module 2, module 3 and module 4 is normal. MP. Replace them with the debugged module 3.
22 Maintenance Experience
www.zte.com.cn
the tables again. for the original 129 server again. It is also required
7) R e s y n c h r o n i z e t h e s e c u r i t y to create the performance statistic database again
variables. because of the change of the module number.
8) Perform the dialing test and record Otherwise, the database will run abnormally. It is
the result. only required to reserve the original databases
9) Restore the file omccfg.ini under for other systems instead of creating them again.
the directory C:\ZXG10 of the original After the installation, restart the system and
active/standby charging server. Start the check whether each system runs normally (check
process of the 130 charging server from whether the tone Flash file used on the site exists
the resource of the dual servers and check under the directory C:\ZXG10). Adjust the value of
the generation of the bill. the performance statistics database as the original
value, create the CSM module, and open the
4 Adjustment after cutover collection items and the congestion parameters of
After the formal reconstruction, it is the performance statistics.
required to observe the system for one Back up the data on the new 129 server as
week in order to check whether the data the SQL file and restore the data on the original
configuration is correct and whether the 129 server. The original 129 stream control
device runs normally. If the device runs parameters (loadctl.exe) should only includes the
steadily, cooperate with the carrier to configuration of module 2. It is required to delete
perform the following adjustment in order the configuration of this module and then re-
to complete this expansion reconstruction. configure the stream control parameters of the
Adjust the ON/OFF switch on the periphery module 3 and module 4.
reserved MP of the original module 2 and After verifying the configuration data, unplug
use the MP as the MP of module 4. the network cable from the HUB of the new 129
Delete the data under the directory server. Connect the network cable of the original
\DATA of MP, delete the file TIMER.CFG 129 server to the current HUB.
under the directory \CONFIG, and modify It is required to adjust the traffic load of the
the setting of the file \CONFIG\VERSION. outgoing trunk and the signaling link among
CFG. Synchronize all the tables and re- each module according to the planning and the
synchronize them after the system restart. requirement of the carrier. After the single module
Test the active/standby switching after the is expanded to multi-module of the center rack,
active/standby MP runs normally. all the traffic is also on the module 3 (the original
Set the other MP of the original module module 2). At this time, it is required to allocate
2 as the standby MP of module 3. the traffic of module 3 to module 3 and module 4
Back up some important databases regularly.
of the original 129 server, such as the
performance statistics database, the alarm
Note: The trunk wire of the backplane on
database, and the authority database. Back
module 3 should be long enough in order to be
up all files under the directory C:\ZXG10.
connected to module 3 or module 4. Try to distribute
When the original 129 network card
the signaling link and the traffic of each office to
performs the self-loop operation, it is
module 3 and module 4 regularly.
required to install the background software
24 Maintenance Experience
www.zte.com.cn