Internal Use Only

DO Trouble-Shooting and case analyse

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Contents

Common Types of EVDO Service Faults Methods of EVDO Troubleshooting Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Contents

Common Types of EVDO Service Faults Methods of EVDO Troubleshooting Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Common Types of EVDO Service Faults

The following are common types of EVDO service faults:

Access failure: Fail to set up a connection. Performance problem of data transmission: The transmission rate fails to reach a reasonable level for data uploading or downloading. Call drop: The connection is interrupted (omitted)

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Contents

Common Types of EVDO Service Faults Methods of EVDO Troubleshooting

Collecting and Feeding back Fault Information Access failure Data transmission performance fault Call drop

Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Collecting and Feeding back Fault Information

Normally the following information should be collected and then feed back so as to facilitate the troubleshooting process:

Fault phenomena; (Customer/User complain) Alarms and notification; (OMC) OMC operation logs; (OMC) Performance indexes; (OMC/CNO2) Affected scope; (Customer/OMC/CNO2) Time of occurrence; (OMC/CNO2) Fault cause statistics; (CNO2) Signaling trace; (OMC/Test) Service Observation; (OMC) Running versions; (OMC) Networking topology of packet domain; (Customer) Abis interface transmission infomation; (Customer/OMC)

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Contents

Common Types of EVDO Service Faults Methods of EVDO Troubleshooting

Collecting and Providing Fault Information Access failure Data transmission performance fault Call drop

Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

Access failure :

Faults occur during the access phase, and lead to connection failures. What are the possible causes for Access failures?

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

Common Causes of Access Failures

Link failure of R-P interface; (inter-connection parameter, route, physical links) Link failure of A12 interface; (inter-connection parameter, route, physical links) Incomplete configuration data on BSC side; (IPCF, UIM, Resource) Internal interference; (PN, neighbor cell) External radio interference; Configuration error of wireless parameter; (Cell radius, Power, Frequency) Hardware fault of BTS; (RF, clock system) Terminal problem; (PRL, account) Internal fault of BSC; (Packet lost) Wrong running version;

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

The following fault information of an access failure should better also be reported:

Configuration of R-P interface inter-connection parameters Configuration of A12 interface inter-connection parameters Configuration of BTS and cells wireless parameters Print information of terminals PPP-Logs

2007, ZTE Corporation. All rights reserved.

Access fault declare

Internal Use Only

Access failure

Comfirm fault phenomena and affect scope

Trouble shooting flow(1)

Failure causes analyseing N Resource overload? Y

System fault?

Lawless users?

Check alarms and notifications

Reply to the Customer

Correlational alarms or notifications?

Time of Alarm occured

Other correlational NE fault?

Turn to the broken NE

System changed?

Service recovered after other NE recovered?

Y Comfirm whether the change lead to the fault?

Roll back the change

Solve the alarms

Fault recovered?

Confirm the motive of the changes

End

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

Trouble shooting flow(2)

A

Signal trace of complain users or test user

Messages in UM are reported? N

a
Turn ot UM messages report flow

Messages in A12 are reported? N

b
Turn to A12 messages report flow

Pass Authentication of A12 N

c
Turn to A12 authentication flow

Pass A11 register? N

d
Turn to A11 register flow

Success to set up the PPP connection N

e
Turn to PPP connection flow End

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

UM messages report flow

N

a
Are other users in the site/sector normal? Turn to the terminal productor

Are the fault users in the other sector of the same site normal? N Is the fault user normal in the other sites?

Check the radio para, running version, RSSI, Neighbor list; Change over RFS boards, feeders, antennas, link between BDS and RFS Check the radio para, ratio power, and running version, link between BDS and RFS, RSSI and neighbor list Is it normal by checking on OMC? Clear the abnormal item

Is it a global fault? Y Check BSC clock, media stream, running version, ABIS boards and transmission statue

Confirm the commonness of the sites (ABIS, geographical distributing, RSSI and radio para.

Test on site and collect the CNT LOG, to confirm the radio environment

Are messages in Um normal?

Confirm whether the 1X service is normal, and turn to BSC fault handling flow

Confirm whether the 1X service is normal, and turn to BTS fault handling flow

return

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

A12 messages report flow

N Are other users in the site/sector normal? Y Is the fault user normal in the other sites/sector?

Are other users in the BSC normal?

Compare the signal trace between fault user and normal ones.

Clear the session of the user, and try again.

Check the A12 configuration inter-connect para, IPCF, MDM, IPCF port statue Are the configuration or the port statue normal? Y Turn to BSC fault handling flow

Turn to BTS fault handling flow N Is A12 request report?

Y N

Is it recovered after correct the configuration?

2007, ZTE Corporation. All rights reserved.

Internal Use Only

N has AN-AAA received the A12AccessRequ est messages

Access failure

A12 authentication flow

Are there AbrCHAPAuthI nd messages

Check the inter-connection para of A12; Check the route between AN-AAA and PCF. Correct the error, and check whether ANAAA has received the A12AccessRequ est? Trace the packet of A12, to confirm whether BSC, ANAAA or transmission problem.

Check the route between IPCF and AN-AAA; Check the inter-connection para of A12; Ask AN-AAA to check its configuration.

Has AN-AAA replied to PCF?

Turn to AN-AAA to handle

Is there AbrCHAPAuthI nd

Compare to the table description of AbrCHAPAuthInd reason codes, to check it N Pass A12 auth?

Trace the packet of A12, to confirm whether BSC, ANAAA or transmission problem.

Return

2007, ZTE Corporation. All rights reserved.

Internal Use Only

N Check the config of R-P interface. Check whether PDSN is normal

Access failure

A11 register flow

Y Receive more than one A11RegReply messages? N

Has A11RegRequest been sent Y

Has reA11RegReply heen received?

Has A11RegRequest been received by PDSN?

Check the PDSN address is normal in A11RegReply; Check the route.

Can the A11RegRequest be demodulated successfully N

N Y

Is there FACN?

Y Has FACN received the A11RegRequest ? N Check the config of R-P interface Check the route to PDSN Check the IP add in messages are correct or not.

Y Check the config of R-P interface Check the route to PDSN

Message correct? The route normal? Y N

Check the SPI config

Ask FACN to handle it

Ask the FACN or PDSN to check the config Y

Has PCF received the A11RegReply

Ask PDSN to check the config of R-P interface; Check the route to PCF;

Recovered after clear the item?

N Trace packet of R-P interface, to confirm BSC, PDSN or transmission problem

Can the A11RegRespone be demodulated successfully

Check the SPI config

Pass A11 auth? According ot <code description in A11RegReply>, ask PDSN to handle it

N Return

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Compare to the table description of AbrCHAPAuthInd reason codes in the following to find the possible reason.
Possible resson Congradulation Authentication is Success! Access reject. Please check the Radius logs(\Zte.corp\ZXPDSSA100\log\radius*.log), to find the ReplyMessage of Access-Reject massage 1User unavailable: User is not allocated, or the domain is not configured in AAA. 2password mismatched: Password in AT is different from in AAA. 3Invalid IMSI: Maybe the running mode of Radius service is wrong. 1wrong state of A12 port in IPCF. 2A12 ip address is not configured in RPU. 3A12 ip address configured in RPU is different form in data base of AAA. 1Physical connection between AAAServer and IPCF broken: Try to ping A12 Ip of IPCF in AAAServer. Check physical connection, check A12 ip configure of IPCF. 2Radius service of AAAServer is not running. Use command netstat a n to check whether the IP address of AAAServer has bind with Port 1812 and port 1813. And check whether Radius service is running. 3Check log of AAA to confirm the A12 massage is received by AAA. If the A12 Request arrived AAA, and AAA sent the Reply, but AN didnt receive, please check config of AAA. Its possible error in A12 IP address. 4Check A12 ExceptionProbe, to find if there are some ERR_SPS_HRPD_A12_FunParaIncorrect_ TransmitUserAuthRsp_ucRevDataLen error. If so, check the profile of user in AN-AAA. Delete the unused attribute. Solution

Value 0

601

602

Socket setup failure. Telnet RPU to check the port state of IPCF

603/505

AAAServer reply timer expired. Check the radius log file of AAA.

507

Lcp Reply timer Expired. It shows that BSC doesnt receive the Reply of LCP request.

1The state of AT is wrong that AT cant reply to the received massage. Try to reset the AT. 2Poor radio signaling, try to increase the power.

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

Reason code of A11regreply.

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

PPP connection flow

e
Trace the PPP log in terminal side, and the signaling of PDSN. And ask PDSN to check whether there are messages lost?

Messages lost?

Y Use PPPsniffer to trace the PPP messages in BSC, to compare with the log in PDSN and the terminal

Turn to PDSN and terminal productor to check it.

Messages lost?

Y Maybe the Fault is in Um, ABIS, media stream or board. Turn to BSC fault handling flow

Transmission or PDSN problem

Return

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Contents

Common Types of EVDO Service Faults Methods of EVDO Troubleshooting

Collecting and feeding back fault Information Access failure Data transmission performance fault Call drop

Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Data transmission performance faults mainly refer to abnormal data transmission rates of forward and reverse links in DO services, e.g. the transmission is 0, quite low or unstable.

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Data transmission performance fault Failures

Data transmission performance faults mainly refer to abnormal data transmission rates of forward and reverse links in DO services, e.g. the transmission is 0, quite low or unstable.

Normal data transmission rates of forward link in DO services are: The downloading rate of a single user at a near point should be about 340KBps (DO Rev.A) or 250KBps (DO Rls.0). Normal data transmission rates of reverse link in DO services are: The upload rate of a single user at a near point should be about 120KBps (DO Rev.A) or 15KBps (DO Rls.0).

What are the possible causes for data transmission performance faults?

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Common Fault Causes (1)

Common Fault Causes

Poor radio environment (e.g. interference, poor coverage, too many users) Bad terminal performance (including computer, terminal, connection ports, setting of TCPwindowsize) ) BTS software/hardware failures (including clock, RF performance, baseband-RFS link, media plane link, board and version) Improper BTS radio configuration Poor transmission quality of ABIS interface BSC software/hardware failures (including clock, media plane link, version, board and resource overload) Improper DO radio configuration at BSC Incorrect user group setting of AAA

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Common Fault Causes (2)

Common Fault Causes

R-P link failure (including network interfaces mode mismatch, incorrect route configuration, packet loses/delay on transmission link and bandwidth limit) PDSN failure (including packet loss, PPP link setup failure, incorrect office direction configuration, frame header compression problem and traffic limit) P-I link failure (including network interfaces mode mismatch, incorrect route configuration, packet loses/delay on transmission link, improper setting of firewall and bandwidth limit) Poor FTP server performance and small sending buffer

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Additional Information to Be Fed back

the following information has to be fed back in case of a DO performance fault:

Packets captured at the terminal, two ends of R-P interface, two ends of P-I interface and FTP server (using Wireshark\Ethereal) DO LOG data of CHM CDT data Logs of terminals used in the site test (using CNT or other test tool) Performance configuration of FTP server Modes of all network interfaces (including R-P and P-I)

2007, ZTE Corporation. All rights reserved.

A transmission performance fault is reported in DO data service

Internal Use Only

Access failure

Collect such information as phenomenon, occurrence time and range affected

Trouble shooting flow(1)

Check alarms and notifications N Is there related alarm? Y

Check whether any system change was made when the alarm is raised

Does another NE fail?

Forward the fault to related NE engineers and follow up the progress

Is the system changed?

Is the fault solved after related NE resumes to work?

Y Is the fault caused by the change?

Try to recover the system change

Handle the alarm

Is the alarm cleared?

Make clear why the change is made and check if the change is reasonable and implemented again.

Is forward rate abnormal?

Y Go to abnormal forward rate handling process Y Is the fault a global one (check it with network test)? N

f1

f2

Is reverse rate abnormal?

Go to abnormal reverse rate handling process

End

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Trouble shooting flow(1)

Items need checked at first:

Fault phenomena Occurrence time Affect scope of fault Other NE fault System change in BSS or other NE recently Alarms and Notifications Confirm the fault occure on forward rate or reverse rate

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure
N

f1

Global abnormal forward rate handling process (1)

Are BSC versions correct? Y Is the fault solved after versions are corrected? N

Is A8GRELost found in the printout?

Check DO radio parameters of BSC, BTS and CHM

Check networking mode at R-P interface, Check whether working modes of network interfaces on R-P transmission link match and make correction If necessary

Is the fault solved?

Collect RDS information of BSC, seeing if packets are lost within the BSC Y

Check or replace FTP server or use Iperf tool to conduct tests

Do PING tests suffer delay or packet loss?

N Is the system recovered after test or server replacement? Capture packets at multiple points and find out the segment where delay or packet loss takes place Reverse link Y Transmission system Air Interface / BSC Check BSC, including media plane, board and data configuration

PDSN Request PDSN side to conduct necessary check

Server setting or performance problem

Check transmission configuration and routes

Y Return

Is the fault solved?

f2

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Global abnormal forward rate handling process (1)

As its a globle fault, check the BSC side:

Fault phenomena Occurrence time Affect scope of fault Other NE fault System change in BSS or other NE recently Alarms and Notifications Confirm the fault occure on forward rate or reverse rate

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Access failure

f2

Sites abnormal forward rate handling process (2)

Y Is there any abnormal alarm or notification of the BTS Y Handle abnormal alarm and see if it affects DO performance

Are BTS versions correct?

N Is the fault solved after version correction?

N Are ABIS interface resources limited? N Check DO radio parameters and carrier status of BTS Is the rate normal during the tests? Conduct tests in the night when the traffic is low to make confirmation

N Replace the terminal and PC. Is it solved?

Is the system recovered after all above are done? Y Check BTS RFS and antenna feeder system to see if there is any external interference

Y The cause lies in the terminal, PC or their connection

Are RSSI and main-todiversity locking ratio correct?

Collect BTS DOLOG, terminal Log at the site, capture packets at the terminal end, and feed them back for further analysis

Is the fault solved?

Air Interface

Is it an Air/Abis Interface failure?

Abis Interface

Check Abis Interface, and cut over the BTS to another board if necessary. Transmission / engineering /BSC problem? Transmission Engineering BSC

Interference

Use RF fault location method to determine it is a BTS problem or an interference

BTS

Check RFS for forward or reverse interference and handle it

Try to find out the fault at the BTS side

Request the transmission side to assist

Improve the engineering

Check BSC further

Is the fault solved?

f1

Return

2007, ZTE Corporation. All rights reserved.

Internal Use Only

N

Access failure

abnormal reverse rate handling process (2)

Y Check if QoS parameter of RTCMAC stream under QoSClass of QoS Parameter in DO Radio Parameter is default Checkvalue if non-QoS parameter of RTCMAC stream under non-QoS Parameter in DO Radio Parameter is default value

Is the fault a global one?

Check QRAB generation mode and QRAB algorithm of DO CHM of the BTS

Are they correct?

Is the system recovered after correction? Y

Are RSSI and main-todiversity locking ratio correct? Y

Check BTS RFS and antenna feeder system to see if there is any external interference

Is the system recovered?

N Check or replace FTP server or use Iperf tool to conduct tests Do PING tests suffer delay or packet loss?

Collect RDS information of BSC, seeing if packets are lost within the BSC Y

Is the system recovered after test or server replacement? Y Server setting or performance problem

Transmission system

Capture packets at multiple points and find out the segment where delay or packet loss takes place PDSN Request PDSN side to conduct necessary check

Air Interface / BSC

Check transmission configuration and routes

Check BSC, including media plane, board and data configuration

Is the fault solved?

Return

f1/f2
2007, ZTE Corporation. All rights reserved.

Internal Use Only

Data transmission performance fault

Case Analyse

2007, ZTE Corporation. All rights reserved.

Internal Use Only

Contents

Common Types of EVDO Service Faults Methods of EVDO Troubleshooting Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only Common Tools and Commands in EVDO Troubleshooting

2007, ZTE Corporation. All rights reserved.

Internal Use Only

2007, ZTE Corporation. All rights reserved.