Вы находитесь на странице: 1из 37

2G Test Methodology and Definition

2G Test Methodology and Definition

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

2G Test Methodology and Definiton

2G Test Methodology and Definition


Version V1.00 Date 2011-04-27 Author ZTE Approved By Remarks Not open to the Third Party

2014 ZTE Corporation. All rights reserved. ZTE CONFIDENTIAL: This document contains proprietary information of ZTE and is not to be disclosed or used without the prior written permission of ZTE. Due to update and improvement of ZTE products and technologies, information in this document is subjected to change without notice.

2G Test Methodology and Definition

TABLE OF CONTENTS
2G RAN Radio Parameter Mapping Proposal .............................................................................. II TABLE OF CONTENTS.................................................................................................................. III FIGURES TABLES 1 2 2.1 2.2 3 3.1 3.2 3.2.1 3.2.2 3.2.3 3.2.4 3.2.5 3.3 3.3.1 3.3.2 3.3.3 3.3.4 3.3.5 4 4.1 4.2 4.3 4.4 4.4.1 4.4.2 4.4.3 4.4.4 4.5 4.6 4.7 5 5.1 5.2 5.2.1 5.2.2 5.3 5.3.1 5.3.2 5.4 5.5 V V Introduction ................................................................................................................ 1 DiGi Field Test and KPI Requirements..................................................................... 1 KPI Requirements ........................................................................................................ 1 Test ItemsRequirements.............................................................................................. 2 2G Field Test Definition for SSV, Cluster and Area ................................................ 3 Test methodology ........................................................................................................ 3 Main Test Items Description ........................................................................................ 6 RF Scanner Test .......................................................................................................... 6 Short Call ..................................................................................................................... 7 Long Call ...................................................................................................................... 7 MOS ............................................................................................................................. 8 Attach and PDP activation & FTP UL/DL .................................................................... 8 Acceptance Requirements for the Drive Test .............................................................. 9 Single site performance measurement ........................................................................ 9 Cluster network wide performance measurement ....................................................... 9 Area network PAC measurement .............................................................................. 11 Area FAC requirements ............................................................................................. 11 Ericsson and Siemens Reference ............................................................................. 11 Field Test Equipment and Specifications .............................................................. 12 MS/UE........................................................................................................................ 12 Scanners .................................................................................................................... 12 Positioning Systems / GPS ........................................................................................ 13 Data Collection Tools (SW and HW) ......................................................................... 14 PC Hardware and Software Requirements ............................................................... 14 OS Requirements ...................................................................................................... 14 Introduction to TEMS Investigation 11.0 Data Collection .......................................... 14 Applications................................................................................................................ 15 Internal / External Antennas ...................................................................................... 16 Network Servers ........................................................................................................ 16 Laptop ........................................................................................................................ 16 Test Kit Setup and Methods .................................................................................... 17 Laptop/Data Collection Gear Setup ........................................................................... 17 Trace and Data Test .................................................................................................. 17 Scanner Test ............................................................................................................. 18 Multiple Test............................................................................................................... 18 Tools Kit Configuration .............................................................................................. 19 TEMS Configuration Flow .......................................................................................... 19 TEMS Configuration Procedures ............................................................................... 20 Voice Configuration Parameter ................................................................................. 20 Data Configuration Parameter ................................................................................... 20

III

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

2G Test Methodology and Definiton

5.6 5.6.1 5.6.2 5.6.3 5.6.4 5.6.5 6 6.1 6.1.1 6.1.2 6.2 6.3

Measurement and PS/CS Configuration Specifications ............................................ 21 Voice call.................................................................................................................... 21 Short call .................................................................................................................... 23 Long call..................................................................................................................... 24 FTP test ..................................................................................................................... 24 Scanning .................................................................................................................... 25 Acceptance Reports Contents ................................................................................ 28 SSV/SSOA ................................................................................................................. 28 Single Site Verification (SSV) .................................................................................... 28 Single Site Optimisation Acceptance (SSOA) ........................................................... 28 Cluster Acceptance Report ........................................................................................ 28 Area Network Acceptance Report ............................................................................. 30

2G Test Methodology and Definition

FIGURES
Figure 1 Figure 2 Figure 3 Figure 4 Trace and Data Test Connection ................................................................................. 17 Scanner Test................................................................................................................ 18 MultipleTest .................................................................................................................. 18 TEMS Configuration Flow ............................................................................................ 19

TABLES
Table 1 Table 2 Table 3 Test KPI Requirements .................................................................................................. 1 Test Item Requirements................................................................................................. 2 2G KPI Measurement Methodology............................................................................... 3

2014ZTE CORPORATION. All rights reserved.

ZTE Confidential Proprietary

Introduction
Key performance indicators (KPI) are being used to measure and evaluate the performance of the networks. The quality of the network is ultimately determined by the satisfaction of the users of the network. Field tests such as drive-test gives the 'feel' of the designed network as it is experienced in the field. The testing process starts with selection of the 'live' region of the network where the tests need to be performed, and the drive-testing path. This document describes the 2G RAN field test KPIs and requirements for future ZTE solution based DiGi network. As we describe field test KPIs, the test definitions for 2G RAN network implementation, the testing equipments, its set-up and specifications, the collection and processing will be introduced.

Field Test and KPI Requirements


Because contract files are the criterion of the final acceptance, in this chapter, we describe the requirements of field test KPI in contract.

2.1

KPI Requirements
Based on the final acceptance requirement KPI in the contract, the required field test KPIs for the cluster, Area acceptance are the same. But they have different working targets for ZTE to achieve in different project implementation stages.

Table 1

Test KPI Requirements

No. KPI Parameter Voice Network Accessibility GT_01 GT_02 Voice Retainability GT_03 GT_04 Coverage & Quality GT_05 GT_06 GT_07 GT_08 Packet Data GT_09 GT_10 GT_11 Attach success rate PDP context activation success rate Attach Time (setup time) RxLevSub (dBm) Call Setup Time Voice Quality (MOS) % RXQUAL between 0-4 TCH Drop Call Rate Handover Success Rate Call Set up Success Rate Location Update Success Rate

Copyright DiGi 2011

2G Test Methodology and Definiton

GT_12 GT_13 GT_14 GT_15

PDP context activation time EDGE DL throughput TBF Drop Rate EDGE UL throughput

2.2

Test ItemsRequirements
In this section, we will define the test items according to the KPI items in contract For example, the voice CSSRCall Setup Success Rate will be tested by Short Call. The following table gives the test items according to the field test KPI requirements.

Table 2

Test Item Requirements Related Test Item

No. KPI Parameter Voice Network Accessibility GT_01 Call Set up Success Rate GT_02 Location Update Success Rate Voice Retainability GT_03 TCH Drop Call Rate GT_04 Handover Success Rate Coverage & Quality GT_05 RxLevSub (dBm) GT_06 GT_07 GT_08 Call Setup Time Voice Quality (MOS) % RXQUAL between 0-4

Short Call Short Call Long Call Long Call Long Call Short Call MOS Long Call FTP UL/DL FTP UL/DL FTP UL/DL FTP UL/DL FTP DL FTP UL/DL FTP UL

Packet Data GT_09 Attach success rate GT_10 GT_11 GT_12 GT_13 GT_14 GT_15 PDP context activation success rate Attach Time (setup time) PDP context activation time EDGE DL throughput TBF Drop Rate EDGE UL throughput

For cluster swap benchmarking, in order to evaluate the Coverage Equivalencethe RF scanner test is required for cluster benchmarking test before swap and after swap. Category KPI Item Test description Coverage Coverage Equivalence RF Scanner Test

The summary of the basic 2G KPI measurement methodology is contained as below

Table 3

2G KPI Measurement Methodology

End-User Services Voice Packet-Service

Drive test for outdoor YES YES

Measurement Method Walk test for Stationary test indoor YES YES YES YES

OSS Statistics Measurement YES YES

During SSV, the chief measure for single site is the drive test. SSV check list and drive test will performed during SSV/SSOA. For cluster and Area acceptance, the drive test KPI and OSS statistics KPI will both considered.

3
3.1

Field Test Definition for SSV, Cluster and Area


Test Methodology
The test methodology for each test case defined in this Section will be used for Cluster Acceptance, Area Acceptance, and Network Performance Stability Acceptance in accordance to Annex(Acceptance Procedures). The table below is a list of different test methods from the contract

Method reference T01 Voice Call GSM Voice Call Dual Mode Video Short Call

MOC UE1

MTC SUE1

Method desctription Originate call repeatedly with holding time of 120s and waiting time of 5s Alternating MOC/MTC. Originate call at the start of the test and held on for the entire test duration Alternating MOC/MTC. Originate call repeatedly with holding time of 60s and waiting time of 10s If abnormal call release, a new call of holding time of 60s is setup immediately after waiting time of 10s Originate call at the start of the test and held on for the entire test duration If abnormal call release occurs, a new call will be set up immediately Set up a PDP context and immediately follow by FTP session to upload a 450kBytes file The procedure is repeated upon completion of upload after a waiting time of 10s

Remarks

T02

UE2

SUE2

T03

UE3

UE9

T04

Video Long Call

UE4

SUE3

T05

UL PS-64 Short Call

UE5

FTP server

For stationary test, the test is repeated for 5 times For in building test, the test is conducted once per every static test point.

T06

DL PS-384

UE6

FTP

Set up a PDP context and

Copyright DiGi 2011

2G Test Methodology and Definiton

Method reference Long Call

MOC

MTC server

Method desctription immediately follow by FTP session to download a 250MBytes file The procedure is repeated upon completion of download after a waiting time of 5s Set up a PDP context and immediately follow by FTP session to upload a 5MBytes file The procedure is repeated upon completion of upload after a waiting time of 10s

Remarks

T07

HSUPA Call

UE7

FTP server

For stationary test, the test is repeated for 5 times For in building test, the test is conducted once per every static test point.

T08

HSDPA 5 codes 16 QAM Long Call

UE8

FTP server

Set up a PDP context and immediately follow by FTP session to download a 650MBytes file The procedure is repeated upon completion of download after a waiting time of 5s Originate call with holding time of 60s and waiting time of 10s Repeat for 5 times Originate call with holding time of 60s and waiting time of 10s Repeat for 5 times Set up a PDP context and immediately follow by FTP session to download a 2500kBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s Set up a PDP context and immediately follow by FTP session to download a 2500kBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s Set up a PDP context and immediately follow by FTP session to download a 450kBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s Set up a PDP context and immediately follow by FTP session to

T09

Static Voice Call Static Video Call DL PS-384 Short Call

UE1

UE2

T10

UE1

UE2

T11

UE1

FTP server

T12

DL PS-128 Call

UE1

FTP server

T13

DL PS-64 Call

UE1

FTP server

T14

HSDPA 5 codes 16

UE3

FTP server

Method reference QAM Short Call

MOC

MTC

Method desctription download a 25MBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s

Remarks

T15

HSDPA 10 codes 16 QAM Call

UE3

FTP server

Set up a PDP context and immediately follow by FTP session to download a 50MBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s Set up a PDP context and immediately follow by FTP session to download a 100MBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s Perform a ping with 100 byte packet size to FTP server Repeat 5 times Perform a ping with 100 byte packet size to FTP server Repeat 5 times Setup PDP context and immediately originate a call with holding time of 60s Repeat for 5 times

T16

HSDPA 15 codes 16 QAM Short Call

UE3

FTP server

T21

Ping Test R99 Ping Test HSDPA Voice with PDP context

UE1

FTP server FTP server UE2

T22

UE3

T23

UE1

Based on the above list, the 2G test methodology used for the drive test is suggested as following Method reference MOC MTC Method desctription T01 Voice Short Call GSM UE1 UE2 Originate call repeatedly with holding time of 60s and waiting time of 10s If abnormal call release, a new call of holding time of 60s is setup immediately after waiting time of 10s T02 Voice Long Call GSM UE3 SUE1(or call the sp specific test number provided by DiGi) Originate call at the start of the test and held on for the entire test duration If abnormal call release occurs, a new call will be set up immediately after waiting time of 10s

Copyright DiGi 2011

2G Test Methodology and Definiton

Method reference T03 FTP UL

MOC UE4

MTC FTP server

Method desctription Attach and set up a PDP context and immediately follow by FTP session to upload a 450kBytes file The procedure is repeated upon completion of upload after a waiting time of 10s

T04

FTP DL

UE5

FTP server

Attach and set up a PDP context and immediately follow by FTP session to download a 450kBytes file Repeat for 5 times The procedure is repeated upon completion of download after a waiting time of 5s

T05

MOS

UE6

SUE2

Originate call at the start of the test and held on for the entire test duration If abnormal call release occurs, a new call will be set up immediately after waiting time of 10s

3.2
3.2.1

Main Test Items Description


RF Scanner Test
Test Description

RF Scanner Test is used for measuring the coverage of particular clusters/regions. Through RF scanner test, we can get the Rxlev comparison between before and after swap in cluster, and to keep the same coverage after sites cutover. For coverage comparison, the RF scanner will be chosen to survey the coverage of cluster regions. In Cluster coverage benchmarking, we will use one scanner to scan both 900M and 1800M frequencies with same test route before and after swap. Before frequency scan, connect the RF scanner to the notebook; choose the correct channel frequencies of DiGi used to be surveyed. Its best to scan are all BCCHs used in the cluster. For outdoor cluster benchmarking test, DT routes should include all the available roads of test region, including border. Walk-test will be chosen to survey the indoor coverage when necessary. Output KPIs Before Swap and After Swap mean Rxlev and distribution Plot Before Swap and After Swap Rxlev CDF and PDF

3.2.2

Short Call
Test Description

Short Call test is used for evaluating network access performance. Call details such as the call setup success rate, call setup time and so on will be measured by short call. Network access related problems can be found through short call test. For outdoor short call test, spot-test is selected in SSV. DT is chosen for Cluster verification. For indoor short call test, spot-test is also selected in SSV. For cluster, if those areas cannot drive-test, walk-test will be used. For short call during SSV, we will lock to the frequency of the particular cell under test. Short call will be performed fully follow the SSV checklist, and repeat throughout the entire drive test in cluster test. For SSV spot-test, the principle of spot selection is that 1 point in each sector, for OMNI site, 1 point is selected. Output KPIs Call Setup Success Rate Call Setup Time Location update Success Rate

3.2.3

Long Call
Test Description

Long Call test is used for testing the consistency during a call. Problems such as handover and the voice quality related can be discovered by long call testing. For outdoor test, DT is preferred. For those areas cannot drive-test, like indoor case, walk-test will be selected. Test equipment is located in the car.MS will be connected with a test notebook, GPS is also connected. The MO MS will dial the MT phone number for long call test. There is no limitation on the call duration. Long call test will be realized automatically on the test equipment. If drop-call happens, it will make a new call with 10s delays before the redial. For outdoor test, DT routes should include all the available roads of test region, including the border. Walk-test will be chosen to survey the whole indoor area. Output KPIs Rxlev and Rxqual Plots

Copyright DiGi 2011

2G Test Methodology and Definiton

Handover Success Rate Call Drop Rate Rxqual 0~4 share

3.2.4

MOS
Test Description

MOS test is used for testing the overall performance in a particular region or network. MOS Test will be executed by 2 mobile phones. The MOS with Mobile-to-Mobile test with have the calling mobile located in the moving vehicle, whereas the called mobile will be located in a fixed location of good RF environment. MO MS and MT MS will connect with an audio box and also GPS is connected with the test notebook. The MO MS will make long call to MT MS. The call time is no limitation and test will be realized automatically on the test equipment. If drop-call happens, the MS will redial automatically. DT is chosen to carry out the MOS test. Test routes should include all the available roads of test region and across the cluster border. Output KPIs Mean value of MOS

3.2.5

Attach and PDP activation & FTP UL/DL


Test Description

Attach and PDP activation test is used to analyze the access performance of PS service. File Transfer Protocol (FTP) is a technology used to connect two network elements together to transfer files. FTP test is used to analyze the ftp upload and download performance of PS service. The test will be executed by sequence of Attach, PDP Activation, FTP UL/DL and Detach (Suggested there will be 1s interval between 2 round of test). The FTP UL/DL test will perform separately due to the throughput will affect both mobile phones if they are doing the test simultaneously. Output KPIs Attach Success Rate and attach time PDP Activation Success Rate and time FTP DL/UL Average Throughput(EDGE) Throughput DL/UL EGDE Plot TBF Drop Rate

3.3
3.3.1

Acceptance Requirements for the Drive Test


Single Site Performance Measurement
Single Site Verification (SSV)

The single site verification inclusive drive test for outdoor site(s) or walk test for inbuilding site(s). The test cases below are generally described at a high level which includes: Site Data Verification Cell Data Verification Check Points Test Accessibility Test Which Includes of Voice Call Data Accessibility Throughput Test Handover Test Coverage Plot (Drive Test or Walk Test) Quality Plot (Drive Test or Walk Test)

Single Site Optimization Acceptance (SSOA)

SSOA is required when a new site is being brought on air to an accepted cluster. This site is required to be optimised and accepted as part of Provisional and/or Final Acceptance. Drive test/ Walk test and OSS statistic KPIs are required to be measured for the new site and its 1st tier neighbour. Drive test/ Walk test route will be defined by DiGi and this will be discussed and mutually agreed between DiGi and ZTE. The SSV check result and SSOA result will contain in SSV report after the single site passed all the required items.

3.3.2

Cluster Network Wide Performance Measurement


If a swap or new rollout is done Cluster Acceptance is required for Provisional Acceptance. Cluster Acceptance Test Requirements The drive test measurement shall be conducted continuously:

Copyright DiGi 2011

2G Test Methodology and Definiton

Speech traffic: normal working day from 9am to 7pm Normal speed regulation for the area shall be followed The measurement shall be done using the tools specified in Section 5 The measurement shall contain and get all the samples per cell. Antenna of drive test tools has to be inside car (use of external antenna out sidecar is not allowed). Cluster Size and Measurement Routes A cluster typically consists of 10 to 15 sites. DiGi will select the sites and drive test measurement routes. This will be discussed and mutually agreed between the Supplier and Purchaser. The site clusters and routes shall be documented with electronic maps The drive test routes must include Serving areas for all cells in the cluster Handover between cells Most highways, major roads, secondary roads and VIP area will be included Cluster Acceptance Procedures

Before a swap, a benchmarking needs to be done for targeted cell clusters. The benchmark KPIs will be based on drive test result. The purpose of cluster acceptance is the quality and reliability after swap shall be at least on the level it was before swap. Cluster Acceptance Test Cases

The table below summarizes the tests to be done for Cluster Acceptance from contract. No 1 2 3 4 5 6 7 8 9 Test Case Voice Call GSM Voice Call Dual Mode Video Short Call Video Long Call UL PS-64 Short Call DL PS-384 HSUPA Call HSDPA Long Call Voice Walk-In test Long Call Methodology Reference in Section 7 T01 T02 T03 T04 T05 T06 T07 T08 T17 Cluster Drive Test Yes Yes Yes Yes Yes Yes Yes Yes No

10 11 12

Voice Walk-Out Test Voice Drive-In Test Voice Drive-Out Test

T18 T19 T20

No No No

Based on the above, the 2G drive test will perform as following Cluster Drive No Test Case Test 1 2 3 4 5 6 Video Short Call GSM Video Long Call GSM FTP UL FTP DL MOS RF Scanner Test Yes Yes Yes Yes Yes Yes

3.3.3

Area Network PAC Measurement


The drive test for the area acceptance will perform mainly for the cross boarder of different cluster after the cluster in one area were all passed acceptance. The drive test routes will be planned to allow handover between cells. Most highways, major roads, secondary roads and VIP area will be included. The Area Network will consist of the cluster(s) that have been presented to DiGi for Provisional Acceptance. The area size can be defined same as the pilot network which follow the Scope of the Contract DiGi will propose the borders, measurement drive test routes (planned and/or randomly routes/optimization activities) to ZTE. The KPI for the Area acceptance will focus on the overall performance of particular region / area; most of the region/area cell KPI should meet the KPI requirement in contract.

3.3.4

Area FAC requirements


To be defined by DiGi and ZTE in near future.

3.3.5

Ericsson and Siemens Reference


The same testing methods and items were used to collect Ericsson and Siemens cluster and network performance before swap.

Copyright DiGi 2011

2G Test Methodology and Definiton

4
4.1

Field Test Equipment and Specifications


MS/UE
Handsets The mobiles are used for data collection during the field test. Handset Model Support Test Software Sony Ericsson W995/Z750 TEMS Investigation 11.0 Data Collection

The above mobile sets are the exact model that used for field test in this project. Specification Size Weight Networks Connectivity

105.0 x 47.0 x 22.0 mm 115.0 g UMTS 2100 GSM/GPRS 900/1800/1900 Bluetooth technology Infrared port Modem Synchronization PC USB mass storage USB support

For more detail about the mobile phone, please see the user manual of Sony Ericsson W995/Z750, or go to http://www.sonyericsson.com for more information.

4.2

Scanners
Scanners are used for scanning the frequency distribution when doing coverage comparison in GSM testing. BCCH and BSIC decoding are from the scanner test.

Scanner Model
PCTEL EX WCDMA/GSM Scanning Receiver Support Network Standards:

Support Test Software


TEMS Investigation 9.1 Data Collection

WCDMA/HSDPA (2100, 900, 1900, 850, AWS) MHz GSM (1800, 900, 1900, 850) MHz Feature

Concurrent Measurements of Dual Technologies

RSSI Scan across Multiple Bandwidths Simultaneous Testing of All Modes Built-in GPS Available Measurements WCDMA/HSDPA

GSM/EDGE/GPRS

Top N BCH Ec/Io (CPICH, PSCH and SSCH) Io and Peak Ec/Io (CPICH) Signal to Interference Ratio (SIR) Rake Finger Count Delay Measurements

BSIC Decoding Carrier to Interference (C/I) Option BCCH Decoding Option

The scanner can make fast scanning than use mobile scanning, can provides high density measurements and the result from scanner is much accurate in frequency detection.

4.3

Positioning Systems / GPS


GPS is used for locating the position when doing DT test and scanner test in outdoor.

GPS Model
GlobalSat BU-353

Support Test Software


TEMS Investigation 11.0 Data Collection

The above GPS or the hardware support for 2G test is used for the position system for the outdoor test.

Specification Datum Dimension USB Cable Length Voltage Current Operating Temperature

WGS-84 2.08''diameter*0.75''(53mm dia.*19.2mm) 60''(152cm) 4.5-5.5V DC input 80mA typical 40---85

If you want more information about this GPS, please visit www.usglobalsat.com . The USB GPS receivers requires a Windows laptop computer, with a CDROM*, USB port, and NMEA compatible navigation software. The 3rd party mapping/navigation software may have its own system requirements, please check with software vendor for details.

Copyright DiGi 2011

2G Test Methodology and Definiton

4.4

Data Collection Tools (SW and HW)


TEMS Investigation 11.0 Data Collection is used as the GSM drive-testing software in this project (Or the software version can support all the test items as required). The hardware for the testing includes laptops, mobile phones in accordance with the test software, GPS, USB cables and Hubs etc.

4.4.1

PC Hardware and Software Requirements


TEMS Investigation GSM is designed to run on a standard PC and interfaces with the connected devices through serial ports and/or a USB interface. Processor and RAM requirements are strongly dependent on what external devices are connected and what tasks they perform. -- 1 mobile and 1 GPS: 800 MHz Pentium class processor, 256 MB RAM. -- Multiple mobiles, 1 scanner, and 1 GPS: 2 GHz Pentium class processor, 512 MB RAM. Other hardware and software requirements: -- Serial port(s) for mobile(s). -- Serial port for TEMS Scanner or GPS. -- Sound card and loudspeakers for event audio indications. -- Graphics: 1024 x 768 (SVGA) with at least 16 bit colors (High Color). Software requirements: -- Internet Explorer 6.0 or higher required for Report Generator and online help.

4.4.2

OS Requirements
The following operating systems are supported: -- Windows XP Professional with Service Pack 2. -- Windows XP Tablet PC Edition. -- Windows 2000 with Service Pack 4. Installation under other operating systems is not supported.

4.4.3

Introduction to TEMS Investigation 11.0 Data Collection


Features

TEMS Investigation 11.0 software has the following features:

Supports multi-mode test: WCDMA and GSM/GPRS Networks. Supports multiple phones test simultaneously, which includes ten terminals at most (six UE, three Scanner and one GPS), each phone can be controlled independently Supports test of indoor radio signals, and supports test with or without pre-defined paths. Supports all GPS that comply with NMEA and c ommunicate through the RS-232 port, and can test and configure GPS rapidly and intelligently. Supports Scanner measurement including pilot frequency of tow methods for TOPN and User List, continue wave (CW) and spectrum scan. Can display the process of 2/3G handover directly and provide the contrast of serving cell between before handover and after handover. Supports link fault detection and automatic recovery of COM ports, supports alarming for insufficient battery capacity and storage space, GPS location failure, link loss of handset and PNSCanner, and serial port CRC errors, as well as alarming for parameter definition of test items. Can collect GPS location data accurately and rapidly, features powerful geographic display function (for real-timely displaying parameters in different colors), and call event display. Supports real-time capturing and graphical analysis of L1, L2, L3 and layer NAS data, and powerful browsing, real-time decoding, filtering, and categorized display of L3 messages. Supports elaborate real time acquisition and parameters display, including view of physical channel parameters called layer 1, transport channel named layer 2, logical channel, namely MAC layer and RLC related parameters. Powerful voice test, supporting test plan customization, automatic test based on the plan customized, supporting re-connection upon link disconnection, intelligent analysis for causes of call failure, rapid problem location, and real-timely recording the statistics of the voice service dialing test. Powerful data service test, with multiple protocols such as PPP, FTP, and PING integrated, supporting PPP re-dialing at link disconnection, PPP Call by Call test, supporting FTP and PING test. All the test processes can be displayed on the interface as graphics or messages. Can display the multiple paths of each pilot frequency, Ec/Io, and other parameters. Supports simultaneous test of multiple handsets and multi-Scanner.

4.4.4

Applications
TEMS can be used to test air interface of WCDMA/GSM and HSDPA. It enables collection of the signal transmitted and received by UE, and capturing the layer3 messages.

Copyright DiGi 2011

2G Test Methodology and Definiton

TEMS can make automatic call test according to the relevant configuration, including configuration of different call types and different duration. TEMS is easy to operate and the user interface is friendly. TEMS can smoothly upgrade. TEMS can support HSUPA drive-test data collection.

4.5

Internal / External Antennas


Only the scanner will use external antenna. The antennas are also provided by the scanner equipment vendor and in accord with the scanner.

4.6

Network Servers
Data FTP upload / download server The FTP upload and download server is provided by DiGi and the entire FTP DL/UL test will use the same FTP address.

4.7

Laptop
The laptops are used for data collection and also data processing. The Minimum data processing requirement is able to support UEs/MSs, data collection application and testing scenarios for longer time. The laptop cannot run any of applications which are not related to the test, it may slow down the processing power and affect the upload & download throughput. The least requirements for the test laptops are as following: Operating System: Windows2000/XP Lowest Configuration: CPU: 1GHz Memory: 1024MB Monitor resolution: 800*600

Video card: SVGA, 16-bit color display Recommend Configuration:

CPU: Duo CPU with 2.0 GHz Memory: 2.00 GB Video card: SVGA 32-bit color display Hard disk: 20GB remaining space Monitor resolution: 1280*800

5
5.1

Test Kit Setup and Methods


Laptop/Data Collection Gear Setup

5.2

Trace and Data Test

Figure 1

Trace and Data Test Connection

Install and configure test USB port of the adapter box(USB Hub) if some notebook has not enough usb port for testing.

Copyright DiGi 2011

2G Test Methodology and Definiton

5.2.1

Scanner Test

Figure 2

Scanner Test

The Scanning Receiver can be connected directly to the serial port of your laptop if applicable. The direct serial port connection provides RF measurement information and GPS coordinate results.

5.2.2

Multiple Test

Figure 3

MultipleTest

Install and configure test USB port of the adapter box. Refer to manufacturers instructions for more information. Use the RS- 232 cable connects laptop and scanning receiver. Connect the RF input port of the scanning receiver communications cable to the RF antenna. Connect USB port of the adapter box to the Laptop USB port.

Connect the GPS antenna to the GPS ANT port of the adapter box. And use date/trace cable connect test phone to the adapter box.

The test engineer will define the equipment based on the actual configuration of his hardware.

5.3

Tools Kit Configuration


The main equipment for the test includes 2 mobile phones, 1 scanner, a notebook, a GPS, sometimes it will need a USB hub (in case the USB port is not enough), a hard dog for TEMS. We need configure several parameters in TEMS before test and to check whether all the equipment status is normal.

5.3.1

TEMS Configuration Flow


In this section, we will introduce the configuration flow of TEMS, as this is important before the test.

Figure 4

TEMS Configuration Flow

Copyright DiGi 2011

2G Test Methodology and Definiton

5.3.2

TEMS Configuration Procedures


Install TEMS Investigation on the portable computer. The installation is automatic. Just insert the CD and follow the on-screen instructions. Install drives of the test hardware, such as drives for GPS, Scanner, and Mobile phone. USB drivers need to be installed in order to use the hardware for test. Whether for TEMS measurements or data service testing. These drivers are specific to each equipment (hardware) supplier. Connect the hardware for testing. Ensure the equipment is in good connection and the test mobile phones and GPS can be identified and assigned ports. Equipment Configuration for different tests. Most of the configuration is the same for both packet-switched and circuit-switched service. Where differences exist, they are pointed out on each occasion. Command sequence and parameters setting for test. Command sequence determines the procedures of the test and parameters are used to control the test when the test starts. After the configurations of the above steps, the test investigation can be start.

5.4

Voice Configuration Parameter


Description The call number A call with unlimited times The number of calls The duration of the call The idle time Short Call Value MT number No unlimited times 60s 10s Long Call Value MT number/ specific Number Yes N/A unlimited times 10s

Parameters Call Number Continuous Call Call Count Call Time(S) Idle Time(S)

5.5
Item Attach PDP APN

Data Configuration Parameter


Parameter explanation The APN in the PDP Activate Request During the test, the test software will dialup the network automatically when this option is selected; otherwise system will indicate the user to make the dial-up manually. The proper APN should be choose and activated if DiGi has more than one APN. Server IP User name DiGi to provide DiGi to provide Value DiGi to provided

FTP download test

Dial-Up Networking

DiGi to provide

Password The local path to save the download file The local path to save the download file During the test, the test software will dialup the network automatically when this option is selected; otherwise system will indicate the user to make the dial-up manually. The proper APN should be choose and activated if DiGi has more than one APN. IP address of the server User name Password Path of the file server The path and name of upload file used for test in the FTP server

DiGi to provide Defined by the tester

Dial-Up Networking FTP upload test

DiGi to provide

DiGi to provide DiGi to provide DiGi to provide Defined by the tester on PC

5.6

Measurement and PS/CS Configuration Specifications


This section will describe each test step of all features in detailed, for example: Voice all, Ping test, FTP test and so on. ZTE will use TEMS to test at overlay network. TEMS test setup Description

5.6.1

Voice call
Connect among the test phone GPS soft dog and laptop. Configure the COM port of all equipment, and connect with laptop.

Copyright DiGi 2011

2G Test Methodology and Definiton

Set up long call and short call parameters

5.6.2

Short Call

After navigating to Voice Call Test, Call Monitoring window appears as shown in, In Call Monitoring window user can configure voice call parameters. Software is used for automatic dial-up, at least 100 times.

Copyright DiGi 2011

2G Test Methodology and Definiton

Set hold-on time with 10s; set idle time with 10s; set call setup time with 10s. Record number of call success times and total number of origination times. In the test process, the drive speed is according to the road traffic environment. .

5.6.3

Long Call
You can dial the special service number (provided by DiGi), so that you can use only one MS to do the long call test .Or tester can dial the MT phone number then 2 mobile phones will be used for testing.The duration you should set as large as possible. Besides, if you set redial options, when call-drop happens, the MS will redial.

5.6.4

FTP Test
The purpose of the FTP test is to verify the stability of data transfer rate, one MS is used to originate the PS data service through the system. FTP download and upload function test will be performed. Following is the testing procedures: On the test route, use MS to originate the PS data service through the system

The MS uses the FTP software to download a data file (450K), meanwhile the UE uses the ftp software to upload a data (450K) to the server, continuous download and upload in the test route. In the test process, the drive speed is according to the road traffic environment. Ftp server setting is following: APN and IP Address will provided by DiGi.

5.6.5

Scanning
The purpose of the scanner test is to verify the coverage of the network, Following is the testing procedures: Select appropriate test route. On the test route, connect the scanner to the laptop, set the frequency and record the signal data. In the test process, the drive speed is according to the road traffic environment.

Copyright DiGi 2011

2G Test Methodology and Definiton

For the most part we dont choice decode the BSIC. It will consume lots of system resource and will lead to inaccurate results. So if it is not really necessary we dont choice decode BSIC.

Copyright DiGi 2011

2G Test Methodology and Definiton

6
6.1
6.1.1

Acceptance Reports Contents


The reports for 2G test for SSV, Cluster and Area will include the following content.

SSV/SSOA
Single Site Verification (SSV)
The Supplier shall commence and perform the site verification inclusive drive test for outdoor site(s) or walk test for in-building site(s). The test cases below are generally described at a high level which includes: Site Data Verification Cell Data Verification Check Points Test Accessibility Test Which Includes of Voice and Video Call Data Accessibility Throughput Test Handover Test Coverage Plot (Drive Test or Walk Test) Quality Plot (Drive Test or Walk Test)

6.1.2

Single Site Optimisation Acceptance (SSOA)


Drive test/ Walk test and OSS statistic KPIs are required to be measured for the new site and its 1st tier neighbour. Drive test/ Walk test route will be defined by DiGi and this will be discussed and mutually agreed between the Supplier and Purchaser

A report which consists of the SSV and SSOA will be submitted to DiGi after all the test items passed.

6.2

Cluster Acceptance Report


Table of Contents Executive Summary KPI Results


(1) (2)

Cluster Drive Test KPI results Relevant GERAN sites KPI OSS statistics Cluster Information Cluster Map Location Cluster Site Information
Table containing Site ID, Site Name, Latitude, Longitude, Height, No. of antenna, Antenna type, Antenna Tilt, Scrambling Code, BTS Type, RNC BSC Attachment of the RRM parameter settings

Defined Measurement Test Routes Cluster Drive Test Plots Cluster Drive Test RSCP Plot Cluster Drive Test Ec/Io Plot Cluster Drive Test Voice BLER Plot Cluster Drive Test Voice MOS Plot Cluster Drive Test Video BLER Plot Cluster Drive Test PS384 DL Throughput Plot Cluster Drive Test Failed Events Plot (fail events include call setup fail, drop call, handover failure, location update failure) Site Link Budget Calculation Coverage Prediction Plots from planning tool Issues and action/recommendation Analysis of Failed Events and actions taken List of other optimisation activities List of outstanding items

Monthly Performance Reports Table of Contents Executive Summary KPI Results

Copyright DiGi 2011

2G Test Methodology and Definiton

Summary table of current and previous round(s) of Cluster Drive Test KPI results Summary table of current and previous round(s) of Cluster Stationary Test results KPI

Summary table of current and previous round(s) of Cluster OSS Statistics KPI results

Area Drive Test Plots Cluster Drive Test RSCP Plot Cluster Drive Test Voice BLER Plot Cluster Drive Test Voice MOS Plot Cluster Drive Test Video BLER Plot Cluster Drive Test PS384 DL Throughput Plot Cluster Drive Test Failed Events Plot (fail events include call setup fail, drop call, handover failure, location update failure) Issues and action/recommendation Analysis of Failed Events and actions taken List of other optimisation activities List of outstanding items

6.3

Area Network Acceptance Report


Table of Contents Executive Summary KPI Results Summary table of Cluster Drive Test KPI Summary table of monthly stationary tests KPI (optional) Summary table of monthly Cluster OSS Statistics KPI Area Network Information Updated Node B cell design data Table containing Site ID, Site Name, Latitude, Longitude, Height, No. of antenna, Antenna type, Antenna Tilt, Scrambling Code, BTS Type, RNC BSC

Attachment of the up to date RRM parameter settings RNC configuration settings and Performance Key Core Network and Service Network settings that may affect the biasness of the Final Acceptance test Defined Measurement Test Routes Summary of Drive Test Plots Cluster Drive Test RSCP Plot Cluster Drive Test Voice BLER Plot Cluster Drive Test Voice MOS Plot Cluster Drive Test Video BLER Plot Cluster Drive Test PS384 DL Throughput Plot Cluster Drive Test Failed Events Plot (fail events include call setup fail, drop call, handover failure, location update failure) All 6 rounds of monthly Cluster Stationary Test (optional) Table of results for every stationary test spots Area Network OSS Statistic KPI trend for each KPI Issues and action/recommendation Analysis of Failed Events and actions taken List of other optimisation activities List of outstanding items

Copyright DiGi 2011