Вы находитесь на странице: 1из 50

TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.

Nokia Siemens Networks


TD-LTE KPI Optimization Guidelines for RL15TD

Copyright Nokia Siemens Networks. This material, including documentation and any
related computer programs, is protected by copyright controlled by Nokia Siemens Networks.
All rights are reserved. Copying, including reproducing, storing, adapting or translating, any or
all of this material requires the prior written consent of Nokia Siemens Networks. This
material also contains confidential information which may not be disclosed to others without
the prior written consent of Nokia Siemens Networks.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 1 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Revision History
Date Revision Revison History Author

2011-12-1 V1.0 Draft version Zhengwei Gao

Add Chapter 1 introducing main purpose of the


document to avoid any ambiguity and
misunderstanding.
2012-1-12 V1.1 Zhengwei Gao
Add hyperlinks of reference materials to Chapter 6
so that people can find them more easily.
Other minor changes.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 2 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Table of Contents
TD-LTE KPI Optimization Guidelines for RL15TD ........................................................................................ 1
Revision History ............................................................................................................................................ 2
Table of Contents .......................................................................................................................................... 3
Glossary ........................................................................................................................................................ 5
List of Figures ................................................................................................................................................ 6
List of Tables ................................................................................................................................................. 7
1 Introduction ............................................................................................................................................ 8
2 KPI Overview ......................................................................................................................................... 8
2.1 KPI Categories .............................................................................................................................. 8
2.1.1 Accessibility ............................................................................................................................... 8
2.1.2 Retainability (Reliability) ............................................................................................................ 9
2.1.3 Throughput ................................................................................................................................ 9
2.1.4 Latency ...................................................................................................................................... 9
2.1.5 Mobility .................................................................................................................................... 10
2.2 Services Levels ........................................................................................................................... 10
2.3 Field and Network KPIs ............................................................................................................... 11
3 Field KPIs ............................................................................................................................................. 11
3.1 Field KPI Definitions .................................................................................................................... 11
3.1.1 (FTP) Service Accessibility Ratio ............................................................................................ 12
3.1.2 (FTP) Completed Session Ratio ............................................................................................. 13
3.1.3 (FTP) Single User Throughput ................................................................................................ 14
3.1.4 Attach Time ............................................................................................................................. 16
3.1.5 Attach Success Rate ............................................................................................................... 18
3.1.6 Service Request (EPS) Time, UE Initiated ............................................................................. 19
3.1.7 Service Request (EPS) Success Rate .................................................................................... 22
3.1.8 Service (EPS Bearer) Drop Rate ............................................................................................ 23
3.1.9 Handover Procedure Time ...................................................................................................... 24
3.1.10 Handover Success Rate ..................................................................................................... 25
3.1.11 (LTE) Round Trip Time ....................................................................................................... 26
3.1.12 (LTE) Single User Throughput ............................................................................................ 27
3.1.13 (LTE) Service Interrupt Time (HO) ...................................................................................... 28
3.1.14 (RB) Single User Throughput .............................................................................................. 29

Copyright Nokia Siemens Networks 2011 Company Confidential Page 3 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

3.1.15 Cell Throughput ................................................................................................................... 30


3.2 Test Conditions and Target Values ............................................................................................. 31
4 Network KPIs ....................................................................................................................................... 32
4.1 Network KPI Definitions .............................................................................................................. 32
4.2 Test Conditions and Target Values ............................................................................................. 33
5 Appendix I: Trial Network KPIs ............................................................................................................ 34
5.1 CMCC Large Scale Trial ............................................................................................................. 34
5.1.1 Test Environment and Baseline Configuration ........................................................................ 34
5.1.2 Field Results ............................................................................................................................ 35
5.2 Bharti Airtel Trial .......................................................................................................................... 39
5.2.1 Test Environment and Baseline Configuration ........................................................................ 39
5.2.2 Field Results ............................................................................................................................ 40
5.3 Maxis Aircel Trial ......................................................................................................................... 42
5.3.1 Test Environment and Baseline Configuration ........................................................................ 42
5.3.2 Field Results ............................................................................................................................ 44
5.4 Voentelecom Russia ................................................................................................................... 46
5.4.1 Test Environment and Baseline Configuration ........................................................................ 46
5.4.2 Field Results ............................................................................................................................ 47
5.5 Appendix II: Early Network KPIs ................................................................................................. 49
5.5.1 STC Saudi ............................................................................................................................... 49
5.5.2 Sky Brazil ................................................................................................................................ 49
6 References ........................................................................................................................................... 49

Copyright Nokia Siemens Networks 2011 Company Confidential Page 4 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Glossary
IBLER Initial BLock Error Rate

Copyright Nokia Siemens Networks 2011 Company Confidential Page 5 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

List of Figures
Figure 1 LTE Bearer Service Architecture .................................................................................................. 10
Figure 2 Attach Procedure with Initial EPS Bearer Establishment (3GPP 23.401) .................................... 18
Figure 3 EPS Bearer Setup Procedure, Part I ............................................................................................ 21
Figure 4 EPS Bearer Setup Procedure, Part II ........................................................................................... 22
Figure 5 Test Environment of CMCC Large Scale Trial in Hangzhou ........................................................ 34
Figure 6 Test Environment of Bharti Airtel TD-LTE Trial ............................................................................ 39
Figure 7 Test Environment of Maxis Aircel TD-LTE Trial ........................................................................... 43

Copyright Nokia Siemens Networks 2011 Company Confidential Page 6 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

List of Tables
Table 1 List of Field KPIs ............................................................................................................................ 12
Table 2 List of Network KPIs ....................................................................................................................... 33
Table 3 Baseline Configuration of CMCC Large Scale Trial in Hangzhou ................................................. 34
Table 4 Field Results of CMCC Large Scale Trial in Hangzhou ................................................................. 35
Table 5 Baseline Configuration of Bharti Airtel TD-LTE Trial ..................................................................... 39
Table 6 Baseline Configuration of Maxis Aircel TD-LTE Trial..................................................................... 43
Table 7 Baseline Configuration of Voentelecom TD-LTE Trial ................................................................... 46

Copyright Nokia Siemens Networks 2011 Company Confidential Page 7 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

1 Introduction
This document is focused on what kind of KPI values can be achieved in TD-LTE with certain network
configurations by collecting KPIs from trial networks and commercial networks. It can be seen as a
complementary to [3], in which the KPI targets mainly come from lab tests and/or system-level
simulations. KPIs from this document can be taken as the baseline KPIs which are useful for KPI
negotiation with our customers and for KPI optimization.
This document doesnt actually contain any approaches or methods which can be used in network
optimization phase. Please refer to [5] for optimization guidelines for TDDFDD- LTE which is a
counterpart of the FDD. Unfortunately, there is no official version for TDD-LTE version [6]yet.
KPIs in this document are specified for RL15TD.
This document is organized as follow:
Chapter 1: This chapter gives an introduction to main purpose of this document.
Chapter 2: This chapter gives an overview of KPIs in LTE, including KPI categories, KPI hierarchies and
general information about field KPI and network KPI.
Chapter 3: This chapter firstly introduces which kinds of field KPIs are interested and their definitions,
and the second part summarizes field KPI baselines and conditions to achieve the expected KPIs.
Chapter 4: This chapter firstly introduces which kinds of network KPIs are interested and their definitions,
and the second part summarizes network KPI baselines and conditions to achieve the expected KPIs.
Chapter 5: This chapter compiles achieved KPIs from global TD-LTE trial networks and early networks.
Chapter 6: This chapter contains a list of useful reference materials.

2 KPI Overview
Key performance indicators (KPIs) are a set of selected indicators used for measuring the network
performance which determine the quality of a network.
They are used during RFI/ RFQ phases and during the network/cluster acceptance process. Additionally,
network KPIs can be used to monitor a network continuously and warn in time about potential problems.

2.1 KPI Categories


There are several KPIs categories defined, each one looking at a different performance aspect. The
names indicate the respective category according to 3GPP 32.410 and 32.450. The names in brackets
indicate the terminology commonly used if different from 3GPP.

2.1.1 Accessibility
Accessibility KPIs characterize the availability of a service, or service element to its users. They are
expressed as a ratio of successful service requests to the total number of attempts. E.g. Call success
ratio.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 8 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Service request attempts, which are rejected due to insufficient access rights, bad user authentication, or
errors in request parameters are not considered and should be taken out of the samples used for
calculating the metric.

2.1.2 Retainability (Reliability)


Reliability KPIs can be referred to Service Drop Rates or to Transport Error Rates, that is, the ratio
between erroneous or lost data units and the overall number of data units sent.
The Service Drop Rates KPIs (i.e. drop rates) can be measured in the field with an FMT (Field
Measurement Tool). They represent the ratio between unsuccessful (abnormal) terminations of a service
request (e.g. network attach, register, service request) to the overall established services.
For certain services, like bearer creation or VoIP call the service drop rate is also related to the duration
of the session. This is motivated by the fact that the possibility of network errors, e.g. hand-over failures is
higher if the service duration is longer. In this case, the drop ratio is expressed in [1/s] units instead of
[%].
As always-on connectivity is the basic idea for LTE, traditional CDR KPIs may cause abnormal results.
CDR per Data Volume has been proposed to solve this problem.

2.1.3 Throughput
Throughput KPIs show the data rates that can be provided to the users of a cell under different radio
conditions. The location of the UE in the cell and the number of active UEs in that cell will strongly
influence the data rate that can be experienced by one user.
Peak/Average cell throughputs and peak user throughputs are the metrics that characterize the capacity
of an LTE network, and the average UL/DL user throughputs along a drive route (i.e. under different radio
conditions) represent the end user experience. However, only cell throughput and peak user throughput
metrics will be covered in this document.

2.1.4 Latency
Latency KPIs in this document are referred to RTT (Round Trip Time) measurements and to signalling
procedures (e.g. setup times)
RTT for data applications is measured with the Ping application of the UEs operating system or with a
comparable measurement tool. It records the time difference between sending an ICMP Echo Request to
an IP host and the reception of the corresponding ICMP Echo Reply message.
The measurement can be executed between a mobile terminal and an IP host, which is configured to
respond to ICMP Echo Requests.
The test is to be done for stationary and mobile end users. The measurement with stationary UE should
be performed under average radio link conditions with normal SINR.
As mentioned above, the latency category also refers to the time for signalling procedures on the network
or on application level (e.g. RAB service setup, VoIP call setup flow). In case of measuring latencies over
the radio link, the time needed to attach the mobile to the network and to setup the necessary radio
resources is a significant component.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 9 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

2.1.5 Mobility
Mobility KPIs build a group because all of them are related to hand-over procedures. They could,
however, be discussed in the Latency and Accessibility groups as well, because HO-related KPIs are
either service interrupt time or success ratio type KPIs.
A successful handover is understood to be any successfully performed make before break procedure and
it is marked by an entry in the log, which carries the identifier of the new cell (destination).

2.2 Services Levels


In addition to the different categories, KPIs are also defined for different service levels as per 3GPP
36.300. These are:
Application Services
LTE E2E Network Service
Radio Bearer (RB) Service
IP based LTE Bearer Services: X2, S1, S5/S8 and external

Figure 1 shows the LTE Bearer Service Architecture where the KPIs categories above can be mapped
into.

Figure 1 LTE Bearer Service Architecture

When it comes to field performance evaluation the most interesting KPIs are the LTE E2E Network
Service KPIs as they characterize the overall LTE network performance. Another important group is the

Copyright Nokia Siemens Networks 2011 Company Confidential Page 10 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Application Service KPIs as they show the users perceived quality of application services as impacted
by the LTE network. The KPIs of this group are service specific. The services of common interest for
mobile network operators and therefore, the ones that can be of interest during an acceptance process
are:
Web Browsing (HTTP)
Data Upload / Download (FTP)
Email (POP/SMTP)
Conversational Audio Video (VoIP)

The current document concentrates on the Data Upload / Download application service.

2.3 Field and Network KPIs


Field KPIs are the scope of drive testing campaigns either in a trial network or in a live network. They are
also called drive test KPIs and can be obtained with a FMT, whereas network KPIs are based on counters
collected via NetAct.
Drive test KPIs measurements are recommended in all types of projects (trial, roll-out, swap and
optimization). They reflect the network performance along the different drive routes and somehow, reflect
the end user experience. They can also be collected in static or indoor locations (e.g. airports, shopping
centres, office buildings). However, drive testing campaigns are expensive, time consuming and cannot
cover the whole network as the NetAct measurements do. Self-optimization related features in later
releases, part of the SON concept, may have an impact on the amount and need of drive testing.
Network KPIs do have statistical significance only if the traffic levels are high enough. This is a reason
why, especially during the cluster acceptance process when there is not enough load in the network, the
network KPIs may not reflect the network performance and the acceptance process should be based on
field KPIs. With the network KPIs it is also unknown if the users causing bad statistical data samples are
well inside the coverage area or at the coverage borders. However, network KPIs can be used to monitor
networks permanently and highlight potential problems.
Network KPIs are valid and recommended in swap and optimization cases where networks are supposed
to be mature and with enough traffic to provide reliable results. In future swap cases (too early at the
moment), network KPIs shouldnt be used to compare with other vendors KPIs as counters and counter
triggering conditions are likely to be different.

3 Field KPIs

3.1 Field KPI Definitions


The defined field KPI lists do not present by any means a complete set of KPIs to be used in a customer
case. It is the project team/CBT responsibility to define and agree with the customer the final list of KPIs
to be used in each and every case.
The measurement procedure of each KPI is just informative, actual test procedure depends on
customers requirements and may quite different from what has been stated here.
The most interested field KPIs are listed below:

Copyright Nokia Siemens Networks 2011 Company Confidential Page 11 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Table 1 List of Field KPIs

KPI Name KPI Category


Service Accessibility Ratio [%] Accessibility
Application PS Data Services
Services (FTP, HTTP etc) Completed Session Ratio [%] Reliability
Single User Throughput [Mbps] Throughput
Attach Time [ms] Latency
Attach Success Rate [%] Accessibility
Service Request (EPS) Time [ms] Latency
Control Plane Service Request (EPS) Success Rate [%] Accessibility
LTE E2E Network Service (EPS) Drop Rate [%] Reliability
Service Handover Procedure Time [ms] Latency
Handover Success Rate [%] Accessibility
(LTE) Round Trip Time (RTT) [ms] Latency
User Plane (LTE) Single User Throughput [Mbps] Throughput
(LTE) Service Interrupt Time (HO) [ms] Latency
Radio Bearer (RB) Single User Throughput [Mbps] Throughput
User Plane
Services Cell Throughput [Mbps] Throughput
Note that: Single User Throughput KPI can be specified on each protocol layer, such as application
layer, IP layer, L2 layer (i.e. PDCP/RLC/MAC layer) and L1 layer (i.e. physical layer).

3.1.1 (FTP) Service Accessibility Ratio

(FTP) Service Accessibility Ratio [%]

Definition It denotes the probability that the user can establish the necessary bearer
(EPS) and access the FTP service successfully.
Service access covers starting the FTP client on the UE, setting up mobile
access and creating a TCP connection to the FTP server.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 12 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Measurement methodology 1. Set the FMT to generate calls to the FTP Server automatically with the
following script/sequence:
a. Connection attempt (network attach)
b. Log into the FTP Server
c. FTP Download 800MB file (or other size with transfer duration
>1min)
d. Wait 3 sec. after session finishes
e. FTP Upload 300MB file (or other size with transfer duration
>1min)
f. Log out the FTP Server If UE gets into UE deregistered status
between calls, this test also valid to check the Attach Success
Rate
g. Connection release (network detach)
h. Leave 5 seconds before start next session
i. Repeat all previous steps (from a)
2. Continue drive testing till the drive route is completed.
3. Stop the FMT at the end of the route.
4. If the route does not generate more than 100 calls repeat the
measurement on the same measurement route.
5. The FMT shall record the CRS RSRP, RSRQ and CRS SINR during
the measurement.
6. Verify the PS Data Call success rate as per formula below

Formula (logical)

Assumptions, pre-conditions The FTP transfer shall be executed in binary mode.


Preconditions: UE EMM-DEREGISTERED

Message flow, trigger points Trigger points:


Start: ftp get / put command issued on the UE.
Stop: first data byte sent / received by the UE, or ftp indicates network error
(e.g. timeout expired).

Related E2E KPIs (FTP) Completion Session Ratio.

3.1.2 (FTP) Completed Session Ratio

(FTP) Completed Session Ratio [%]

Definition The Completion Session Ratio is the proportion of completed FTP


sessions and sessions that were started successfully. The session is not
completed if a predefined timer expires, or a network failure occurred that
could not be repaired by automatic restarts.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 13 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Measurement methodology 1. Set the FMT to generate calls to the FTP Server automatically with the
following script/sequence:
a. Connection attempt (network attach)
b. Log into the FTP Server
c. FTP Download 800MB file (or other size with transfer duration
>1min)
d. Wait 3 sec. after session finishes
e. FTP Upload 300MB file (or other size with transfer duration
>1min)
f. Log out the FTP Server If UE gets into UE deregistered status
between calls, this test also valid to check the Attach Success
Rate
g. Connection release (network detach)
h. Leave 5 seconds before start next session
i. Repeat all previous steps (from a)
2. Continue drive testing till the drive route is completed.
3. Stop the FMT at the end of the route.
4. If the route does not generate more than 100 calls repeat the
measurement on the same measurement route.
5. The FMT shall record the CRS RSRP, RSRQ and CRS SINR during
the measurement.
6. Verify the Completion Session Ratio as per formula below

Assumptions, pre-conditions The FTP transfer shall be executed in binary mode.


FTP Application DL Throughput as measured by the DOS ftp application.
If above measurement methodology is followed the UE needs to attach to
the network between FTP UL/DL sessions if the Attach Success Rate
wants to be measured.

Formula (logical)

Message flow, trigger points Trigger points:


Start: ftp get / put command issued on the UE
Stop: after error free execution of the command, ftp returns with a
prompt and indicates the number of bytes transmitted.

Related E2E KPIs (FTP) Service Accessibility Ratio.

3.1.3 (FTP) Single User Throughput

(FTP) Single User Throughput [Mbps]

Copyright Nokia Siemens Networks 2011 Company Confidential Page 14 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Definition After the connection to the FTP server has been successfully established,
the parameter describes the average data transfer rate measured over the
data transfer phase. I.e. prerequisite to the data transfer is successful
Service Access (availability of an EPS bearer and connection to the FTP
service). The data transfer phase shall also be successfully terminated.
The throughput is measured on application level in UL/DL direction.

Measurement methodology 1. Set the FMT to generate calls to the FTP Server automatically with the
following script/sequence:
a. Connection attempt (network attach)
b. Log into the FTP Server
c. FTP Download 800MB file (or other size with transfer duration
>1min)
d. Wait 3 sec. after session finishes
e. FTP Upload 300MB file (or other size with transfer duration
>1min)
f. Log out the FTP Server If UE gets into UE deregistered status
between calls, this test also valid to check the Attach Success
Rate
g. Connection release (network detach)
h. Leave 5 seconds before start next session
i. Repeat all previous steps (from a)
2. Continue drive testing till the drive route is completed.
3. Stop the FMT at the end of the route.
4. If the route does not generate more than 100 calls repeat the
measurement on the same measurement route).
5. The FMT shall record the CRS RSRP, RSRQ and CRS SINR during
the measurement.
6. Verify the average DL/UL throughput of each session and determine
the final average throughput of all sessions.
7. The measured mean throughputs can be displayed in diagrams as
function of time to show the impact of FTP slow start.

Assumptions, pre-conditions The initial status of UE before starting the application:


EMM-REGISTERED, ECM-CONNECTED. If above measurement
methodology is followed the UE needs to attach to the network between
FTP UL/DL sessions if the Attach Success Rate wants to be measured.
The FTP application shall be executed in binary mode, and the file to be
transferred shall be binary.

Formula (logical)

Message flow, trigger points The average throughput is measured from opening the data connection to
the end of the successful transfer of the content (file, e-mail or web page).
Trigger points for ftp upload / download are defined in [ETSI102.250-2]
Chapters 4.6.1.7 and 4.6.2.7.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 15 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Related E2E KPIs (FTP) Service Access Time, (FTP) Session Time.

3.1.4 Attach Time

Attach Time (Control Plane) [ms]

Definition With Attach, the mobile terminal registers at the LTE network. At the
end of the procedure the UE is authenticated, and a default (non-
GBR) bearer is established.
The Attach Time is the interval between the RRC CONNECTION
REQUEST (carrying the Attach Request) and the reception of a
positive response by the UE (Attach Complete).

Measurement methodology 1 LTE UE/terminal with FMT


Set FMT to generate data calls to the data server automatically
Check the UE DEGERISTERED between data calls
Start drive test and continue driving until drive route is completed
and enough samples are collected (re-drive route if necessary)
Verify the Attach Time

Assumptions, pre-conditions UE status before measurement: EMM-DEREGISTERED


UE locked to LTE network.

Formula (logical)

Message flow, trigger points Trigger points on UE R interface (see Figure 2Figure 2, and refer to
[3GPP23.401]):
Begin: 1. RRC CONNECTION REQUEST sent by UE carrying
L3 NAS: Attach Request in its body.
End: 21. RRC DIRECT TRANSFER message with L3 NAS:
Attach Complete in its body sent by UE (refer to [3GPP24.008]).

Related E2E KPIs a) Attach Success Rate.


b) No 3GPP defined KPI for Attach Time found exists, or
measurements defined in R8 documents.

Detailed attach procedure with initial EPS bearer establishment is shown below.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 16 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

UE eNodeB new MME Old Serving GW PDN GW PCRF HSS


MME/SGSN
BUE
1. Attach Request
EIR
2. Attach
Request 3. Identification Request
3. Identification Response
4. Identity Request
4. Identity Response
5a. Authentication / Security
5b. Identity Request/Response
5b. ME Identity Check
6. Ciphered Options Request
6. Ciphered Options Response
7. Delete Bearer Request 7. PCEF Initiated IP-CAN
Session Termination
7. Delete Bearer Response
(A)
8. Update Location Request
9. Cancel Location
9. Cancel Location Ack

10. Delete Bearer Request 10. PCEF Initiated IP-CAN


Session Termination
10. Delete Bearer Response
(B)

11. Update Location Ack


12. Create Default Bearer Request
13. Create Default Bearer Request
14. PCEF Initiated IP-CAN
Session Establishment
(C)
15. Create Default Bearer Response
First Downlink Data (if not handover)
16. Create Default Bearer Response
17. Initial Context Setup Request / Attach Accept
18. RRC Connection Reconfiguration
19. RRC Connection Reconfiguration Complete
20. Initial Context Setup Response
21. Direct Transfer
EUE
22. Attach Complete
First Uplink Data
23. Update Bearer Request
23a. Update Bearer Request
23b. Update Bearer Response
(D)
24. Update Bearer Response
First Downlink Data
25. Notify Request
26. Notify Response
Copyright Nokia Siemens Networks 2011 Company Confidential Page 17 (50)
TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Figure 2 Attach Procedure with Initial EPS Bearer Establishment (3GPP 23.401)

For initial Attach, the following specifics of the message flow in Figure 2Figure 2 have to be considered:
A signaling bearer (SRB1) is to be established, before an Attach Request can be sent, i.e. the UE is
expected execute a contention based Random Access (RA) procedure.
Step 3. is not executed, since the IMSI is queried from the UE directly using Step 4. The optional
Identity Request / Response messages in step 5.b do not have to be exchanged if the MEI is already
returned by the UE using the Security Mode Complete message as part of step 5.a.
Steps 7-11 are not executed, no EPS bearers are to be deleted.

3.1.5 Attach Success Rate

Attach Success Rate (Control Plane) [%]

Definition The Attach Success Rate is defined as the ratio between the number of
successful registrations (RRC DIRECT TRANSFER: Attach Complete) and the
number of all requests (RRC CONNECTION REQUEST: Attach Request). It is the
probability that a user can attach to the LTE network at any moment of time.
The calculated success ratio figure excludes attach requests, which are rejected
by authentication failures. On the other hand, network attach requests which are
terminated by timer expiry (due to the unavailability of some LTE resource) are
considered as unsuccessful registrations.
If the success rate is calculated on the eNB by counting incoming RRC requests,
RRC CONNECTION REQUEST retries are to be excluded, since they would
increase the overall number of establishment attempts, and thus reduce the
success ratio.
Measurement For defining the success ratio, the same series of Attach Time measurements
methodology can be used:
1 LTE UE/terminal with FMT
Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and enough
samples are collected (re-drive route if necessary)
Verify the Attach Success Rate as per formula below
The KPI cannot be measured as end-user perceived ratio on certain mobiles,
which do not indicate network attachment status to the end-user.

Assumptions, UE status before measurement: EMM-DEREGISTERED


pre-conditions
UE locked to LTE network.

Formula
(logical)

Copyright Nokia Siemens Networks 2011 Company Confidential Page 18 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Message flow, Trigger events on UE R interface (see Figure 2Figure 2):


trigger points
Success: 21. RRC DIRECT TRANSFER message with L3 NAS: Attach
Complete in its body sent by the UE
Attempts: 1. RRC CONNECTION REQUEST messages (with cause Attach
Request) sent by UE

Related E2E KPIs Attach Time, Service Request (EPS) Success Rate.

3.1.6 Service Request (EPS) Time, UE Initiated

Copyright Nokia Siemens Networks 2011 Company Confidential Page 19 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Service Request Time (EPS Bearer Setup), UE Initiated, Control Plane [ms]

Definition Time taken by the LTE network to setup an EPS bearer on request by the UE.
The EPS bearer can be new (dedicated) or an existing one (e.g. the default EPS
bearer). The latter is needed to re-assign Uu radio and S1 bearer resources to
the existing EPS bearer of a previously idle UE.
The EPS bearer has to be created before IP packets can be exchanged.
The UE Initiated EPS Bearer Setup Time is the interval between the submission
of the message RRC CONNECTION REQUEST: Service Request (refer to
[3GPP36.331]) and the reception of the RRC CONNECTION
RECONFIGURATION response on the UE. The Bearer Setup procedure is
based on the Service Request procedure according to [3GPP23.401] and
[3GPP36.300].

Measurement 1 LTE UE/terminal with FMT


methodology Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and
enough samples are collected (re-drive route if necessary)
Verify the EPS Bearer Setup Time
Note below that UE pre-status is EMM-REGISTERED so this KPI excludes
the Attach time

Assumptions, pre- UE status before measurement: EMM-REGISTERED and idle initial state.
conditions
UE locked to LTE network.

Formula (logical)

Message flow, The message flow diagram is shown in Figure 3Figure 3 and Figure 4Figure 4.
trigger points The user initiated EPS bearer setup time is measured between the trigger points
BUE and EUE on the UE R interface:
Begin: RRC CONNECTION REQUEST(NAS: Service Request) sent by UE
End: corresponding RRC CONNECTION RECONFIGURATION
COMPLETE message or first PDU in UL sent by the UE (refer to
[3GPP24.008]).

Related E2E KPIs a) Attach Time.


b) No 3GPP defined KPI for EPS bearer setup found (exists).

Detailed EPS bearer setup procedure is shown below:

Copyright Nokia Siemens Networks 2011 Company Confidential Page 20 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

LTE-Uu S1- MME S11 S6a

UE eNodeB MME S-GW / P-GW HSS


UE EMM-REGISTERED
DL data
and ECM-IDLE D
L2 SCTP: DL DATA
Paging Procedure NOTIFICATION
BN

RRC CONNECTION Begin measurement


BUE
REQUEST

RRC CONNECTION
SETUP Setup SRB1

RRC CONNECTION
SETUP COMPLETE

(L3 NAS: SERVICE


REQUEST) S1-AP: INITIAL UE
MESSAGE

(L3 NAS: SERVICE L2 SCTP: Create Bearer


REQUEST) Request Activate EPS
bearer setup in
L2 SCTP: Create Bearer ePC
Response

If authentication MAP: Send Authentication Info


vectors are not
available in MME,
authenticate UE MAP: Send Authentication Info Ack

S1-AP: UL NAS
TRANSPORT If authentication
timer has expired,
RRC: DL INFORMATION (L3: AUTHENTICATION setup security
TRANSFER AND CIPHERING association for user
REQUEST) plane RB
(L3: AUTHENTICATION
AND CIPHERING
REQUEST)

RRC: UL INFORMATION
TRANSFER
S1-AP: DL NAS
(L3: AUTHENTICATION TRANSPORT
AND CIPHERING
RESPONSE) (L3: AUTHENTICATION
AND CIPHERING
RESPONSE)

Figure 3 EPS Bearer Setup Procedure, Part I

Copyright Nokia Siemens Networks 2011 Company Confidential Page 21 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

LTE-Uu S1- MME S11 S6a


UE eNodeB MME S-GW / P-GW HSS

S1-AP: INITIAL
T1
CONTEXT PM Counter:
EPS_SETUP_ATT
SETUP REQUEST
S1-AP: INITIAL
CONTEXT

SETUP FAILURE

RRC SECURITY MODE


COMMAND
Setup user
RRC CONNECTION plane RB
RECONFIGURATION

RRC SECURITY MODE


COMMAND COMPLETE
RRC CONNECTION
EUE RECONFIGURATION
End measurement
COMPLETE

Means Delay DL
First UL PDU
Packet Notification
Request to stop/
S1-AP: INITIAL enable DL data
CONTEXT PM Counter: transfer
T2
EPS_SETUP_SUCC
SETUP COMPLETE
L2 SCTP: Update Bearer
Request
L2 SCTP: Update Bearer EN
Response

First DL PDU

Figure 4 EPS Bearer Setup Procedure, Part II

The service request times for UE initiated and for network initiated EPS bearer setups are measured
between the trigger points BUE and EUE, and between BN and EN, respectively. These trigger points are
colored green in the diagram.

3.1.7 Service Request (EPS) Success Rate

Service Request (EPS) Success Rate, Control Plane [%]

Copyright Nokia Siemens Networks 2011 Company Confidential Page 22 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Definition Ratio between successfully established EPS bearers compared to the overall
number of EPS bearer establishment attempts. It corresponds to the
probability that a user or the LTE network can establish an EPS bearer at any
moment in time.
Requests which are terminated by timer expiry (due to the unavailability of
some LTE resource) are considered as unsuccessful attempts. Authentication
errors (requests rejected by the MME) are included in the total number of
failures.
Only the first RRC CONNECTION REQUEST is to be considered, since
counting retries of the same message would increase the overall number of
bearer establishment attempts, and thus reduce the success ratio.

Measurement 1 LTE UE/terminal with FMT


methodology Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and
enough samples are collected (re-drive route if necessary)
Verify the EPS Bearer Setup Success Rate
Note below that UE pre-status is EMM-REGISTERED so this KPI excludes
the Attach time

Assumptions, pre- UE status before measurement: EMM-REGISTERED and idle initial state.
conditions
UE locked to LTE network.

Formula (logical)

Message flow, trigger The message flow diagram is shown in Figure 3Figure 3 and Figure 4Figure
points 4. The trigger points for UE initiated EPS bearer setup success ratio on the R
interface are:
Success: RRC CONNECTION RECONFIGURATION COMPLETE
message sent by the UE, see EUE on Figure 4Figure 4.
Total: an attempt is the sending of RRC CONNECTION REQUEST with
appropriate establishment cause by the UE, see BUE on Figure 3Figure 3.

Related E2E KPIs Service Request (EPS) Time, UE Initiated.


For initial EPS bearers, refer to KPI Attach Success Rate.

3.1.8 Service (EPS Bearer) Drop Rate

Service (EPS Bearer) Drop Rate, Control Plane [%]

Definition Ratio between abnormally released bearers, and the overall number of
established EPS bearers. An abnormal release is defined as any EPS bearer

Copyright Nokia Siemens Networks 2011 Company Confidential Page 23 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

termination, which was not triggered by the mobile user (from UE side). Thus, it
reflects the probability that an established bearer gets aborted due to insufficient
network resources.
Dropping the bearer gets visible to the end-user if an application service is
actively using it. If the application automatically re-establishes the bearer, it
remains unnoticed by the user.

1 LTE UE/terminal with FMT


Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and
Measurement enough samples are collected (re-drive route if necessary)
methodology Verify the EPS Bearer Drop Rate
Note below that UE pre-status is EMM-REGISTERED so this KPI excludes the
Attach time.

Assumptions, pre- UE status before measurement: EMM-REGISTERED and idle initial state.
conditions UE locked to LTE network.

Formula (logical)

Trigger points on UE R interface:


Success: submission of the message RRC CONNECTION
RECONFIGURATION COMPLETE by the UE, refer to [3GPP36.331]
Message flow, trigger Drop: successful establishments minus terminations by the user, i.e. the UE
points submitting a L3 DETACH REQUEST (refer to [3GPP24.008]) carried within
the RRC message UL NAS TRANSPORT (refer to [3GPP36.331]).
For the calculation of a network initiated EPS bearer drop rate, EPC initiated L3
DETACH REQUEST-s are to be considered.

Related E2E KPIs Service Request (EPS) Success Rate.

3.1.9 Handover Procedure Time

Handover Procedure Time, Control Plane [ms]

Definition It denotes the total time needed for the handover procedure as seen by the
UE. It begins by receiving a Handover Command from the source eNB and
ends by sending the Handover Confirm response to the target eNB by the UE.
The value of the KPI also depends of the handover scenario. The following
HO scenarios are distinguished (though not directly seen by the UE):
intra- and inter-frequency HO
inter eNB HO via X2/S1 interface

Copyright Nokia Siemens Networks 2011 Company Confidential Page 24 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Measurement 1 LTE UE/terminal with FMT


methodology Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and
enough samples are collected (re-drive route if necessary)
Verify HO time for the different HOs
The handover time shall be measured on the UE R interface.

Assumptions, pre- Since Inter-RAT HO is not part of RL15 the only HO scenarios considered are
conditions intra LTE. Therefore, the LTE UE must be locked to LTE network.
UE status before measurement: registered and connected.

Formula (logical) Average value from all measured samples.

Message flow, trigger The HO time is seen by the UE as the elapsed time between:
points
Start: UE receives Handover Command in the body of an RRC
Connection Reconfiguration request from source eNB.
Stop: UE sends Handover Confirm response to target eNB in an RRC
Connection Reconfiguration Complete message.

Related E2E KPIs a) (LTE) Service Interrupt Time (HO), Handover Success Rate
b) No 3GPP defined KPI for HO procedure time found.

3.1.10 Handover Success Rate

Handover Success Rate, Control Plane [%]

Definition The Handover Success Rate is the ratio between successfully executed
(committed) HO procedures and the number of all Handover attempts.

Measurement 1 LTE UE/terminal with FMT


methodology Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and
enough samples are collected (re-drive route if necessary)
Verify the Handover Success Rate
The measurement is to be executed for different HO scenarios.

Assumptions, pre- LTE terminal must be locked into the LTE network.
conditions

Formula (logical)

Copyright Nokia Siemens Networks 2011 Company Confidential Page 25 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Message flow, trigger Trigger points on eNB X2/S1 interfaces:


points
Total #: Handover Requests sent by Source eNB to Target eNB (HO via
X2), or sent by MME to Target eNB(HO via S1).
Success #: UE Context Release message received by Source eNB from
Target eNB(HO via X2), or UE Context Release Command received by
Source eNB from MME(HO via S1).
In case of intra eNB HO procedures, no Handover Request is sent. Both
trigger points are counted by the Source eNB internally.

Related E2E KPIs Handover Procedure Time

3.1.11 (LTE) Round Trip Time

Copyright Nokia Siemens Networks 2011 Company Confidential Page 26 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Round Trip Time (RTT), User Plane [ms]

Definition RTT in UL is the interval between sending a datagram by the UE & receiving
the corresponding reply from an IP peer entity connected to the Gi interface
of the P-GW.
RTT in DL is the interval between sending a datagram to the UE & receiving
the corresponding reply by the IP host (peer entity).

Measurement Stationary users in different cell positions, which are uniformly distributed
methodology across the cell.
RTT is measured with the Ping application between the UE and an IP host
(peer entity). As average RTT figure, the output of the Ping application is
used (e.g. MS-DOS >ping -n count -l size host name). Size of ICMP
packets: 32, , MTU bytes.
Ping shall be executed at least 100 times to reduce the impact of the first
ICMP message, which triggers the setup of radio bearer establishments in
UL and DL directions.
Record the average RTT shown on the screen after the application has run.
Repeat the test 5 times in each location under different SINR.
Note: Ping tests can also be carried out as part of a drive test sequence in
which case, it is not possible to guarantee certain SINR levels

Assumptions, pre- UE status before measurement: registered and connected. An IP address is


conditions assigned to the UE.
The IP peer entity shall be located as close as possible to the SGi interface.
Usually, 1 hop away the SAE-GW. Additional delays between the P-GW
and the IP host shall be avoided or minimized.

Formula (logical)

Message flow, trigger Trigger points both from UE, and from IP peer entity view:
points
Start: ICMP ECHO REQUEST
Stop: ICMP ECHO REPLY

Related E2E KPIs No 3GPP defined KPI for LTE RTT found.

3.1.12 (LTE) Single User Throughput

(LTE) Single User Throughput, User Plane [Mbps]

Definition The metric describes the data speed available to one user of the LTE
network on UDP/IP level. It is given as the maximum (95%-ile) value that can
be observed over a short period of time (e.g. of 1s) and as a mean value that

Copyright Nokia Siemens Networks 2011 Company Confidential Page 27 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

characterizes longer data transfer periods (minutes). Its value distribution


over the radio cell is given as a function of the SINR. The maximum value is
often referred to in the literature as instantaneous "Peak Throughput" that is
achieved in optimal radio conditions. The
user throughput can be given for a single user active in the cell (single user
data rate), or to one of several concurrently active users.

Measurement Stationary and mobile user covering all kinds of cell positions: near, medium,
methodology far, including different HO scenarios. The data rate is measured in UL/DL
direction with UDP/IP traffic over a time period of several minutes.

Assumptions, pre- The IP host used as traffic generator (e.g. with Iperf) should be connected
conditions directly to the SGi interface. Additional delays and bandwidth restrictions
between the S/P-GW and the server need to be excluded.

Formula (logical)

Message flow, trigger Trigger points on the UE R interface:


points
UL: UE sending UDP/IP packets to the IP peer entity.
DL: IP peer entity sending UDP/IP packets to the UE.

Related E2E KPIs Cell Throughput, (FTP) Single User Throughput, (RB) Single User
Throughput.

The Single User Throughput KPI has been defined on E2E network level to verify if the capacity of large
non-GBR EPS bearers (QoS profile with large maximum bit rates) can in fact be exploited by IP based
services. The same holds for the E-RAB and RB bearers being part of the EPS. Especially, the maximum
achievable throughput of the RB bearer is interesting, since it is the critical resource of the LTE network.
For this reason, the Peak Throughput is defined as a separate KPI on the RB level (see KPI: (RB) Single
User Throughput), too. Because of this relationship (containment) between EPS, E-RAB and RB, user
throughput KPIs can be verified with the same series of measurements. The KPIs are interesting as
values calculated over the full duration of long data transfers (~minutes). Of equal interest are peak
values observed for a short time period only (~seconds). Please note that throughput KPIs have also
been defined for application services, e.g. Single User Throughput KPI of the Data Download / Upload
service, but these do not characterize the capability of the LTE network alone, but also depend on the
application service and its protocols.

3.1.13 (LTE) Service Interrupt Time (HO)

(LTE) Service Interrupt Time (HO), User Plane [ms]

Definition Service interrupt is the discontinuity (increased packet inter-arrival time) of an


IP packet flow caused by hand-over. It is the interval between the last
sent/received user IP packet of a continuous UL/DL data stream in the old cell,

Copyright Nokia Siemens Networks 2011 Company Confidential Page 28 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

and the first sent/received user IP packet in the new cell (user plane break).
The value of the KPI also depends of the handover scenario.

Measurement 1 LTE UE/terminal with FMT


methodology
Set FMT to generate data calls to the data server automatically
Start drive test and continue driving until drive route is completed and
enough samples are collected (re-drive route if necessary)
Verify the Service Interrupt Time
The measurement is to be executed for different HO scenarios: intra eNB, inter
eNB via X2/S1 etc.
Only mobile user scenarios are considered. FTP upload/download data
transfer, or real-time media flow during the handover.

Assumptions, pre- It is assumed that the X2 interface is enabled between the source and target
conditions eNBs (for inter eNB HO scenarios).
LTE terminal must be locked into the LTE network to measure the Service
Interrupt Time.

Formula (logical)

Message flow, trigger The trigger points for service interrupt time (from UE point of view):
points Start: UE sends/receives last packet to/from source eNB
Stop: UE sends/receives first packet to/from target eNB.

Related E2E KPIs Handover procedure time.

3.1.14 (RB) Single User Throughput

(RB) Single User Throughput, User Plane [Mbps]

Definition The metric describes the RB throughput achievable by one user. It can be
given as single user throughput if only one user is active in the cell, or as multi-
user throughput for a given number of concurrently active users. The user
throughput distribution over the cell is given as function (CDF) of the radio
conditions characterized by the SINR.
The time variation of the KPI value in a given cell position is given with its
maximum (95%-ile) (also referred to as instantaneous Peak User
Throughput), and with its average (mean) value.

Measurement Stationary and mobile user covering all kinds of cell positions: near, medium,
methodology far, including different HO scenarios.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 29 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

The eNB assigns all resources (resource blocks) to this user, except the
resource blocks used for signaling and controlling. With the help of trace
analysis, PDCP, RLC, MAC and PHY layer throughputs can be defined.
With and w/o concurrency in UL / DL traffic.
The mean user data rate is calculated over a few minutes, peak data rate
values are averaged over short periods of time (e.g. 1s).

Assumptions, pre- All transport bearers between the UE and IP peer entity should have higher
conditions capacity than the radio link in order to avoid bandwidth bottlenecks.

Formula (logical)

Message flow, trigger Trigger events on the eNB Uu interface or the UE R interface:
points
UL: eNB receiving PDUs, or UE sending PDUs
DL: eNB sending PDUs, or UE receiving PDUs
Measured at the upper SAPs of the protocols PDCP, RLC, MAC or PHY.

Related E2E KPIs (LTE) Single User Throughput, Cell Throughput.

3.1.15 Cell Throughput

(RB) Single User Throughput, User Plane [Mbps]

Definition The metric shows the sustainable aggregate throughput of the cell (in UL/DL)
available to n stationary users distributed uniformly in the cell and running a
typical mix of applications. The cell throughput is the sum of all bits
transported in all resource blocks carrying PDUs (i.e. bits in ULSCH/DL-SCH
transport blocks) during one second.
The cell capacity is also given as peak value (called peak cell capacity, or
throughput), which is defined as the aggregate throughput of n users all
located in best radio conditions.
The cell throughput value is defined here on PHY level, but could be given for
other protocol levels (UDP/IP, PDCP, RLC, MAC), too. When the (peak,
average) cell throughput is expressed on UDP/IP level, it corresponds to the
(peak, mean) user throughput value at comparable radio conditions.

Measurement Stationary users uniformly distributed in the cell. UE categories according to


methodology application mix. Peak value measured with one user in LoS cell position using
UDP/IP load to approach the full buffer condition.
Measured with UDP/IP traffic on PHY layer (UL-SCH/DL-SCH transport block
bits) over several minutes. With the help of trace analysis PDCP, RLC and
MAC layer values can also be given.

Assumptions, pre- All IP transport bearers (S1, S5/S8, SGi) between the UE and IP peer entity

Copyright Nokia Siemens Networks 2011 Company Confidential Page 30 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

conditions should have higher bandwidth than the cell capacity to avoid bandwidth
bottlenecks.

Formula (logical)

Message flow, trigger Trigger events on the eNB Uu interface or the UE R interface:
points
UL: eNB receiving PDUs, or UE sending PDUs
DL: eNB sending PDUs, or UE receiving PDUs

Related E2E KPIs (RB) Single User Throughput.

3.2 Test Conditions and Target Values

Field KPI Target Values Comments


In case UL/DL conf=2:2(#1) and special
Single User UL subframe conf=10:2:2(#7): 15.4M
Peak
Throughput In case UL/DL conf=3:1(#2) and special CMCC Hangzhou TD-LTE trial:
subframe conf=10:2:2(#7): 7.5M *Cat 3 UE is considered and MCS20 is the
In case UL/DL conf=2:2(#1) and special max MCS in UL for RL15.
subframe conf=10:2:2(#7): *PUCCH overhead is 4RB and PDCCH
TM3: 59.6M overhead is 3 OFDM symbols.
Single User DL TM7: 34M *L3 TCP throughput is considered.
Peak *Network is unloaded.
Throughput In case UL/DL conf=3:1(#2) and special *System bandwidth is 20M.
subframe conf=10:2:2(#7):
TM3: 80.7M
TM7: 40.3M
In case UL/DL conf=2:2(#1) and special
subframe conf=10:2:2(#7):
0% load: 14.5M
Throughput
50% load: 13.3M CMCC Hangzhou TD-LTE trial:
Cell UL
70% load: 15.6M *Cat 3 UE is considered and MCS20 is the
Average
max MCS in UL for RL15.
Throughput
In case UL/DL conf=3:1(#2) and special *PUCCH overhead is 4RB and PDCCH
subframe conf=10:2:2(#7): overhead is 3 OFDM symbols.
0% load: 7.6M *L3 TCP throughput is considered.
50% load: 6.5M *System bandwidth is 20M.
In case UL/DL conf=2:2(#1) and special *20 UEs distributed by 2:4:8:6.
subframe conf=10:2:2(#7): *DL MIMO is TM3/TM7 adaptive and UL is
0% load: 35.6M SIMO.
50% load: 21.77M *eNB SW version is different for each
Cell DL
70% load: 24.82M neighbor load: RL15_2_CD1.0 for 0% load,
Average
RL15_2_CD2.0 for 50% load and
Throughput
In case UL/DL conf=3:1(#2) and special RL15_2_CD3.0 for 70% load.
subframe conf=10:2:2(#7):
0% load: 40M
50% load: 25.1M
Voentelecom Russia TD-LTE Trial:
Latency In case UL/DL conf=2:2(#1) and special *Radio condition is classified as:
(Control Attach Latency subframe conf=10:2:2(#7): Good: CRS SINR >25
Plane) 0% load: 168ms Average: 10<CRS SINR<25
Poor: 0<CRS SINR<10

Copyright Nokia Siemens Networks 2011 Company Confidential Page 31 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

In case UL/DL conf=2:2(#1) and special


Idle to Active
subframe conf=10:2:2(#7):
Latency
50% load: 106ms
In case UL/DL conf=2:2(#1) and special
subframe conf=10:2:2(#7):
CMCC Hangzhou TD-LTE trial:
0% load and single UE:
*Radio condition is classified as:
for 32 bytes payload: 26ms
Very Good: CRS SINR > 22dB
for 1500 bytes payload: 30ms
Good: 15dB< CRS SINR < 20dB
Latency Medium: 5dB < CRS SINR < 10dB
Ping Latency 50% load and single UE:
(User Plane) Poor: -5dB < CRS SINR < 0dB
for 32 bytes payload: 27ms
*Pre-scheduling is off for 'Ping Latency' test,
for 1500 bytes payload: 31ms

50% load and 3 UEs:


for 32 bytes payload: 28ms
for 1500 bytes payload: 38ms
Handover
Latency
In case UL/DL conf=2:2(#1) and special
(Control Plane, CMCC Hangzhou TD-LTE trial:
subframe conf=10:2:2(#7):
intra-eNB or *Drive route includes both intra-eNB
0% load: 32ms
inter-eNB via handover and inter-eNB handover (via X2)..
X2) *C-plane latency is counted as MAC layer
Mobility
Handover latency, and U-plane latency is counted as
Latency PDCP layer latency, which is not strictly
In case UL/DL conf=2:2(#1) and special
(User Plane, complied with CMCC's test specification.
subframe conf=10:2:2(#7):
intra-eNB or
0% load: 61ms
inter-eNB via
X2)
In case UL/DL conf=2:2(#1) and special CMCC Hangzhou TD-LTE trial:
Attach subframe conf=10:2:2(#7): *2.6G frequency band with 20M system
Success Rate 0% load: 100% bandwidth.
50% load: 99.17% *Drive test, speed is about 30Km/h.
*Attach SR is defined as:
RRC In case UL/DL conf=2:2(#1) and special
#(Attach complete) / #(UE Power on).
Connection subframe conf=10:2:2(#7):
Accessibility *RRC Connection Establishment SR is
Establishment 0% load: 99%
defined as:
Success Rate 50% load: 97.84%
#(RRCConnectionReconfigurationComplete)
In case UL/DL conf=2:2(#1) and special / #(Data service request)
Handover subframe conf=10:2:2(#7): *Handover SR is defined as:
Success Rate 0% load: 100% #(RRCConnectionReconfigurationComplete
50% load: 98.37% containing Handover Confirm to Target
eNB) / #(Handover Request)
In case UL/DL conf=2:2(#1) and special *DR is defined as:
subframe conf=10:2:2(#7): #(RRCConnectionRelease and/or without
Reliability Drop Rate
0% load: 2% application-layer throughput for more than
50% load: 3.42% 10s) /
#(RRCConnectionReconfigurationComplete)

4 Network KPIs

4.1 Network KPI Definitions


The defined network KPI lists do not present by any means a complete set of KPIs to be used in a
customer case. It is the project team/CBT responsibility to define and agree with the customer the final list
of KPIs to be used in each and every case.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 32 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

The most interested network KPIs are listed below. Formula is indicated with the counter ID (e.g.
M8007C1) which is unique for each counter instead of the counter name. More information about these
KPIs and other network KPIs can be found in RISE database (refer to RL20 instead).
Table 2 List of Network KPIs

KPI Name Formula

Data Radio Bearer Setup Success Ratio


100*sum(M8007C1) / sum(M8007C0)
LTE_5003a

Radio Bearer Drop Ratio 100*sum(M8007C5+M8007C6) / sum(M8007C3+


LTE_5004a M8007C4+M8007C5+M8007C13+M8007C6)

RRC Connection Establishment Success Rate 100*sum(M8013C5) / sum(M8013C17+M8013C18+


LTE_5218a M8013C19+M8013C20)

E-RAB Setup Success Ratio


100*sum(M8006C1) / sum(M8006C0)
LTE_5017a

E-RAB Drop Ratio 100*sum(M8006C12+M8006C14+M8006C13) /


LTE_5025a sum(M8006C6+M8006C7+M8006C8+M8006C9)

HO Success Ratio (intra eNB)


100*sum(M8009C7) / sum(M8009C6)
LTE_5035a

HO Success Ratio (inter eNB)


100*sum(M8014C7) / sum (M8014C6)
LTE_5048a

Cell Availability Ratio


100*sum(M8020C3) /sum(M8020C6)
LTE_5750a

4.2 Test Conditions and Target Values


TBD

Copyright Nokia Siemens Networks 2011 Company Confidential Page 33 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

5 Appendix I: Trial Network KPIs

5.1 CMCC Large Scale Trial

5.1.1 Test Environment and Baseline Configuration


Test environment of CMCC large scale trial in Hangzhou is illustrated as below. There are 24 eNBs and
50 cells in total. Selected route for DT is also shown.

Figure 5 Test Environment of CMCC Large Scale Trial in Hangzhou

Baseline configuration is shown as below.


Table 3 Baseline Configuration of CMCC Large Scale Trial in Hangzhou

Parameter Baseline Configuration


Morphology Typical Urban
Carrier Frequency 2.6G
System Bandwidth 20M

Copyright Nokia Siemens Networks 2011 Company Confidential Page 34 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Normal CP
UL/DL Configuration #1 (2:2) Special Subframe
Frame Structure Configuration #7 (10:2:2)
CFI 3
DL: TM3/TM7 Adaptive
Transmission Mode UL: SIMO
Enable PUCCH, PUSCH, and Sounding RS Power
Power Control Control
HARQ Enable
AMC Enable
eNB Transmission
Power 85W
Random Access Contention-based

UE location is selected based on CRS SINR according to the following rules:


*Very Good: >22dB
*Good: 15dB~20dB
*Medium: 5dB~10dB
*Poor: -5dB~0dB

5.1.2 Field Results


Note that interference/load is generated by real UEs distributed evenly in medium point and poor point,
performing UL/DL FTP traffic at the same time, which results in 50%/70%/100% load in DL and
5dB/8dB/11dB IoT in UL respectively.
Table 4 Field Results of CMCC Large Scale Trial in Hangzhou

KPI Name Test Condition Field Results Comments

Attach SR Drive test(~30km/h), Without load: 100% Defined as:


without load or under 50% load: 99.17% Count(Attach complete)/Count(UE
50% load. Power on)

RRC Drive test(~30km/h), Without load: 99% Defined as:


Connection without load or under 50% load: 97.84% Count(RRCConnectionReconfigurati
Establishment 50% load. onComplete)/Count(data service
SR preambleTransMax=3. request)

Paging SR Drive test(~30km/h), Without load: 98% Defined as:


without load or under 50% load: 95.11% Count(Initial UE Message, NAS:
50% load. Service Request)/Count(Paging)
preambleTransMax=3.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 35 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

DR Drive test(~30km/h), Without load: 2% Defined as:


without load or under 50% load: 3.42% Count(RRCConnectionRelease
50% load. and/or without application-layer
throughput for more than
10s)/Count(RRCConnectionReconfi
gurationComplete)

Handover SR Drive test(~30km/h), Without load: 100% Defined as:


without load or under 50% load: 98.37% Count(RRCConnectionReconfigurati
50% load. onComplete containing Handover
Confirm to Target
eNB)/Count(Handover Request)

Long-term Drive test(~30km/h), Without load: 23.3M in DL and


Retainability without load or under 10.5M in UL
and Average 50% load. 50% load: 4.6M in DL and 5.9M in
Throughput UL

Single User Stationary test and When UL/DL conf=2:2 and S Cat3 UE and physical layer TP is
Peak TP without load. subframe conf=10:2:2: used.
DL TM is TM3 or TM7, UL: 15.27M (16.10M) Figures in the bracket are
and UL is SIMO theoretical peak TP.
DL TM3: 57.70M(59.57M)
MCS20 is the maximum available
DL TM7: 32.87M(34.02M) MCS in UL for RL15.

When UL/DL conf=3:1 and S


subframe conf=10:2:2:
UL: 7.44M (7.98M)
DL TM3: 76.89M(79.98M)
DL TM7: 39.99M(45.18M)

Cell TP Stationary test, without When UL/DL conf=2:2 and S Cat3 UE and L3 TP is used.
load or under 50%/70% subframe conf=10:2:2,
load. MCS20 is the maximum available
*without load (RL15_2_CD1.0) MCS in UL for RL15.
20 UEs are distributed
evenly: 2:4:8:6. UL only:14.53M

TM3/TM7 adaptive is DL only: 35.56M


used in DL and SIMO for UL&DL: 12.34M/27.51M
UL.
*50% load (RL15_2_CD2.0)
UL only:13.27M
DL only: 21.77M
UL&DL: 13.81M/20.46M
*70% load (RL15_2_CD3.0)
UL only:15.62M
DL only: 24.82M

Copyright Nokia Siemens Networks 2011 Company Confidential Page 36 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

UL&DL: 13.51M/23.67M

When UL/DL conf=3:1 and S


subframe conf=10:2:2:
*without load (RL15_2_CD1.0)
UL only:7.55M
DL only: 40.02M
UL&DL: 6.13M/26.71M
*50% load (RL15_2_CD2.0)
UL only:6.48M
DL only: 25.09M
UL&DL: 5.22M/22.82M

*Ping Latency, also RTT latency


Single user ping latency, without load
Max Min Avg
Payload Test Condition
Latency Latency Latency
Very Good 30ms 14ms 26ms
Good 30ms 14ms 26ms
32 Bytes
Medium 41ms 14ms 25ms
Poor 42ms 14ms 26ms
Very Good 44ms 18ms 31ms
Good 45ms 21ms 29ms
1500 Bytes
Medium 44ms 18ms 30ms
Poor 102ms 18ms 31ms

Single user ping latency, with 50% load


Test Max Min Avg
Payload
Condition Latency Latency Latency
Very
Good
40ms 24ms 27ms

32 Bytes Good 37ms 14ms 26ms


Medium 51ms 17ms 26ms
Poor 51ms 24ms 29ms
Very
Good
41ms 26ms 31ms

1500 Bytes Good 41ms 18ms 30ms


Medium 51ms 19ms 30ms
Poor 61ms 23ms 33ms

Copyright Nokia Siemens Networks 2011 Company Confidential Page 37 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Multiple users ping latency, with 50% load


Test Max Min Avg
Payload
Condition Latency Latency Latency
Good 43 24 28
32 Bytes Medium 49 18 29
Poor 43 24 28
Good 57 26 37
1500 Bytes Medium 69 24 38
Poor 66 31 39

*Control-plane Latency, also idle-to-active latency


Control-plane latency is defined as the duration between MSG1 of random access and the
RRCConnectionConfigurationComplete sent by UE to eNB.
With 50% load:
Test Min Max Avg
Condition Latency Latency Latency
Very
100 110 105
Good
Good 100 110 107
Medium 100 120 107
Poor 100 130 107

*Handover Latency
Handover latency of control-plane is defined as the duration between MeasurementReport sent by UE to
source eNB and the RRCConnectionConfigurationComplete sent by UE to target eNB.
Handover latency of user-plane is defined as the duration between UE/source eNB receives the last
packet from source eNB/UE and UE/target eNB receives the first packet from target eNB/UE in DL and
UL respectively.
Without load:
Min Max Avg
Latency Latency Latency
Control
24 53 32
Plane
User Plane
45 155 61
UL
User Plane
45 155 61
DL

Copyright Nokia Siemens Networks 2011 Company Confidential Page 38 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

5.2 Bharti Airtel Trial

5.2.1 Test Environment and Baseline Configuration


There are 4 sites for Bharti Airtel TD-LTE Trial, which are shown as below.

Figure 6 Test Environment of Bharti Airtel TD-LTE Trial

Baseline configuration is listed as below.


Table 5 Baseline Configuration of Bharti Airtel TD-LTE Trial

Parameter Baseline Configuration

Frequency Band 2.3G

Carrier Bandwidth 10M/20M

Maximum Tx power DL 3 x 20 Watts


(Pout @ eNodeB)

Antenna System Dedicated Antenna System

DL MIMO Configurations 2x2 TM2/TM3

Copyright Nokia Siemens Networks 2011 Company Confidential Page 39 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Test UEs Class 3

Test UE Mobility Stationary and drive test

5.2.2 Field Results


*Handover Latency: DL TCP with speed 30km/h, unloaded network.
Handover latency for Intra-eNB case is defined as:

Total latency for intra-eNB handover is 65.31ms.

Handover latency for inter-eNB via X2 is defined as:

Copyright Nokia Siemens Networks 2011 Company Confidential Page 40 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Total latency for inter-eNB HO via X2 interface is 90.07ms.

*Service Request Time, also idle-to-active time


UE location is selected based on following radio conditions:
-Near: CRS SINR >25dB
-Middle: 12dB<CRS SINR <15dB
-Far: 2dB<CRS SINR<5dB

Copyright Nokia Siemens Networks 2011 Company Confidential Page 41 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Field results are:

Note that, herein definition of idle-to-active latency, which is the duration between ServiceRequest and
RRCConnectionSetupComplete, is not correct.
*RTT, also ping latency
Field results(pre-scheduling is on) are:

*Single User Throughput


Field results are:

In above figure, Config-1 means UL/DL configuration 1 with special subframe configuration 7, and
Config-2 means UL/DL configuration 2 with special subframe configuration 7.

5.3 Maxis Aircel Trial

5.3.1 Test Environment and Baseline Configuration


There are 3 sites for Maxis Aircel TD-LTE Trial, which are shown as below.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 42 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Figure 7 Test Environment of Maxis Aircel TD-LTE Trial

Baseline configuration is listed as below.


Table 6 Baseline Configuration of Maxis Aircel TD-LTE Trial

Parameter Baseline Configuration

Frequency Band 2.3G

Carrier Bandwidth 20M

Maximum Tx power DL 3 x 20 Watts


(Pout @ eNodeB)

Frame Configuration UL/DL Configuration 2 (3:1)


Special Subframe Configuration 7 (10:2:2)

DL MIMO Configurations 2x2 TM2/TM3

Test UEs Class 3

Test UE Mobility Stationary and drive test

Copyright Nokia Siemens Networks 2011 Company Confidential Page 43 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

5.3.2 Field Results


*Peak Throughput
Single user peak throughput with frame configuration 2-7(3:1 and 10:2:2) is: max=80.6M, avg=71.6M.

*Ping Latency (pre-scheduling is on)


Payload=32Bytes, min=12ms, avg=14ms, and max=17ms.

*Cell Throughput
10 UEs all located in good/medium/poor location, and neighbour load is 0% or 70%. Test results are:

*Handover Latency
In case of intra-eNB handover, test results are:

Copyright Nokia Siemens Networks 2011 Company Confidential Page 44 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

In case of inter-eNB handover via X2 interface, test results are:

In case of inter-eNB handover via S1 interface, test results are:

Copyright Nokia Siemens Networks 2011 Company Confidential Page 45 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

5.4 Voentelecom Russia

5.4.1 Test Environment and Baseline Configuration


There are 3 sites for Voentelecom TD-LTE trial, and the baseline configuration is as below.
Table 7 Baseline Configuration of Voentelecom TD-LTE Trial

Parameter Baseline Configuration

Frequency Band 2.3G

Carrier Bandwidth 20M

Maximum Tx power DL 3 x 20 Watts


(Pout @ eNodeB)

Frame Configuration UL/DL configuration 1 (2:2) and 2 (3:1)

DL MIMO Configurations 2 x 2 TM4

Copyright Nokia Siemens Networks 2011 Company Confidential Page 46 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

Test UEs Class 3

Test UE Mobility Stationary and drive test

5.4.2 Field Results


*Single user throughput
UDP throughput with UL/DL configuration 1: 55.8M in DL and 15.2M in UL.
UDP throughput with UL/DL configuration 2: 80.2M in DL and 7.4M in UL.
TCP throughput with UL/DL configuration 2: 69.3M in DL

*Peak cell throughput


UDP throughput with UL/DL configuration 2: 111M (55.8M+55.2M) in DL and 19.3M (10.8M+8.5M) in UL

*Attach Latency
UE location is selected based on following radio conditions:
-Good: CRS SINR >25
-Average: 10<CRS SINR<25
-Poor: 0<CRS SINR<10

Copyright Nokia Siemens Networks 2011 Company Confidential Page 47 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

*Handover Latency:

Note that inter-eNB HO is based on X2 interface.

Copyright Nokia Siemens Networks 2011 Company Confidential Page 48 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

*RTT Latency, also ping latency:

5.5 Appendix II: Early Network KPIs

5.5.1 STC Saudi


TBD

5.5.2 Sky Brazil


TBD

6 References
[1] <e-UTRAN Field and Network KPI list for RL10>, Lorena Serna
https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/D411459148
[2] <E2E Field Network Performance - LTE, KPI Definitions>, Andras Balazs

Copyright Nokia Siemens Networks 2011 Company Confidential Page 49 (50)


TD-LTE KPI Optimization Guidelines for RL15TD Version: 1.1

https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/D428312882
[3] <E2E Field Network Performance - KPI Targets for RL15TD>, Andras Balazs
https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/D428312884
[4] <Cluster Acceptance Guidelines for TD-LTE >, Ric Kong
https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/D435855209
[5] <TDD-LTE Optimization Guidelines>, Lorena Serna
https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/439397953
https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/D415266561
[6] <FDD-LTE Optimization Guidelines>, Lorena Serna
https://sharenet-ims.inside.nokiasiemensnetworks.com/Open/D415266561
[7] NWS-TDD-LTE Wiki
https://twiki.inside.nokiasiemensnetworks.com/bin/view/LTETechSup/TdLteHome
[87] RL15 Trials and Pilots
https://sharenet-
ims.inside.nokiasiemensnetworks.com/livelink/livelink?func=ll&objId=421758333&objAction=Browse
[89]

Copyright Nokia Siemens Networks 2011 Company Confidential Page 50 (50)