Analysis of Service Quality in 3G Mobile Networks

Heidi Lagerstrm
Supervisor: Professor Heikki Hmminen Instructors: M.Sc. Sami Vesala & M.Sc. Katja Koivu
Omnitele Ltd. 2005 1 Heidi.lagerstrom@omnitele.fi

Contents
1. Introduction to the study Background, research problem, research methods 2. Quality of Service (QoS) in UMTS Networks

3. Measuring service quality Defining Key Performance Indicators (KPI)

4. Case study

Omnitele Ltd. 2005

Heidi.lagerstrom@omnitele.fi

Background
UMTS introduces new real time services to mobile networks, such as video telephony. These real time services require QoS guarantees to function properly. For operators to maintain satisfactory service quality constant network monitoring is needed. Network measurements are based on correctly defined KPIs for each service. Operators possibilities to utilise QoS in practice have not been widely researched. Key Performance Indicators have not been defined for the new services from the end-user perspective.
Omnitele Ltd. 2005 3 Heidi.lagerstrom@omnitele.fi

Research problem
How should service quality be measured in 3G networks and how the QoS mechanisms can be used to affect the service quality perceived by subscribers.
Objectives: 1. What are the KPIs that measure service quality, from end user perspective, in 3G networks for the key services (AMR voice, video telephony, video streaming, web browsing and e-mail)? 2. What are the QoS mechanisms in Release 99 and how can they be used to improve service quality?

Omnitele Ltd. 2005

Heidi.lagerstrom@omnitele.fi

Research methods
Literature study 3GPP, ETSI, ITU specifications Several books and publications Interviews Network equipment vendors: Ericsson, Nokia Operators: Elisa Several other radio network experts Case study Field measurements for two operators in live networks

Omnitele Ltd. 2005

Heidi.lagerstrom@omnitele.fi

Contents
1. Introduction to the study Background, research problem, research methods 2. Quality of Service (QoS) in UMTS Networks

3. Measuring service quality Defining Key Performance Indicators (KPI)

4. Case study

Omnitele Ltd. 2005

Heidi.lagerstrom@omnitele.fi

Why do we need QoS?

UMTS networks support services with very different performance requirements Real-time services require performance guarantees Customer acceptance closely tied to service quality Optimal usage of network resources Radio resources scarce Cost-effectiveness Return of investment Service and user differentiation Meet different needs of customers (e.g. business vs. consumer) Performance Requirements Support different services Sensitivity Application Bandwidth (real-time vs. best effort) Delay Jitter Video call High High High Competitive advantage!
Streaming High Med Med Low Med Low Low Web browsing E-mail Med Low

Loss Med Med High High

Omnitele Ltd. 2005

Heidi.lagerstrom@omnitele.fi

QoS Traffic Classes

Traffic class Conversational Characteristics Preserve time relation between information entities of the stream. Conversational pattern (stringent and low delay) Preserve time relation between information entities of the stream. Request-response pattern. Preserve payload content. Destination is not expecting the data within a certain time. Preserve payload content. Example application Speech Video calls Demanding Delay Jitter Demanding Bit rate Jitter Tolerant Delay and bit rate can vary Integrity Easiest Delay and bit rate can vary Integrity

Streaming

Real-time streaming video Web browsing E-mail File downloading

Interactive

Background

Omnitele Ltd. 2005

Heidi.lagerstrom@omnitele.fi

QoS Profile Attributes

R99 QoS attribute
Residual BER SDU error ratio

Example value
10 5 10 4

Delivery of erraneous SDUs

Maximum SDU size (octets)

No
1500

Delivery order
Transfer delay ARP Traffic Class

No
100 ms (conversational) 280 ms (streaming) 1, 2 or 3 Conversational, streaming, interactive, background 1, 2 or 3 (same as ARP) e.g. 64, 128 or 384 kbps e.g. 64, 128 or 384 kbps
Heidi.lagerstrom@omnitele.fi

Depends on operators QoS strategy Depends on the QoS strategy and UE/RNC capabilities

THP Maximum allowed bit rate Maximum guaranteed bit rate

Omnitele Ltd. 2005 9

QoS Differentiation
Video telephony

Streaming

Conversational RAB Streaming RAB

Push-to-talk

Interactive RAB, THP/ARP = 1 Interactive RAB, THP/ARP = 3

Web browsing MMS

Background RAB

Each service gets the treatment it requires according to the QoS profile Network resources are shared according to the service needs Network resources can be used more efficiently
Omnitele Ltd. 2005 10 Heidi.lagerstrom@omnitele.fi

QoS Mechanisms
PDP context with the requested QoS capabilities RRM ATM QoS

UE Different channel types

Node B
UTRAN

RNC

Iu

Gn

3G-SGSN

Gn
Inter-PLMN Backbone NT

3G-GGSN

PS Domain

DiffServ on transport level IP (ATM QoS for CS)

Firewall

Diffserv on transport level IP

External IP (Internet)

Different QoS techniques in different parts of the network Appropriate QoS must be provided in every network so that the user can experience good service quality
Omnitele Ltd. 2005 11 Heidi.lagerstrom@omnitele.fi

Diffserv in Gi
TE

IP

Operators QoS Strategy

Node B Application server 3G GGSN

RNC

3G SGSN

Conversational RAB Streaming RAB Interactive RAB, THP/ARP = 1 Interactive RAB, THP/ARP = 3 Background RAB

HLR
User profiles are stored in HLR. Each user can have several user profiles, which correspond to different services and are mapped to different bearers according to the operators strategy.

Operators can practise user differentiation by

giving each user set of QoS profiles, which he/she is entitled to use Operators can practise service differentiation by mapping each service to the bearer that meets its requirements
Omnitele Ltd. 2005 12 Heidi.lagerstrom@omnitele.fi

Meet the needs of different customers Offer each service the quality it requires Optimise network resource usage

Contents
1. Introduction to the study Background, research problem, research methods 2. Quality of Service (QoS) in UMTS Networks

3. Measuring service quality Defining Key Performance Indicators (KPI)

4. Case study

Omnitele Ltd. 2005

13

Heidi.lagerstrom@omnitele.fi

Measuring network performance

Customer feedback

Network Performance Monitoring Optimisation

Network statistics from different counters and interfaces

Performance statistics from application server

E2E service quality, QoE

Node B RNC 3G SGSN 3G GGSN

Application server

UTRAN
Omnitele Ltd. 2005 14

Core nw
Heidi.lagerstrom@omnitele.fi

External nw

Defining the right KPIs

Different services have different quality requirements KPIs must be defined separately for each of the key services
KPI categories Service Accessibility Service Integrity Service Retainability With inadequate performance indicators and monitoring Hidden problems in network performance and user perceived quality of service Poorly defined indicators may show better quality than in the reality Incorrect formulas and counters Unreasonable measurement periods (too much averaging etc.)

Omnitele Ltd. 2005

15

Heidi.lagerstrom@omnitele.fi

Example: Voice Services - CS

Customer demand Service accessability

Indicator Availability & Coverage Call setup success rate Call setup delay

Measure Ec/No, RSCP Admission control RAB assignment Noisy frames (FER), MOS Handover failure No coverage Interference

Service integrity Voice quality Service retainability Call drop rate

Omnitele Ltd. 2005

16

Heidi.lagerstrom@omnitele.fi

Example: Data Services - PS

Customer demand Service accessability Indicators Availability & Coverage Access success rate Service access delay Video quality Audio quality Web page download time E-mail sending time, etc. Dropped data connection Connection timeouts Measures Ec/No, RSCP Admission control Attach, PDP context activation, IP service setup BLER, FER, throughput, delay, jitter

Service integrity

Service retainability

Dropped PDP context/attach No coverage etc. Handover failure

Omnitele Ltd. 2005

17

Heidi.lagerstrom@omnitele.fi

Contents
1. Introduction to the study Background, research problem, research methods 2. Quality of Service (QoS) in UMTS Networks

3. Measuring service quality Defining Key Performance Indicators (KPI)

4. Case study

Omnitele Ltd. 2005

18

Heidi.lagerstrom@omnitele.fi

Measurement plan
Operator 1 Drive test: AMR speech Drive test: FTP download Video telephony Streaming Web page download X X X X X Operator 2 X X N/A X X

E-mail
Data connection: attach, PDP context activation, RTT, FTP DL & UL

X
X

X
X

Tools: Nemo Outdoor, Optimi x-AppMonitor, Ethereal

Omnitele Ltd. 2005 19 Heidi.lagerstrom@omnitele.fi

AMR voice Drive test statistics

Call setup time (s) Call setup success rate (%) Call completion rate (%) Soft handovers per call Soft handover interval (s) ave. Soft handover success rate (%) Best active Ec/N0 (dB) ave. Best active RSCP (dBm) ave. Tx Power (dBm) ave. BLER Pilot BER Operator 1 4.649 70 100 10.33 7.901 100 -4.03 -79.3 -15.5 0.254 1.917 Operator 2 2.494 100 100 10.38 7.953 100 -3.97 -66.9 -29.5 0.221 2.363

Shoud be ~100%

Good > -10dB Good > -92dBm Good < 21dBm

Reasons for call failure: Ec/N0 was not at adequate level Call setup was unsuccessful (unsuccessful RACH procedure) Look at L3 signalling
Omnitele Ltd. 2005 20 Heidi.lagerstrom@omnitele.fi

CPICH coverage Ec/N0

Operator 1

According to Ec/N0 values both operators have good coverage. Couple of RED areas, which need to be further investigated!
Operator 2

If large interference areas are generated, the problem could be minimised later by adjusting the antenna direction or height, or by down tilting the antenna or by slightly tuning the pilot power levels.
Omnitele Ltd. 2005 21 Heidi.lagerstrom@omnitele.fi

Data Connection

Typical RTT in UMTS network is ~200ms, which enables good quality conversational PS services, such as VoIP.

Omnitele Ltd. 2005

22

Heidi.lagerstrom@omnitele.fi

Video streaming

In mobile phone display ~60 kbps streaming bit rate produces good video quality.
Omnitele Ltd. 2005 23 Heidi.lagerstrom@omnitele.fi

Web browsing
Service access time
0.26 0.26

Web page download time

22.79 25 20 16.59

Time (s)

0.255 0.25 0.25 0.245

Time (s)

15 10 5

Operator 1
Throughput
200

Operator 2

Operator 1

Operator 2

Throughput (kbps)

150 100 50 0 1 2 3 4 5 6 7 8 9 10 11 12 Time Instantaneous Average

Operator 1 Operator 2 Service accessibility (%) 100 % 100 % Service access time (s) 0.25 0.26 Web page download time (s) 22.79 16.59 Service retainability (%) 100 % 100 %

Sample web page 319 kB

Heidi.lagerstrom@omnitele.fi

Omnitele Ltd. 2005

24

Conclusions
In 3G networks QoS management is required
Real-time services require QoS guarantees Need to support different kinds of services With QoS mechanisms operators can use their network resources more efficiently and gain competitive advantage

To maintain and improve the network performance and user experienced service quality constant monitoring and performance follow-up is needed
Successful network measurements are based on correct KPI definitions A combination of end-to-end field measurements, interface probes, network element counter statistics and customer feedback is required

The measurement results show that there are big differences in the performance of operators UMTS networks
Currently UMTS networks are not fully optimised there is a clear need for optimisation! Majority of 3G measuring equipment and terminals are still quite immature

Omnitele Ltd. 2005

25

Heidi.lagerstrom@omnitele.fi

For more information about Omnitele, please visit our web site www.omnitele.fi

Omnitele Ltd. 2004 2005

26

KPI Definitions

Omnitele Ltd. 2004 2005

27

AMR Speech KPIs

Service coverage Parameters Service accessibility Service access time Trigger points
T0

Speech quality Codec usage Service retainability Alerting message

T1 ALERTING

Place a call

Speech interchange
T2
Start of audio stream

Intentional termination of session

T3
RELEASE

Channel request

Omnitele Ltd. 2005

28

Heidi.lagerstrom@omnitele.fi

Video Telephony KPIs

Speech quality Service coverage Parameters Service accessibility Service access time Trigger points
T0 Channel request

Video quality Video call setup success ratio Video call setup time

Audio/video synchronisation
Service retainability Intentional termination of session
T3 Audio/video output ends RELEASE

Video call Request

T1

Alerting message

Audio/video output
T2 Audio/video output starts

ALERTING / Call accepted

Omnitele Ltd. 2005

29

Heidi.lagerstrom@omnitele.fi

Video Streaming KPIs

Video quality Service coverage Parameters Streaming reproduction start failure Streaming reproduction start delay Buffering message appears on player
T1 RTP: payload 1st data packet BUFFERING

Audio quality Audio/video synchronisation Streaming reproduction cut-off ratio Stream reproduction
T2 Streaming reproduction starts picture appears PLAY

Service accessibility
Service access time Trigger points
T0
RTSP: SETUP

Stream Request

Intentional termination of session

T3

Video/audio stream ends

RTSP TEARDOWN

Omnitele Ltd. 2005

30

Heidi.lagerstrom@omnitele.fi

Web Browsing KPIs

Service coverage Parameters Service accessibility Service access time Trigger points
T0 1st TCP [SYN]

Web page download time Service retainability Data transfer

T1 1st HTTP: GET

Service access

Intentional termination of session

T2 HTTP: FIN/ACK Reception of last data packet Display data T3

Omnitele Ltd. 2005

31

Heidi.lagerstrom@omnitele.fi

E-mail KPIs

Service coverage Parameters Service accessibility Service access time Trigger points
T0 1st TCP [SYN]

Sending time

Receiving time

Service retainability E-mail sending

T1 SMTP: 250 ACK (HELO) T2 T3

Service access

E-mail download
T4 Last data packet received TCP [FIN/ACK]

Last data IMAP: FETCH Body packet send TCP [FIN/ACK]

Omnitele Ltd. 2005

32

Heidi.lagerstrom@omnitele.fi