Huawei MBB QoE Assurance

Solution

HUAWEI TECHNOLOGIES CO., LTD.

One real case. How to avoid it happen?

Customer Operator Customer

Bad experience No complaint Don’t know what No action

Can’t tolerate，
happened
n Page download slowly say ”Goodbye“
complain No any change
n Drop frequently Get ,but don’t know
how to treat it?

3 Requirements Solution
1 QoE evaluation Evaluation
How to evaluate whether QoE has problem?

2 QoE problem is known ,How to locate it? Fast troubleshooting Fault

location
QoE problem is Located, How to solve it?
3 QoE Optimization Optimization

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 2
E2E QoE optimization
Service quality is key of QoE assurance
QoE assurance including evaluation, problem location and optimization should focus on service KQI
QoE is impacted by many factors including market service, service quality, etc. Service quality is key
Service quality can be matched by E2E service KQI (Key Quality Indication).
Example: Browsing service QoE
Service
Service KQI Mapping Network KPI
Quality
QoE Accessibility
Page download setup SR Attach SR
(Quality of … PDP Active SR
Experience) Page download time Attach time
Page refresh download time PDP active time
Integrity/ Page download peak rate throughput
DNS analysis
Market Application Service Quality Page download error
time
service content quality
…. Ps connect SR
(34%) (4%) (62%)
Ps setup SR
uPrice uAccessibility Interrupt time HO SR
uUsability
uDevice Retainablity Interrupt frequency Drop rate
uConvenient uIntegrity
uBrand uEntertainment
Drop rate
uCustomer care /Quality
uService after sale uSecurity uRetainablity

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 3
E2E QoE optimization
QoE Manager Solution-QoE evaluation

n By installing agent software on terminal , terminal can collect and report

subscriber QoE related info to WEB server in network.
Subscriber ü Improving customer satisfaction and loyalty by customer care before complaint
ü Saving OPEX greatly by reducing dedicate dial and drive testing task by 60% due to
subscriber with agent is one full time testing volunteer.
Page download Http Browsing
WEB Server time average throughput
Agent .

collect Evaluate

Bad experience:
Page download slowly， service KQI Radio KPI VIP Coverage
Drop frequently

Try again and again ,

No any change
Sending care message to customer before he complains

10:00 pm Abnormal KQI find 10:15 10:20 pm

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 4
QoE Manager Solution-Mobile Manager (M2) Overview

Agent

Terminal AN AMS DB SPAS MT

n System Components: Main functionalities：

ü Terminal: Datacard, commercial mobile phone
Category Sub-Cat. Function description Type
ü Agent: Data collection and upload
Service quality Voice Call setup time, MOS, Drop etc basic
ü AMS: Agent distribution and update and data
Data PDP Act. Delay, Jitter/Throughput/PLR. basic
pre-process.
Message SMS/MMS delivery Suc. Rate, Text Size, basic
ü DB: Data storage and post-process
Streaming Setup delay, Jitter, Video/Audio Syn, MOS etc basic
ü SPAS: Network and service quality evaluation
Network KPI Radio quality Support 800 measurement items basic
report
Experiences QoE Evaluation Service KQI evaluation basic
n Network: GPRS, EDGE, UMTS, HSPA
n Mutli-Operator Benchmark User Behavior Statistics Switch on/off time, functionality usage etc optional
API Service Customize interface for specific application optional
Statistics

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 5
QoE Evaluation M2 Solution-System Architecture

Problem KPI anomalies & QoE / QoS alarms Service/wireless quality

localization monitoring monitoring
Analysis and
report
25.00%

20.00%

Call DropRate
15.00%

10.00%

5.00%

Relevant KPI
0.00%
Cell 1 Cell 2 Cell 3 Cell 4 Cell 5

Relating trouble T o tal Dr op Rate HO Dr op Rate

tickets Reports
Relating alarms

Data
Classification DB SPAS
AMS
(Service Performance &Quality
and storage (Agent Management System)
Analysis System)

Passive measurement in RT (KPIs / log file / …), Active probing measurement in RT or periodical (automated) or
automated data upload manual data uploading

Data collect TA TA
Data card Mobile phone
layer
Commercial terminals GENEX TEST
(PHU/Probe/special test terminal)

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 6
QoE Evaluation M2 Solution- Agent Distribution

Terminal AMS

Datacard
ØFor new Datacard, Agent kit cooud be integrated with installation disk for customer installation.
Application Scenarios
ØFor Datacard in subscriber, AMS push agent installation guide via SMS/MMS.
ØInstallation is default enabled but shall be explicitly reminding for customer’s choice. • Internal user: voluntary to install
ØAgent will contact AMS server during idle period after dial-up. agent to QoE monitoring。
ØAgent automatically record and analysis the radio quality service quality without disturb subscriber. • Group user: Express service
ØAgent could be uninstalled by user-self. company, Taxi company, etc.
• Unlimited volume user, default
agreement with user
Commercial Mobile phone • VIP user, explicitly agreement with
ØSystem inform subscriber via SMS/MMS and push the installation directory to subscriber. user.
ØSubscriber could logon network to install agent by user-self • Tariff promotion to get volunteer
ØAgent will contact AMS server during idle period after dial-up. • …
ØAgent automatically record and analysis the radio quality service quality without disturb subscriber.
ØAgent could be uninstalled by user-self.

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 7
M2 Solution- Examples of radio quality parameters

Radio KP 覆盖 差 的区
Bad Coverage
域
CPICH RSCP
CPICH Ec/N0
Rx Power Downlink
Tx Power Interference
Radio KPI
BLER
Random Access Success Ratio /Delay
HSUPA RLC Throughput
RRC Setup Success Ratio /Delay
HSDPA RLC Throughput Radio KPI
HSUPA RAB Setup Success Ratio
HSDPA RAB Setup Success Ratio Bearer Service Drop Ratio
Abnormal HO
PDP Context Activation Success Ratio HSUPA Soft HO Success Ratio /Delay

Hard HO Success Ratio /Delay

HSDPA Inter-RAT HO Success Ratio

/Delay
HSUPA Inter-RAT HO Success Ratio
/Delay

TA

TA
AMS DB SPAS

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 8
M2 Solution- Examples of mobile data service KPIs

Listening quality MOS (MOS-LQ)*, call setup success ratio, call setup delay, call success
ratio, one way delay, packet loss ratio, jitter, etc…
VoIP & *)Perceptual Evaluation of Speech Quality PESQ (ITU-T P.862.x) for active measurements and E-Model (ITU-T G.107) for passive measurements
Video Call

Browsing completed ratio, page download delay, active session throughput, FTP
download/upload delay, IP QoS (throughput, delay, jitter, packet loss)
Mobile Data
Internet

E-mail session setup success ratio, e-mail send/receive completed ratio, send/receive
delay, etc…
E-Commerce
E-Mail

Video MOS**, streaming setup success ratio, first buffering delay, streaming
audio/video non-synchronization, RTP over IP QoS
Game/TV/ **)PEVQ (ITU-T Rec. J.247)
Content

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 9
M2 Solution- Service triggered questionnaires
Customer
Pop-up Experience
1：E2E Service
quality What about the page loading
measurement delay?
5 very quick 2：Questionnaires
4 quick
3 normal
2 slow
Loading completed 1 very slow
Close the explorer
Passive measurements
Loading delay
Throughput KPI Mapping
RF signal … Data analysis 3：Correlation

Start loading page KPI KQI

1）When the service terminates, a questionnaire is sent to the customer to capture his/her
immediate experience
Input a web URL
2）Compare the subjective and objective measurement results, improve service quality KPI
models
3）Correlate the E2E service quality, radio quality and terminal performance, help
troubleshooting

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 10
M2 Solution- Real time QoE monitoring for VIP user

Ensuring a good end user experience is a

cornerstone of SLA contracts with corporate
customers

M2 is able to constantly monitor the received

service level from end-user perspective
› Network coverage
› Data connection reliability and speed
› Service quality for key applications like e-mail and
intranet access

Also collects information also outside of

network coverage when abroad

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 11
Total vision of collection and QoE Evaluation of M2 solution

Basis information Service Quality Evaluation

n APN; Service KQI
n WAP/WEB: First page download delay, page refresh delay, conn. Suc.Rate,
n OS and version; Average Throughput..
n Connection type (GPRS only/GPRS Preferred/3G Only/3G n FTP：Average file size, peak and average throughput, Conn. Suc,Rate..
Preferred); n MMS：Send Suc.Rate, file size, delay, drop rate,
n …
n Channel quality index (CQI); Network KPI:
n Service Cell ID and neighbour cell ID Info. n Attach Success rate [%]
n 3G DL UARFCN e Primary Scrambling Code; n Attach Delivery Time [sec]
n PDP Context Success Rate [%]
n 2G BCCH & BSIC;
n PDP Context Activation Time [sec]
n … n …

Radio Performance Evaluation Questionnaires

n RSCP/CI/Tx Poweer/ n Periodic updated question list
n MOC attempt, succ, Rate. n QoE subjective feedback
n Handover statistics n …
n Cell ID
n Statistics based on cell/time etc.
n GIS support
n …

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 12
Service Quality Inspector (SQI) Solution-Fault location

M2000/N2000 RAN Nastar U2520/ Service n Locate fault fast by correlated

i2000 CN Nastar SQM
O&M ,expert DB and
Subscriber
1 Performance information which collected by
Alarm info service trace for
fault analysis probe of each interface.
ü Improving satisfaction and loyalty by
22 Huawei fast troubleshooting ( e.g. 5+ Hrs to
Huawei Case DB GGSN delay
Case DB abnormal
evaluation Data collection ~20 Min)
Info and correlation
1 + 2 +3 ü Reduce operation cost

GGSN Fault
Bad experience ： 3 Events located
Page download slowly Probe Probe Probe Probe Probe
Drop frequently Sending care message again to
End to end Control/User plan information (C-plan/U-plan)
Try again and again, customer after problem solved
Device Radio Backhaul Core IMS
No any change

Fault located solved

10:00 pm 10:15 10:20 pm

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 13
Fault location Solution-Functional Architecture

Fault location Service Quality RT Monitor Data Drilling

Service quality
presentation

Data Server Layer Web Server

Data Correlation
MD-Database

Database Entry Database Entry Database Entry Database Entry

Data pre-process Data pre-process

Data pre- Data pre- Data pre- RT Monitor
process process process

Data adaptor Data adaptation Data adaptation Data adaptation

Data collecting Statistic CHR Config. Statistics CHR Config.

Network Element Terminal RAN CN Service

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 14
Fault location Solution-E2E Deployed on Huawei Equipment

Analysis Presentation
Characters：
u E2E network equipment from Huawei
u Data comes from Embed probe:
Terminal probe/RAN probe/CN
probe/ network probe
u Fault location pinpoint to module
level

Collection and Resolution by

Hardware platform

M2 CHR CHR
BTS IP解析 WAPGW MMS SP
Gn Gi Data
SN
GG
RN

BS

SN
SG
C/

Gb/Iu-PS
C

NodeB
IP IP
collection
Terminal RAN CN Service Plat. CP/SP

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 15
Fault location Solution-E2E Deployed on Multi-vendor

Characters: Analysis Presentation

u Standard interface probes for
flexible distribution
u Support multi-vendor Scenario
u Integrated data resolution and
composition, fast delimitation and
location

M2 Collection and Resolution

by Hardware platform
CHR CHR
Terminal BTS WAPGW MMS SP
Gn Gi Data
SN
GG
RN

BS

SN
SG
C/

Gb/Iu-PS
C

NodeB
IP IP
collection
Terminal RAN CN Service Plat. CP/SP

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 16
Fault location Solution-E2E Delimitation and pinpoint
E2E procedure analysis Service application layer analysis E2E transport layer analysis

UE RNC SGSN GGSN Service

0.5% 0% 0.5% 0%

. .

l Analysis of signaling calling procedure l Analysis of service delay l Analysis of transfer delay, loss, jitter
l Analysis of service calling procedure l Analysis of service code result l Analysis of bandwidth and traffic flow

Wireless Beare Core App

E2E data correlation analysis to fast Fault delimitation and pinpoint, improve O&M Efficiency.

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 17
 QoE Optimization-Browsing service case
n QoE Optimization key requirement :
ü Optimization must focus on user direct aware service quality
ü Unlike network KPI optimization, optimization is E2E and supported by powerful
tools and rich experience Subscriber

Browsing@google.com Browsing@google.com
QoE KQI Value QOE KQI Value
Evaluation Page download setup SR 91% Page download setup SR 98.9%
Info First Page download time 5.9s First Page download time 4.3s
Downlink average throughput 260k Downlink average throughput 309k
Page download final SR 53% Page download final SR 90.2%

üFACH state 4a spec optimization: reduce 700ms Excellent experience :

üDNS protocol optimization: reduce 500ms üPage download very fast
üTCP window optimization :improve throughput 17% üConnect stably
üDup ACK context optimization:400ms

Channel Setup TCP Transmission Rendering Service quality better

attachPDP contextRadio channel DNS query & TCP slow TCP steady state Rendering than before
setup setup TCPsync. start

Several weeks ago Now

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 18
Huawei MBB E2E QoE Assurance Solution-Highlights
QoE
Comprehensive E2E data service QoE Metrics
KQI KQI KQI l Manageable QoE-KQI Metrics
l Measurable KQI-KPI index

KPI
KPI

KPI

KPI

KPI

KPI

KPI

KPI
l Entirely KQI/KPI relationship and mapping model

Closed loop QoE Assurance: Evaluation/Fault Location/Optimization

User l Terminal Based QoE evaluation-M2 Solution
Expe-rience
l E2E data correlation analysis platform for fast fault location
l QoE Based improve and optimization mechanism

Serials of Embed probe and external probe, support multi-vendor

lu-PS Gn Gi Gw MMX l UE probe, RAN probe, CN probe, network Probe and Service platform probe
l Monitor standard interface such as Gb/lu-PS，Gn，Gi，Gw
Probe
Easy to deploy ，and nearly no impact on Network Performance
l Monitor mirroring port，nearly have no impact on network Performance and
Mirroring Port architecture

E2E QoE Optimization Solution and capability

Wireless Beare Core App l Joint optimization for UE, RAN, CN, Application and network
l System platform and capability improve based QoE requirement

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 19
Discussion
1) Which services/applications are you mostly focus on?
2) What are your comments and suggestion for Terminal based QoE evaluation solution? E.g.:
measurement indicators/resolution, report mechanism, operation model etc ?
4) For network O&M, How long will it take to locate problem? How long it will take to solve it?
6) What are your requirement on data collecting by probe?
7) Do you prefer service monitoring for 7*24 E2E or only for busy hour?
8) What are the most complains on the MBB service usage in your network?
9) Which service/application and which experience factor shall be optimized and improved? E.g
Web first page loading time, Web Page Refreshing Success Ratio etc.
Any other requirements and suggestions???

HUAWEI TECHNOLOGIES
HISILICON CO., LTD.
SEMICONDUCTOR Page 20
Thank You
www.huawei.com