You are on page 1of 8

Open-Source Based Prototype for Quality of Service (QoS) Monitoring and Quality of Experience (QoE) Estimation in Telecommunication Environments

Bastian Huntgeburth, Michael Maruschke

Hochschule fr Telekommunikation Leipzig (HfTL) Deutsche Telekom AG Leipzig, Germany e-mail:

Sebastian Schumann
Department of Telecommunications Slovak University of Technology (STU) Bratislava, Slovakia e-mail: commercial solution named NetIQ Vivinet Assessor (see [5]) works similarly to the authors solution with an additional underlying call generator. As none of these products are free, the authors analyzed the underlying mathematical model (see section II.) and built a useable open solution. Table 1 briefly compares the mentioned vendor solutions and the solution presented within this paper. In addition, some advantages of the authors solution were pointed out. II. THEORY The referred model to determine the conversation quality (QoE) perceived by the user is also known as the E-Model [1]. The model uses several network parameters to calculate the R-Factor, which can be mapped on a subjective evaluation of voice quality represented by the five point scale of Mean Opinion Score (MOS) [6], [7]. The default values for the parameters are written in the standard and represent a clean Integrated Services Digital Network (ISDN) connection with an R-Factor of 93.2. Every additional degradation reduces this value. Figure 1 shows the resulting quality correlating MOS value and R-Factor.

Abstract This paper describes an implementation for monitoring the QoS and expecting the QoE of a voice communication in a Real-time Transport Protocol (RTP) based telecommunication environment. The resulting QoS parameters are evaluated; the QoE is determined with the EModel [1] and processed for graphical presentation. With the use of some open-source programming libraries, the presented prototype can be a helpful alternative for expensive measurement devices and is ready to be deployed in a widespread telecom environment at low cost. (Abstract) Keywords open-source;prototype;QoS;QoE;monitoring;VoIP

I. INTRODUCTION With the recent massive migration from heterogeneous networks to all-IP based infrastructures, operators face new problems. While the circuit-switched alternative is considered stable and consistent in terms of reliability and delivered quality, packet-based voice communication is still regarded unstable and not of constant quality. The lack of an exclusive end-to-end guaranteed communication channel might have a negative impact on the transmitted data. As it is not always possible to enforce QoS for voice calls in unmanaged IP networks, operators need to find a way to monitor the QoS and QoE in their network on certain congestion points. Even if Voice over IP (VoIP) traffic does not share the bandwidth and available resources with other high-demanding services (web traffic, streamed video content or Peer-to-Peer (P2P) traffic), changes in the network configuration might still lead to a loss in quality. This paper describes a distributed setup for measuring the perceived voice call quality on pre-defined congestion points and centrally analyzes and presents the current service quality to an administrative instance. The components have been implemented within a proof of concept. The focal point of this work is monitoring widespread networks on defined points and their impact in the quality of delivered voice communication. For monitoring the quality impact of voice communication processors (e.g., Public Branch Exchange (PBX), announcement services, etc.) as such, the presented measurement setup may be extended by the work the authors presented in [2]. Existing commercial voice quality measurement solutions offer mainly multiple analyzing functions for several purposes and are not focused solely on VoIP QoS/QoE monitoring (e.g., [3]). Other solutions like the Smart RTP Monitoring Probe developed by VOIPFUTURE (see [4]) analyze the loss probability and delay variation of an RTP stream without conversational background, but at a high rate of current calls. Another

Figure 1. Correlation between MOS value and R-Factor

With the rise of VoIP, the focus of some reference guides and recommendations was extended with the QoE assessment (see also Table 9-1 in [8] and the recommendation [9]). The most important well-known impacts on VoIP speech quality are: One-way delay Packet-loss probability Packet-loss distribution

Used speech codec and their packet-loss robustness-factor With a regular measurement monitoring of all these factors, the current QoS and QoE can be determined and any loss would be visible instantly. A further description of the E-Model algorithm is not part of this paper. The authors describe instead how to measure the mentioned impairment factor with the help of opensource programming libraries and other cost-free tools in a cost-efficient, flexible and adaptable way. The main goals of the setup are: Easy but flexible measurement design A non-intrusive online monitoring Informative results Ability to determine the geographical and technical source of degradations III. PRACTICAL MEASUREMENT This section discusses the integral components and the basic architecture of the measurement environment. Furthermore, a possible graphical evaluation of the results is considered. The System Under Test (SUT) has to be a working VoIP environment using the Session Initiation Protocol (SIP) as the signaling protocol. A. Architecture The main task of the architecture is to allow a localization of the point or the leg with a faulty degradation in speech quality. To provide this functionality, quality measurement has to be done at several points on the communication path. The measurement process must not have any influence on the speech signal or its delivery. To get a contemporary, comparable quality result, the measured parameters are sent to a centralized evaluator, which computes the quality value with the help of the E-Model. Figure 2 shows the flow of the Real-Time Transport Protocol (RTP) packets that contain the voice signal (red arrows). Copies of these packets are received by the
Table 1.

Measurement Entities (ME). In this case, the copies are made at the customer routers and are sent out at their monitoring ports. The ME extracts the input parameters for the quality value computation and sends them to the Evaluation Entity (EE) (blue arrows). These components can be hosted on the Telecommunication carrier (Telco) side. The extraction of the input parameters is described in the following paragraphs. B. Analyzing the Packets An instant capturing of the RTP packet is very important for a reliable measurement. The Packet Capture (PCAP) library is used for this purpose. It is commonly used in other well known network-sniffing tools as well, e.g., tcpdump [10]. The PCAP library provides access to the raw data of a packet and creates a time-stamp at receiving time. Based on this library, a short script analyzes every received packet and extracts the needed parameters. It depends on the performance of the programming language how many connections can be measured at the same time. To provide the portability on other operating systems, independent scripting languages like Perl should be used. As the results are required nearly in real-time, the performance of the programming language is important as well. C. Measured Impairments The first speech QoS parameter one-way delay is determined by halving the Round-Trip-Time (RTT) value of the voice packets. With the help of the time-stamp that has been produced by the PCAP library and the time-stamp included in the RTP Control Protocol (RTCP) packet, the RTT can be determined. The discussed calculation in [11] was slightly modified by the authors to get reliable results in an unsynchronized environment. The RTT is calculated acc. [11] based on the time-stamp of the Last Sender Report (LSR), which is delivered by the resulting RTCP Sender Report (SR). This time-stamp was replaced by the time-stamp, which was created by the ME when the first SR passed by.

Comparison QoS/QoE monitoring solutions

Authors solution Conversational quality based on network performance Non-intrusive Standard: ITU-T G.107 Transparent implementation -

Smart RTP Probe (VOIPFUTURE) Measurement Listening quality based on network performance Non-intrusive Standard: ITU-T P.564 Implementation not visible Costs Costs per probe, special hardware needed Reporting Real-time reporting of quality results Fixed time granularity of 5s

Vivinet Assessor NetIQ Conversational quality based on network performance Non-intrusive measurement of self-generated VoIP traffic Standard: ITU-T G.107 Implementation not visible Costs per measurement system, no special hardware needed

No cost, no special hardware needed

Real-time reporting of quality results, flexible time granularity Integration in Telcos own reporting platform possible

Rough overview after a call finished, detailed view after whole schedule finished

Adoptable and extensible to the Telcos needs, since the source code is open

Adaptability No adoption possible

No adoption possible

Figure 3 shows the RTT computation of a call between two User Agents (UAs) with a ME in between.

Figure 3. Sequence diagram to determine packet delay through RTT

A1 represents the time when the first SR1 passes the ME and A2 is the time-stamp of the detection of the following SR2. D2 represents the delay between the reception of SR1 and the transmission of SR2 at UA1, which was determined by UA1 and delivered in SR2. RTT1=A2-A1-D2 RTT2=A3-A2-D3 Half of the value of the RTT is a good estimation for the one-way delay. As a feature of this method, a RTT measurement in both directions is possible. As a second important quality parameter, the packet loss probability is determined by recording the sequence number of each RTP packet that passes the ME. In this case, the loss

probability is updated after every 100 RTP packets. The time distance of 100 RTP packets is a good balance between the applied load on the ME, the network load, and the actuality of the measurement results on the EE. The refresh period can be calculated depending on the size of the packet payload. For instance, a payload size of 20ms results in a refresh period of two seconds. Figure 4 shows a part of the analyzing script in form of a program flowchart. In figure 4, the used variables correspond to the packet loss and are explained in the following: $pls := length of a packet loss sequence $first_seq := sequence number at the beginning $last_seq := sequence number of the last received packet $hpl := overall count of lost packets $AVpls{} := hash, which counts every $pls $all_packets := sum of received and lost packets $Ppl := over all packet loss probability $sum_of_pls := sum of any $pls $count_of_pls := number of $pls $avg_length_of_loss := average length of all loss sequences The values $Ppl and $avg_length_of_loss are send to the EE. This is done for each RTP stream of the communication session, identified by the Synchronization Source Identifier (SSRC).

Figure 2. RTP packet flow in sample test bed

D. Evaluation All four parameters one-way delay, packet loss probability, packet loss distribution, and the used speech codec including their robustness factor are used to be processed according to the E-Model. Therefore they were sent to the centralized EE. This process is triggered every 100 RTP packets. To avoid communication problems due to firewalls, the values were sent over a Hyper Text Transfer Protocol (HTTP) connection. The arrival of each HTTP packet from a ME starts the computation of the current quality of a monitored call. While the E-Model algorithm is processed, the default values that represent a clean ISDN connection are replaced by the measured ones. E. Implementation To provide portability to other operating systems and an extended graphical presentation of the results, Java was used to implement the EE. The informational view of the measurements was divided into three detail levels. The first one provides an overview of the connected MEs. The second level gives a comprehensive overview of the main properties of a call, like the calling and the called party. It also displays a function of the resulting R-Factor values. The function is adopted with every new calculated value and depicted in a relation to the time. The third level offers details of all measured values, which were used for the quality estimation. The measured values on the detail level three can be interpreted as the current QoS and the function on detail level two as the actual QoE at this point of the communication path. The measured values and the painted function are colored in relation to the voice quality categories in Figure 5. The thresholds for the coloring can be extracted out of the algorithm of the E-Model. Figure 5 shows the thresholds for the one-way delay. With a rise above 150ms, the delay has a significant influence on the quality, especially in conversational situations. The thresholds for coloring the packet loss values depend additionally on the packet loss distribution BurstR [9] and the packet loss robustness factor (BPL) of the used codec. Figure 6 shows these thresholds when the used codec is G.711, which has a relatively low BPL as a function of the measured Ppl and BurstR. A dependent packet loss has a greater influence on the perceived quality. The coloring of the QoS and QoE results provides a fast and informal overview. An adjustment of the thresholds for individual needs is also very easy.

Figure 4. Flowchart of quality measurement script

The average length of loss sequences is calculated to determine the packet loss distribution with the help of the patent of McGowan [12]. Using the variables overall packet loss probability and average length of all loss sequences (see also description of figure 4) the final calculation of the packet loss distribution is performed in the EE. As a fourth quality domain of a packet based voice communication, the used speech codec is determined by parsing the Session Description Protocol (SDP) during the session establishment procedure. The knowledge of the used codec is important in relation to the used compression method and its robustness against packet loss (packet loss robustness factor). These values are analyzed and recommended as in [13].

Figure 5. Thresholds for one-way delay

Figure 6. Thresholds for G.711 call as a function of the Ppl and BurstR

IV. SAMPLE MEASUREMENT The following sample measurements were made in a production environment of a German Telco. The voice quality of a national communication and collaboration product has been reviewed. Two branch offices were connected to a hosted VoIP communication infrastructure (see III.A). An IP PBX acts as a back-to-back user agent to solve Network Address Translation (NAT) issues and to provide topology hiding. Branch office A has a plain IP access to the Telcos backbone with a symmetrical bandwidth of 2 Mbit/s. The branch office B was connected via an asymmetrical Digital Subscriber Line (DSL) connection with a bandwidth of 1.5 Mbit/s downstream and 128kBit/s upstream. There was no prioritization of real-time communication implemented. The SUT consists of the complete communication path, as the MEs were placed at the two UAs. The EE was reachable with a public IP-address. The UAs are able to call each other. In the measurements, a call with G.711 as used codec was established. In the first case, there was no other IP-

traffic on the access links. In contrast, in the second measurement the access link of branch office B was loaded with some IP-traffic. Figure 7 shows the communication paths, which were taken into account by each ME (numbered blue arrows). For the first measurement, a very good QoS and QoE were expected. The second measurement should depict a degradation of the QoS and QoE, as one of the access links was under load. Table 2 shows some detailed results of the first measurement scenario. The QoE values are very constant and nearly at their theoretical maximum of R-Factor equal 93.2. The quality of the communication path is depicted in Figures 8-11. In the first and third part of the communication path is nearly no degradation, as this is a local area network (LAN) connection. The second and fourth parts, which mainly consist of wide area network connections, show however a normal transmission delay. The packet loss probability is also zero percent.

Figure 7. Communication paths for measurement Table 2. First measurement results

Part of com. path 1

R-Factor 93.16 93.14

One-way delay [ms] 0.96 2.94 25.91 24.43 0.63 0.47 25.61 19.80

Packet loss probability [%]


Part 2 of communication path

1 0,9 0,8 Packet Loos [%] 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 t [s] R-Factor Delay [ms] Packet Loss

Delay [ms] & R-Factor

90 80 70 60 50 40 30 20 10

0.00 0.00 0.00 0.00 0.00 0.00

92.36 92.41

93.17 93.18

92.37 92.54

Figure 9: Part 2 of the communication path


Part 1 of communication path

100 90 Delay [ms] & R-Factor 80 70 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 t [s] R-Factor Delay [ms] Packet Loss 1 0,9
Delay [ms] & R-Factor 100 90 80 70 60 50 40 30 20 10 0

Part 3 of communication path

1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 t [s] R-Factor Delay [ms] Packet Loss Packet Loos [%]

0,8 Packet Loos [%] 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0

Figure 8: Part 1 of the communication path

Figure 10: Part 3 of the communication path

Part 4 of communication path

100 90 80 Delay [ms] & R-Factor 70 60 50 40 30 20 10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 t [s] R-Factor Delay [ms] Packet Loss 1 0,9

Part 1 of communication path

100 90 Delay [ms] & R-Factor 80 70 60 50 40 30 20 10 0 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 t [s] R-Factor Delay [ms] Packet Loss 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 Packet Loos [%]
Packet Loos [%]

0,8 Packet Loos [%] 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0

Figure 11: Part 4 of the communication path

Figure 12: Part 1 of the com. Path, second measurement

Delay [ms] & R-Factor

Table 3 shows some detailed results of the second measurement. It is visible that in communication path 2 and 4 the quality values are degraded. The quality of the degraded communication path is depicted in Figures 12-15.
Table 3. Second measurement results

Part 2 of communication path

700 600 500 400 300 200 100 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 0,45 0,4 Packet Loos [%] 0,35 0,3 0,25 0,2 0,15 0,1 0,05 0 t [s] R-Factor Delay [ms] Packet Loss

Part of com. path 1

R-Factor 93.16 92.53

One-way delay [ms] 0.79 2.22 428.04 630.87 0.38 0.62 631.38 528.93

Packet loss probability [%] 0.00 0.00 0.00 0.40 0.00

59.90 40.08

Figure 13: Part 2 of the com. Path, second measurement

93.19 93.17

Part 3 of communication path

100 1 0,9 0,8 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 t [s] R-Factor Delay [ms] Packet Loss

Delay [ms] & R-Factor

90 80 70 60 50 40 30 20 10 0

48.14 53.08

0.00 0.00

As the RTT is the base for the estimation of the one-way delay, the source for the faulty connection with bad QoS/QoE can only be located on one of the paths two or four. If the packet loss is also taken into account, the source of degradation is most likely the low upstream rate (0.128 Mbit/s) at the WAN link of branch office B. The traffic on the WAN link was generated at around t=6s. At this point, the delay rose and a significant degradation of the QoE/QoS occurred. To avoid the bad QoS results, prioritization mechanisms for the voice packets could be s useful solution.

Figure 14: Part 3 of the com. Path, second measurement

Part 4 of communication path

700 600 Delay [ms] & R-Factor 500 400 300 200 100 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 t [s] R-Factor Delay [ms] Packet Loss 1 0,9 0,8 Packet Loos [%] 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0

ACKNOWLEDGMENT The third author is affiliated member of the Institut fr Telekommunikationsinformatik at the Hochschule fr Telekommunikation in Leipzig, Germany. STU and HfTL are working together on common topics in the area of Next Generation Networks (NGN) since 2006. This paper also presents some of the results and acquired experience from various research projects such as the project [14], European Celtic-EURECA project Netlab [15], and the Slovak National basic research project VEGA No. 1/0720/09. REFERENCES
[1] [2] ITU-T G.107, The E-model: a computational model for use in transmission planning, 04/2009 B. Huntgeburth, S. Schumann, and J. Londk: VoIP Speech Quality Measurement with Open-Source Software Components, Proc. 52nd International Symposium ELMAR 2010 in Zadar, Croatia, September 2010, pp. 215-218, Print ISBN: 978-1-4244-6371-8 Sage instruments: VoIP Quality test solutions; VOIPFUTURE GmbH, Hamburg: Smart RTP Monitoring Probe; NetIQ Corp.: NetIQ Vivinet Assessor; ITU-T P.800, Methods for subjective determination of transmission quality, 08/96 ITU-T P.805, Subjective evaluation of conversational quality 04/02 ITU-T G.1011, Reference guide to a Quality of Experience assessment methodologies, 06/10 ITU-T P.651, In-service non-intrusive measurement device- voice service measurement, 07/02 Schulzrinne H., Casner S., Frederick R., Jacobson V.: RFC 3550, RTP: A Transport Protocol for Real-Time Applications, 07/2003 McGowan, J.W., Burst Ratio (BurstR) US Patent 6.931.017, 082005 ITU-T G.113, Transmission impairments dueto speech processing, 05/2002 - International initiative devoted to promote NGN technology and services it provides, Netlab - Use Cases for Interconnected Testbeds and Living Labs,

Figure 15: Part 4 of the com. Path, second measurement

[3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]

V. CONCLUSION The presented prototype is capable to measure the QoS and QoE parameters R-factor, one-way delay, and packet loss probability. The authors presented an implementation that can compete with professional equipment to a certain extent, but is completely based on open-source libraries and own extensions. The prototype is ideal for cost-effective quality measurement in distributed VoIP networks. Widespread Telco networks that can consist of managed and unmanaged parts can be measured with multiple MEs. The distribution of the MEs and the geographical granularity of the measurement can be chosen depending on the network. While the proof of concept implementation used Microsoft Windows, the components can be easily ported to other operating systems, e.g., Linux. In the future, the system can be extended in various ways to be able to quicker react on alarms, or follow-up calls whose QoS/QoE was affected negatively: Send mail or alarm trap at certain threshold Record call samples on ME