IxChariot

Course # 985-0300

Copyright 2009

Introduction to IxChariot

What is IxChariot?
IxChariot is a software-based network performance testing and
measurement tool
Tests data, video, voice, broadband and wireless transport networks
Emulates application traffic and measures end-to-end performance and
quality

Used to
Emulate applications to ensure the integrity, performance and reliability of a
network before application deployment
Benchmark network performance and troubleshoot connectivity issues
Test functionality of network equipment under varying traffic profiles

Endpoint 1

Console

2007 Ixia. All rights reserved.

Endpoint 2

What Layers are Tested by IxChariot?

Application Testers (e.g. AppTest)

Application-level Network Testers (IxChariot)

Packet Generators (e.g., IxExplorer)

2007 Ixia. All rights reserved.

{
{

Application
Presentation
Session
Transport
Network
Data Link
Physical

How is IxChariot used?

Qualifying hardware and software

Profiling a new application to predict its impact
Benchmarking to establish Service-Level
Agreements
Verifying the network after changes are made
Voice over IP assessment testing
Multicast interoperability testing

Triple-Play testing with VoIP, Video, and Data

traffic
Wireless network testing
2007 Ixia. All rights reserved.

How is IxChariot Used in Wireless?

Tests wireless roaming performance
IxChariot is part of the official test plans for:
Wi-Fi 802.11a/WPA Test Plan
Wi-Fi 802.11b/WPA Test Plan
Wi-Fi 802.11g/WPA Test Plan

Wi-Fi PDA Test Plan

Wi-Fi WMM Test Plan
Wi-Fi WPA2 Test Plan

2007 Ixia. All rights reserved.

Using IxChariot

IxChariots Basic Design

Results Reporting

.html

.csv

.txt

Test Instructions
Application Scripts
Test Results

TCP, UDP,
RTP
IPX, SPX
IPv4, IPv6
VoIP,
Multicast

2007 Ixia. All rights reserved.

TCP, UDP,
RTP
IPX, SPX
IPv4, IPv6
VoIP,
Multicast

Performance Endpoints

Installed as a service or daemon

Execute scripts
Measure performance
Return observed data
30+ Operating systems supported
Including Windows (including Win CE), Solaris, Linux, AIX, MVS,
OS/2, NetWare, Tru64 UNIX, FreeBSD, SCO UnixWare, IRIX

2007 Ixia. All rights reserved.

Lab 1 Application Rollout

Lab Objectives
Students who complete this lab will be able to:
1.
2.
3.
4.
5.

Create a Basic IxChariot test.

Understand what StackManger is and what it is used for.
Replicate a pair and run a multi-pair IxChariot test.
Edit an IxChariot Script.
Begin to Interpret IxChariot test results.

10
2007 Ixia. All rights reserved.

Defining the Endpoint Pair

From Address

To Address
Protocol
Service Quality
Script
Modify Script

2007 Ixia. All rights reserved.

Selecting Script for Endpoint Pair

2007 Ixia. All rights reserved.

Types of Application Scripts

IPX, RTP, or UDP

protocols only
No acknowledgments or
separate connections
made

2007 Ixia. All rights reserved.

Bader Benchmark scripts

Internet scripts

Business scripts

Application Script
Run on Endpoint Pairs to emulate application traffic

2007 Ixia. All rights reserved.

Choosing Run Options

Ending a Test Run

Run Until. . .

Reporting Timings
Real-Time or Batch Reporting

Polling Endpoints
Whether and how often

2007 Ixia. All rights reserved.

Real-Time versus Batch Reporting

Real-Time Reporting
Lets you see results as they happen but
Test data flows may skew the test results

Batch Reporting
Minimized Console traffic doesnt skew the results but
You must poll to monitor a currently-running test

2007 Ixia. All rights reserved.

Lab 2 Voice over IP (VoIP)

17
2007 Ixia. All rights reserved.

Lab 2 Voice over IP (VoIP)

Lab Objectives
Students who complete this lab will be able to:
1.
2.
3.
4.

2007 Ixia. All rights reserved.

Create a basic IxChariot VoIP test

Learn what MOS means and how IxChariot calculates MOS
Generate a full duplex VoIP call
Analyze call quality using IxChariot VoIP metrics

VoIP Testing with IxChariot

How does IxChariot test VoIP?

IxChariot tests the networks ability to handle VoIP traffic

Real Time Protocol
Voice Quality
Measure E-model and R value
VoIP Standards

2007 Ixia. All rights reserved.

VoIP Overview

Real Time Protocol

Voice Quality
E-model and R value
VoIP Standards

2007 Ixia. All rights reserved.

Voice Quality
Voice Quality is a complex issue:
It depends upon the Codec used
It depends upon any QoS settings
It depends upon consistency of reception
It depends upon lost data
It depends upon throughput
It has several possible scoring methods

Scoring methods:
MOS (ITU P.800)
Mean Opinion Score
PSQM (ITU P.861) /PSQM+
Perceptual Speech Quality Measure
MNB (ITU P.861)
Measuring Normalized Blocks
PESQ (ITU P.862)
Perceptual Evaluation of Speech Quality
PAMS (British Telecom)
Perceptual Analysis Measurement System
The E-model (ITU G.107)

2007 Ixia. All rights reserved.

Why is Quality so Important?

Voice traffic is small compared to data traffic.

A voice conversation is a two-way exchange, but

voice traffic is two one-way unicasts.
The traffic is time sensitive delays can make the
conversation walkie-talkie like
Lost data does not get retransmitted because it is
streaming media

Consecutive lost data can create large gaps in the

conversation

2007 Ixia. All rights reserved.

Voice Quality Measurements

PSQM (Perceptual Speech Quality Measurement)

PSQM+
PESQ (Perceptual Evaluation of Speech Quality)
MNB (Measuring Normalized Blocks)
PAMS (Perceptual Analysis Measurement System)
British Telecom variation, requires licensing fee

MOS (Mean Opinion Score)

Subjective based upon representative samples

2007 Ixia. All rights reserved.

E-model and R value

E model ITU G.107
R value maps to a MOS score

2007 Ixia. All rights reserved.

IxChariots Role in VoIP Testing

2007 Ixia. All rights reserved.

Using the VoIP Features of IxChariot

2007 Ixia. All rights reserved.

Special VoIP Pair Settings

Special Features for VoIP
Five different codec types, emulating
Different compression algorithms
Data rates
Datagram sizes

Flexibility in changing datagram sizes

Silence suppression & voice activity
rate
Configurable jitter buffer
Quality of service
Performance and quality metrics
Scoring methods:
Jitter
lost data
consecutive lost datagrams
one-way delay
Mean Opinion Score

2007 Ixia. All rights reserved.

Lab 3 Video and Multicast

29
2007 Ixia. All rights reserved.

Lab 3 Video and Multicast

Lab Objectives
Students who complete this lab will be able to:
1.
2.
3.

2007 Ixia. All rights reserved.

Use IxChariot to emulate real video traffic

Understand IxChariots video metrics
Analyze and trouble-shoot video quality issues

Video and Multicast testing with IxChariot

IP Video testing with IxChariot

Tests the networks ability to deliver IP Video Quality of

Experience (QOE)
IP Video Metric Ideal
Established Video Metrics
Media Delivery Index (Delay Factor / Media Loss Rate)

2007 Ixia. All rights reserved.

Evaluate Network Performance with Datagrams

Datagram behavior can indicate the following

disorders:
Retransmissions (Duplicates Sent)
Network Loops (Duplicate packets received, packets arriving out of
order)
Lost packets vs. packets received late in either direction
Multiple paths (packets arriving out of order)
Frame size sensitivity

2007 Ixia. All rights reserved.

RTP: Real Time Transport Protocol

Used by many leading voice and video applications

Provides end-to-end transport functions suitable for

applications transmitting real-time data
Independent of the transport and network layers

Documented in RFC 1889

2007 Ixia. All rights reserved.

Jitter

E1

E2
Network

C
E1 sends

A
D1

2007 Ixia. All rights reserved.

B
D2=D1

C
D3D2

E2 receives

Jitter

2007 Ixia. All rights reserved.

Multicast testing with IxChariot

Test multicast interoperability among different devices

Datagrams
Sending/Receiving IP Multicast Traffic
Setting Up Multicast Groups
Evaluating Network Performance with Datagrams
Tuning Datagram Performance

2007 Ixia. All rights reserved.

Multicast and Datagram Statistics

Datagrams

Sending/Receiving IP Multicast Traffic

Setting Up Multicast Groups
Evaluating Network Performance with Datagrams
Tuning Datagram Performance

2007 Ixia. All rights reserved.

Traditional Traffic Flows

10.20.134.3
10.20.134.4

10.20.134.5
10.20.134.6
10.1.14.254

10.20.134.7
10.20.134.8
10.20.134.9
10.20.134.10
10.20.134.11
10.20.134.12

2007 Ixia. All rights reserved.

Applications that Use IP Multicast

225.1.1.1

2007 Ixia. All rights reserved.

10.1.14.254

Ethernet

Receiving IP Multicast Traffic

Router

Router

Ethernet

Tower
box

Ethernet

Router

Router
Router

Router

2007 Ixia. All rights reserved.

Ethernet

Multicast Routing
Protocol

Sending IP Multicast Traffic

Ethernet

Multicast
Receivers
Multicast
Routers

Multicast
Sender

data pruned
where no
receivers

Router

Ethernet

Router

replicated
data send
Single
Data Send

Router

Router

data propagated
to receivers
Router

2007 Ixia. All rights reserved.

Ethernet

Tower
box

Ethernet

Router

Performance Considerations
Is my network ready?
Raw capacity
Upgrades
Configuration

How much IP Multicast traffic?

Receivers quality
Send data rate
Buffering
Lost data rate
Other network traffic
How many simultaneous senders?

What happens during congestion?

TCP applications vs. UDP applications
Does QoS or bandwidth management help?

2007 Ixia. All rights reserved.

Lab 4 Triple Play

44
2007 Ixia. All rights reserved.

Lab 4 Triple Play

Lab Objectives
Students who complete this lab will be able to:
1.
2.
3.

2007 Ixia. All rights reserved.

Comprehensively assess the performance of the Triple Play

network
Interpret Voice, Video, and Date metrics
Generate HTML reports

Designing a Good Test

The Ideal Test:

Runs long enough for small relative precision
Avoids timing records shorter than 10ms.
Doesnt generate too many timing records
But, generates enough records to show fluctuations

2007 Ixia. All rights reserved.

Testing through a Firewall

E1

E2
Firewall
Admissible port =
explicit port number in script

Console

2007 Ixia. All rights reserved.

Test Data through a Firewall

E1

E2

For setup: 10115 default but now

configurable
Firewall

Console

2007 Ixia. All rights reserved.

For routing timing records -User configurable

(Change User Settings notebook)

Test Results

Interpreting Test Results

Timing Records
Measured Time
Confidence intervals
Relative precision

Three primary statistics

Throughput (really Goodput RFC 3511)
Average throughput from Layer 7

Transaction rate
Number of script transactions executed each second

Response time
Time (in seconds) required for one transaction

2007 Ixia. All rights reserved.

IxChariot Main Results Window

All results from tests are viewed in both graphical and
tabular format

2007 Ixia. All rights reserved.

Timing Records
Granular data on each timing record available

2007 Ixia. All rights reserved.

Feeling Good about Test Results

95% Confidence Interval

Well-known statistical measurement for the
reliability of the calculated average
95% likelihood that the actual average lies
between the lower and upper bound indicated
by the 95% Confidence Interval

Relative Precision
"Good" Relative Precision value is 10.00 or less
Can compare relative precision values regardless of
type of script run

2007 Ixia. All rights reserved.

Compare Tests

2007 Ixia. All rights reserved.

Working With Graphs

Creating Bar Graphs

Creating Pie Charts
Creating Histograms

Recognizing a Problem

2007 Ixia. All rights reserved.

Graph Types

2007 Ixia. All rights reserved.

Modifying the Axis

The graph axis scale is created automatically,
sometimes overemphasizing measurements

Adjusting the scale can clarify measurements

2007 Ixia. All rights reserved.

Using Pie Charts

Pie Charts highlight Throughput and Transaction rate

differences

2007 Ixia. All rights reserved.