VPKetonenPhoenixWi Fiperformancev10 1 PDF

Wi-Fi performance measurements
Feb 23rd, 2016

Veli-Pekka Ketonen
CTO, 7signal Solutions
Roadmap
@VPonwireless
Material
Jan at
11th , 2016
go.7signal.com/WLPC2016
Copyright © 2015 7signal Solutions, Inc.
Data collection methods
Ac#ve test Beacon analysis Packet analysis

metrics metrics metrics
Network
Full packet Spectrum
element
capture analysis metrics
counter metrics
2 Copyright © 2015 7signal Solutions, Inc.

Data collection tools
Type of metric Examples
Ac#ve test iPerf, IxChariot, Ekahau Site Survey, AirMagnet Survey, Speedtest
app, nPerf app, 7signal Sapphire, 7signal Mobile Eye app
Beacon analysis/ Ekahau Site Survey, AirMagnet Survey, Fluke Aircheck, Metageek
”Wi-Fi scanners” InSSIDer, Wi-Fi Explorer, Wi-Fi Signal, Acrylic Wi-Fi, CommView,
7signal Sapphire
Packet analysis Omnipeek, Metageek Eye P.A., 7signal Sapphire
Network element Cisco Prime, Aruba Airwave, Netscout (NeKlow, SNMP)

counter
Full packet capture AP in capture mode, Mac, Wireshark, Omnipeek, 7signal

Sapphire
Spectrum analysis Wi-Spy/Chanalyzer, Ekahau Site Survey, Cisco CleanAir, Oscium

Wi-Pry, 7signal Sapphire
* Examples, not a comprehensive list
Wi-Fi Experience Depends on  
more than Radio

Wi-Fi connection setup
L2 Discovery § Beacons, probing process

L2 Open authenPcaPon § Authentication request/response
L2 AssociaPon § Association request/response
802.1X/Radius or
L2 CapPve portal § EAP variants / captive portals
L3-4 DHCP
§ IP address, multi-phase process
L4-7 DNS
§ DNS
L4-7 ApplicaPon traffic
§ Applications, TCP, UDP, HTTP,…
L4-7 Roaming
§ Roaming (layers 2/3)
Performance metrics, active test
Association and DHCP/IP services
802.1X authentication §  Multiple phases
§  Association request –  Phase 1: DHCP discovery
– 
§  Association response – 
Phase 2: DHCP offer
Phase 3: DHCP request
§  EAP authentication –  Phase 4: DHCP acknowledged
– 
– 
Multiple phases
Depends on the EAP variant §  Success rate and time
§  Success rate and time
§  Status/reason codes
DNS services
§  Success rates and delays
§  Primary, secondary, tertiary
h\p://www.soasta.com/blog/page-bloat-average-web-page-2-mb/
Association and 802.1X authentication
1.  Started WPA supplicant
2.  Started scan Scan Pme
3.  Started open authentication

A\ach Open authenPcaPon Pme
4.  Completed open authentication Pme
5.  Started association
AssociaPon Pme
6.  Completed association
7.  Connected
Phase 1
8.  EAP authentication started
–  EAP method proposed by the server Phase 2
–  EAP selected method Phase 3

–  EAP peer certificate validated Phase 4
9.  EAP authentication completed Phase 5
*PracPcal steps which can be reported by a standard WPA supplicant

Throughput Delay/Ping
§  “Goodput”/transfer speed §  Bi-directional delay
§  TCP, HTTP “connection” §  ICMP Echo
§  UDP “connectionless” §  32B typical size
§  One or multiple connections §  “ms” and success %
§  TCP: Slow start & congestion
control must be considered
§  High latency – high
throughput network requires Web page load
long test time §  Load time and success rate
§  “Mbit/s” and success % §  Average page >2MB

TCP behavior with different devices
CongesPon
control
Slow start
h\p://repository.lib.ncsu.edu/dr/bitstream/1840.4/8268/1/TR-2012-6.pdf
Impact of slow start, iPhone 6
*Test by 7signal
Voice quality Video quality
§  G.711 (64 kbit/s), G.729 (8 §  WMV, H.262/MPEG-2, H.263/
kbit/s), ETSI 06.10 (13 kbit/s) MPEG-4, H.264/MPEG-4, H.
§  Max MOS varies/codec 265/MPEG-H
§  MOS score algorithms §  MOS score algorithms
–  PESQ - Perceptual Evaluation of –  PEVQ - Perceptual Evaluation of
Speech Quality Video Quality
–  POLQA - Perceptual Objective
Listening Quality Assessment
–  PSQA - Pseudo-Subjective Quality
Assessment
§  R-value
–  Based on ITU E-model (G.107)
h\p://www.tamos.com/htmlhelp/voip-analysis/mosandr_factor.htm
h\p://what-when-how.com/voip/e-model-based-voice-quality-esPmaPon-voip/
Packet loss Jitter
§  Packets received/packets §  Maximum deviation from
sent * 100% average value
§  Ping, bi-directional §  Unit “ms”
§  UDP, uni-directional
§  TCP, “no packet loss”
§  Unit “%”
h\p://www.embedded.com/print/4017837
Active testing, ports and protocols
§  Test payload for active test
must pass end-to-end
§  Firewalls need to allow
test traffic
§  Port numbers can vary
from 0 to 65535
§  Firewalls, proxies,
application filtering, web
accelerators
h\p://resources.intenseschool.com/ccna-prep-fundamentals-of-tcpip-transport-applicaPons-and-security/

Performance metrics, active/passive
Radio retransmissions Data rate/MCSs
§  802.11: If packet is not §  What share of packets use
ACK’ed, a complete packet certain rate/MCS
retry follows §  Data packets use rate control
§  Radio is persistent, will try up always. More reliable to
to 7-25 times for the same calculate rate usage from
packet, same or lower MCS them
§  Retry % (alt. 1) = packets with §  Rate control adjust rates
retry bit on/all attempted based on quality of link and
packets this indicates well signal
§  Retry % (alt. 2) = What share conditions experienced by AP
of original packets require and client
retry (one or many, no matter) §  Rate control algorithms are
§  Active & passive test not standardized in 802.11
§  Active & passive test

Performance metrics, passive test
CRC/FCS errors Channel utilization
§  Receiver synchronizes to packet §  QBSS based utilization
if the 802.11 header (1 or 6 Mbit/ –  How the AP experiences the
s) CRC is correct air utilization?
§  After synch, whole packet is –  Information in beacon IE
transferred –  802.11e element
§  Frame Check Sequence, FCS at –  255 states
the end
–  Supported increasingly
§  Error rate = packets with correct
FCS/all attempted packets §  Packet capture based utilization
–  Calculate based on packet
§  Some chipsets do not show
capture
other than FCS OK packets =>
cannot calculate error rate % –  How much airtime captured
§  Dilemma: Include or do not packets require vs. whole
capturing time
include bad FCS packets?
Air Utilization: Packet capture based model
Inter Frame ArbitraPon Pme Mandatory IniPal
AIFS (used as an average arbitraPon Pme) Back Off Pme
37-70us/frame 13.5-67.5 us/frame
PLCP training Data packet AIFS Back off

field and DuraPon = size/rate Working assumpPon here is that some
header, 20us back off is always present.
PLCP header/preamble Pme +

Frame transmission Pme +
Air u#liza#on = arbitraPon Pme + back off Pme
Total capture Pme

16
Copyright © 2015 7signal Solutions, Inc.
Beacon availability Channel number
§  Passive and active scan for §  Static or automated allocations
beacon capture §  Balanced use of all channels
§  0% or 100% §  Channel #
§  # of BSSIDs/channel
Signal level Noise level

§  Signal level (or SINR level) §  Reported by the driver
§  Static or automated AP power §  Hard to measure accurately
levels with any Wi-Fi chip
§  Surveys §  dBm
§  dBm (dB)

Data volume Clients/STAs
§  Total volume §  Number of clients
§  Volume/frame type §  QBSS based (beacon)
§  Volume/client §  Observed traffic based
§  MB/Mbit §  Devices manufacturers/types
–  MAC range/OUI
Frame counts Frame sizes

§  Busiest senders §  Average frame size
§  Busiest channels §  Impact of different traffic types
§  Number of frames/min §  Impact of fragmentation
§  Bytes

Performance metrics, active/passive test
802.11 Status codes 802.11 Reason codes
§  802.11 Association Status §  802.11 De-authentication
codes Reason codes
§  Collected from packet capture §  Collected from packet capture
§  Provide information about §  Provides information on
clients association issues
reasons for clients leaving the
§  For example AP
–  Code 17: Association denied due to
overload of the AP §  For example
§  Listed –  Code 5: Disassociated because AP is
–  https://supportforums.cisco.com/document/ unable to handle all currently
141136/80211-association-status-80211- associated STAs. The access point is
deauth-reason-codes busy, performing load balancing, for
–  http://community.arubanetworks.com/t5/
Technology-Blog/802-11-Reason-Codes-
example.
and-Status-Codes/ba-p/257893
h\p://www.aboutcher.co.uk/2012/07/linux-wifi-deauthenPcated-reason-codes/
Performance metrics
TCP socket failure codes Spectrum analysis
§  TCP specific codes for failed §  Spectrum analysis
connections –  QCA/Atheros Wi-Fi chip integrated
– 
§  For example
– 
TI CC2500 chip
Cisco CleanAir (“Cognio”) chip
–  Connection refused
–  Bandspeed chip
–  Network unreachable
–  Socket read error §  Line graphs
–  Attach to AP failed §  Water wall graphs
–  DHCP failure
–  IP address set failure
§  Density graphs
–  Route remove failure §  3D graphs
–  Host unreachable
§  Fingerprinting
–  Connection timed out

How to organize the data?

Why graphs are REALLY needed?
ANSCOMBES QUARTET:
The same mean, variance, correla#on & regression line
Lesson: Always Visualize Data
h\p://data.heapanalyPcs.com/anscombes-quartet-and-why-summary-staPsPcs-dont-tell-the-whole-story/
Organizing the data
§ Key Performance Indicators (KPI)

–  Test results on time axis
–  Visualizations, trending lines
§ Time averages
–  Often does not make sense to show all samples
–  Time averages improve clarity of the data and help with making
the right conclusions
–  Averages: 1min, 10min, 1 hour, 1 day, 1 week, specific period
§ Aggregations
–  BSSID, SSID, frequency band, channel, service area
–  LAN, Wi-Fi
–  Test end points

Service Level Agreement, SLA
§ Helps to quickly determine is the service working as

required
§ Macro KPIs and SLA tables
§ KPI types, consider the difference
–  Individual test is success or fail, value 0 or 100%
§  Test success rate
–  Individual tests have infinite amount of values, “analog nature”
§  Downlink throughput
§ SLA calculation methods
–  Average vs. threshold
–  Sample count vs. threshold

SLA calculation, average based 
Compliancy with different threshold values
SLA% = 21Mbps/30Mbps
= 70%
30 Mbit/s
AVG=21 Mbit/s
SLA% = 21Mbps/5Mbps
=> 100% SLA% = 21Mbps/2Mbps
=> 100%
5 Mbit/s
2 Mbit/s

SLA calculation, sample count based 
Compliancy with different threshold values
SLA% = 10/25 samples

= 40%
30 Mbit/s
SLA% = 19/25 samples

= 76% SLA% = 23/25 samples
= 92%
5 Mbit/s
2 Mbit/s

Charting
§ Basic chart types

–  Line graphs, bar charts, pie charts
§ X-Y charts
–  Correlation mapping
§ Averages
–  Geometric average Geo_avg = 3√(x13 x23 x33)
–  Arithmetic average Arith_avg = (x1 + x2 + x3)/3
§ Min, max, percentile (5%, 95%)
–  Calculated inside the averaging period sample
§ Cumulative Distribution Functions (CDFs)
–  Continuous distribution graph

Impact of sampling density
Event in
Passive test network
Time
Measurement Measurement
Performance
Ac#ve test of network
Time
Measurement Measurement
§ Manual or automated, your test data rarely covers

everything
How to use the data and  
make conclusions?

Collecting the test data
Site Trend Email SNMP

surveys analysis reports alarms
One off SLA Email

tests failure alarms

Drilling in to issue
SSID
Frequency
band
Service
area
AP/BSSID
Segment
Client
vendor
Client

Spotting the problems from data
Missing ProblemaPc
High packet status/reason Too many
beacons/ clients/AP
loss/ji\er codes
dead AP
Failing to High air

High delay Interference uPlizaPon
authenPcate
ConPnuous
Low Failing to channel High retries/
throughput get IP low MCS
changes
No e2e Poor channel

connecPvity allocaPons
Ac#ve test Passive test

Data from a major University in US
•  Network has been already optimized and works well in most

aspects
•  Careful analysis reveals still room for further checks
•  Challenging, open environment
•  HT20 channels used for maximum capacity
•  Example data on observed network performance in the main
student SSID
•  Data from the 1st floor with 8 APs, where a lot of students spend
time
•  2 weeks period around the time school starts

Dashboard Macro KPIs
ConnecPvity IP services
•  Beacon availability •  IP address Pme
•  Radio a\ach success rate •  IP address success rate
•  Radio a\ach Pme
•  AssociaPon success rate
•  Radio associaPon Pme
•  TCP test success rate
AuthenPcaPon Quality
•  Open radio authenPcaPon success rate •  TCP DL throughput
•  Open radio authenPcaPon Pme •  TCP UL throughput
•  Radius: Total EAP authenPcaPon •  Ping RTT
success rate •  Web page load Pme
•  VoIP MOS DL
•  VoIP MOS UL

Dashboard, 1st floor 
1 month period
Period of interest
SLA: Connectivity and Authentication 
One hour averages
School off School starts 5 GHz doing well, but

li\le below 100%
2.4 GHz struggles

with load
School off School starts 5 GHz doing well
2.4 GHz struggles

with load

SLA: Connectivity and Quality 
One hour averages
School off School starts 5 GHz doing well, but

li\le below 100%
2.4 GHz struggles

with load
School off School starts 5 GHz doing well
2.4 GHz struggles

with load

SLA: Table view for 5 GHz 
Daily averages
Retries above
Good performance
10% target
level

SLA: Table view for 2.4 GHz 
Daily averages
Voice quality UL
below target
Retries above 10%

target level
Radio a\ach not on Throughput not on

target level target level

Traffic volume

Station count and channel utilization 
AreaBand average
70 clients/AP at
5 GHz
5 clients/AP at
2.4 GHz
60% uPlizaPon at
2.4 GHz
25% uPlizaPon at
5 GHz

2.4 GHz/AP specific
Max 12 clients/AP
(AP140)
Up to 80%
uPlizaPon (AP140)

5 GHz/AP specific
Max 100
clients/AP
(AP156). Others
have clearly less
Up to 90%
uPlizaPon
(AP 102)

Frame volumes, Uplink/Downlink
5 GHz band carries much

higher traffic load/AP

End-to-end performance KPIs

TCP throughput, Uplink/Downlink (active)
Drop
Drop

Voice quality (MOS) and RTT (active)
Peak load
increases
latency a lot at
2.4

Jitter and Packet Loss (active)

Probe request and response air utilization
Requests and
responses in
balance

Beacon air utilization and frame errors
Looking good: Beacons consume

very limited airPme

Retransmissions, AP/client (passive)
AP retries at both bands

around 50%-60%
Client retries at both

bands around 30%

Spotting specific issues

SLA: Connectivity and Authentication 
2.4 GHz APs
AP 156 Failing Degraded AP 127 Failing

connecPvity connecPvity connecPvity
AP 156 Failing AP 127 Failing

authenPcaPon authenPcaPon

SLA: IP services and Quality 
2.4 GHz APs
Several APs
lacking IP compliancy
Degraded
quality

IP address success rate and time 
2.4 GHz APs
IP is always received
…a bit long delay to get IP.

SLA set at 5s.

Reasons for low IP address score 
2.4 GHz APs
Significant delay: DHCP

host not responsive
Delay gerng offer:

DHCP host not responsive

Reasons for low connectivity scores  
2.4 GHz: AP127 and AP156 connectivity issues
APs send/beacons
received pre\y normally
AssociaPon
success rate degraded AP 156 & 127 open
authenPcaPon fails

Reasons for low connectivity scores  
2.4 GHz
Part of the APs need clearly

longer Pme for associaPon

Throughputs, daily minimum
Significant slowdown under load, but 5 GHz

remains sPll at mulP Mbps speed
AP140 lowest
Significant slowdown under load, but 2.4 GHz

remains sPll at 1 Mbps speed (AP140)

2.4 GHz channel usage
Vast majority of the

nearby APs use channel 1


5 GHz channel usage

In-balanced channel
usage, unused capacity

Reason codes (passive analysis)
“4-way handshake Pme

outs” with mulPple APs
“AuthenPcaPon failed” with

mulPple APs, especially with
AP 140

Plan for action
§ Check DHCP server serving certain APs, like 140, 102-2
–  Impacts both bands
–  Are there two separate DHCP servers being used by measured APs?
–  Allocate more DHCP capacity to speed up responses
§ Restart AP156 and AP127
–  Observe if open authentication issues disappear
–  If not, consider replacing the APs next
§ Balance usage of channels on both bands
–  Helps to reduce throughput drops during high load
–  Helps to perform successful associations and open authentications
§ Improve especially AP140 (2.4 & 5 GHz) RF performance
–  AP140 regularly experiences 60-80% air utilization at 2.4 band
–  AP140 also experiences lowest throughputs with active tests in both
bands
–  “Authentication failed” codes may be due to RF issues
–  => Consider new channel and power settings for better channel
quality and better load balancing between APs

Demo

Cisco, Apple, Xirrus,…

Thank You!
Email: veli-pekka.ketonen@7signal.com
Presentation: go.7signal.com/WLPC2016
www.7signal.com
@VPonwireless

VPKetonenPhoenixWi Fiperformancev10 1 PDF

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

VPKetonenPhoenixWi Fiperformancev10 1 PDF

Загружено:

Авторское право:

Доступные форматы

Wi-Fi performance measurements

Feb 23rd, 2016

Ac#ve test Beacon analysis Packet analysis

2 Copyright © 2015 7signal Solutions, Inc.

Network element Cisco Prime, Aruba Airwave, Netscout (NeKlow, SNMP)

Full packet capture AP in capture mode, Mac, Wireshark, Omnipeek, 7signal

Spectrum analysis Wi-Spy/Chanalyzer, Ekahau Site Survey, Cisco CleanAir, Oscium

4 Copyright © 2015 7signal Solutions, Inc.

L2 Discovery § Beacons, probing process

3. Started open authentication

– EAP selected method Phase 3

9. EAP authentication completed Phase 5

*PracPcal steps which can be reported by a standard WPA supplicant

8 Copyright © 2015 7signal Solutions, Inc.

13 Copyright © 2015 7signal Solutions, Inc.

14 Copyright © 2015 7signal Solutions, Inc.

PLCP training Data packet AIFS Back oﬀ

PLCP header/preamble Pme +

Total capture Pme

Signal level Noise level

17 Copyright © 2015 7signal Solutions, Inc.

Frame counts Frame sizes

18 Copyright © 2015 7signal Solutions, Inc.

20 Copyright © 2015 7signal Solutions, Inc.

21 Copyright © 2015 7signal Solutions, Inc.

§ Key Performance Indicators (KPI)

23 Copyright © 2015 7signal Solutions, Inc.

§ Helps to quickly determine is the service working as

24 Copyright © 2015 7signal Solutions, Inc.

25 Copyright © 2015 7signal Solutions, Inc.

SLA% = 10/25 samples

SLA% = 19/25 samples

26 Copyright © 2015 7signal Solutions, Inc.

§ Basic chart types

27 Copyright © 2015 7signal Solutions, Inc.

§ Manual or automated, your test data rarely covers

29 Copyright © 2015 7signal Solutions, Inc.

Site Trend Email SNMP

One oﬀ SLA Email

30 Copyright © 2015 7signal Solutions, Inc.

31 Copyright © 2015 7signal Solutions, Inc.

Failing to High air

No e2e Poor channel

Ac#ve test Passive test

• Network has been already optimized and works well in most

33 Copyright © 2015 7signal Solutions, Inc.

34 Copyright © 2015 7signal Solutions, Inc.

School oﬀ School starts 5 GHz doing well, but

2.4 GHz struggles

School oﬀ School starts 5 GHz doing well

2.4 GHz struggles

36 Copyright © 2015 7signal Solutions, Inc.

School oﬀ School starts 5 GHz doing well, but

2.4 GHz struggles

School oﬀ School starts 5 GHz doing well

2.4 GHz struggles

37 Copyright © 2015 7signal Solutions, Inc.

38 Copyright © 2015 7signal Solutions, Inc.

Retries above 10%

Radio a\ach not on Throughput not on

39 Copyright © 2015 7signal Solutions, Inc.

40 Copyright © 2015 7signal Solutions, Inc.

41 Copyright © 2015 7signal Solutions, Inc.

L2 Discovery § Beacons, probing process

3.  Started open authentication

–  EAP selected method Phase 3

9.  EAP authentication completed Phase 5

§ Key Performance Indicators (KPI)

§ Helps to quickly determine is the service working as

§ Basic chart types

§ Manual or automated, your test data rarely covers

•  Network has been already optimized and works well in most