MTS documentation / Benchmark document

Benchmark
document
Load performances
Version: %VERSION%

Version: %VERSION% 1
MTS documentation / Benchmark document

Table of contents
1. Introduction..................................................................................................................2
2. Load figures on host #1...............................................................................................2
3. Load figures on host #2 (Linux Redhat)......................................................................4
4. Load figures on host #3 (Linux Ubuntu) chez FT/RD................................................5
5. Load figures on host #3 (Windows XP)......................................................................6
6. Load figures on host #4 (Windows 7).........................................................................8
7. Load test description....................................................................................................9
7.1. SIP: UAC-UAS....................................................................................................9
7.2. SIP/RTP: UA-Sender.........................................................................................10
7.3. Diameter: Client-server......................................................................................11
7.4. Http: Client-server.............................................................................................11
7.5. SIP: register-ua..................................................................................................12

1. Introduction
This document contains the benchmark test results for the MTS product containing the
performance figures for the mainly used protocol.
All the tests are made with one host acting as the client and the server with a loopback
protocol exchange. We put a IP route to force the network packet to leave the host to the
IP gateway and come back.

This benchmark document contains some figures measured on 2 hosts: a standard one
and another which suits more to have higher performance.

Here is a short summary of the load figures on a Dell 5500 computer for the mainly used
protocols (IMS architecture):

SIP: 125 CAPS out + 125 CAPS in, 67000 sessions (SIP call only)
SIP/RTP: 750 sessions (SIP/RTP send only)
SIP UA: 600000 subscribers (SIP register call) => 30000 calls (0.1 erlang)
Diameter: 5000 transaction/s
HTTP: 2800 transaction/s

Note: that we manage the protocol sockets in a nio (non-blocking) way; ie that we can
open a large number of listenpoint or channel (TCP connection) simultaneously.

The means duration of calls is also set to 300 seconds. We assume in the following a
traffic of 0.1 erlang; it means than 1/10 of the subscribers are involved in a call at any
time.

2. Load figures on host #1

Hardware: Dell 5500

Version: %VERSION% 2
MTS documentation / Benchmark document
Processor: 1x6 cores processeurs à 2.5 Gb (ou 1x8 hyperthreading ?)
RAM: 4Gb
Network: 100Mhz

Software:
System: LINUX Ubuntu 10.0.4
JRE: 1.6
MTS: V5.2.1

DellTest: 192.168.90.233

Protocol Type test Comment Flow CAPS CPU RAM

Trans/s out
/s
DIAM Client- *tutorial/diameter/load/ 5000 70% 1.4Gb
server test.xml
* 10 testcases

SIP Uac-Uas * tutorial/sip/load/invite 1800 300 75% 6.2Gb

udp * 10 testcases
* call duration 0s

SIP Uac-Uas * tutorial/load/async_ 1200 200 55% 4.2Gb

Async basic_call
* call duration 300s
* 60000 calls
*1200000 subscribers

HTTP Client- * tutorial/http/ load/get 4388 60% 1.1Gb

Server * 1000 bytes
* without close channel

CORE Empty *tutorial/core/load/empt 78237 70% 262Mb

y/test.xml testcase/s
* 100 testcases

CORE Empty *tutorial/core/load/empt 29107 Thread ? 70% 4.1Gb

y/test.xml
* 50000 testcases

SIGTRAN Client- * tutorial/sigtran/iua/ 1580 112 62% 1.3Gb

IUA Server load/test-load.xml msg/s
* keypad_facility
* without close channel
* 100 testcases

Version: %VERSION% 3
MTS documentation / Benchmark document
MSRP Client- tutorial/msrp/load/send 5090 78% 4.2Gb
server * 1000 bytes
* without close channel

MSRP Client- tutorial/msrp/load/send 2145 60% 4.2Gb

server * 10000 bytes
* without close channel

MSRP Client- tutorial/msrp/load/send 300 55% 4.2Gb

server * 100000 bytes
* without close channel

3. Load figures on host #2 (Linux Redhat)

Hardware: HP
Chassis: HP Z600 650W 85% Efficient Chassis –
Processor: 2*Intel Xeon E5506 2.13Ghz 2*4 cores 4MB/800 QC CPU -
RAM: HP 4GB(2x2GB)DDR3-1333 ECC -
Video card: NVIDIA Quadro FX380 256MB,1st Card -
Hard disk: 2*HP 500GB SATA 7200,1st HDD –
Network Card : 100Mbits

Software:
Red Hat Linux Entreprise 5 OS
JRE: 1.6.0 open jdk
MTS: V5.5.2

Dragon: 192.168.190.13

Protocol Type test Comment Flow CAPS CPU RAM

Trans/s out
/s
SIP Uac-Uas * tutorial/sip/load/invite 1050 175 55% 2.5Gb
Udp * 10 testcases
* call duration 0s

SIP Uac-Uas * tutorial/load/async_ 720 120 55% 2.2Gb

async basic_call
* call duration 300s

Version: %VERSION% 4
MTS documentation / Benchmark document
* 36000 calls
*720000 subscribers

SIP/ UA *tutorial/rtpflow/load/ca 15 (SIP) 5 (SIP) 35% 1.5Gb

RTP flow -Sender ll-sendonly 50000
* Send flow only packets
* 1000 sessions RTP (RTP)
* call duration 180s

DIAM Client- *tutorial/diameter/load 1000 50% 950Mb

server * 10 testcases

HTTP Client- * tutorial/http/ load/get 1150 45% 696Mb

Server * 1000 bytes
* without close channel
* 10 testcases

RADIUS Client- 1400 55% 1.5Gb

server

4. Load figures on host #3 (Linux Ubuntu) chez

FT/RD
Hardware: Dell 5400
Processor: 2x4 cores processeurs à 2.5 Gb
RAM: 4Gb
Network: 100Mhz

Software:
System: LINUX Ubuntu 10.0.4
JRE: 1.6
MTS: V5.0.0

Protocol Type test Comment Flow CAPS CPU RAM

Trans/s out
/s
SIP Uac-Uas * tutorial/sip/load/invite 1200 200 50% 2.6Gb
Udp * 10 testcases
* call duration 0s

Version: %VERSION% 5
MTS documentation / Benchmark document
SIP Uac-Uas * tutorial/load/async_ 900 150 50% 2.6Gb
async basic_call
* call duration 300s
* 45000 calls
*900000 subscribers

SIP/ UA * Send flow only 14500 50% 1.5Gb

RTP flow -Sender * 500 sessions RTP
* call duration 300s

SIP Register- * tutorial/load/async_ 60 10 20% 2Gb

end-end Ua basic_register
* call duration 300s
* 3000 calls
*60000 subscribers
*Linux only not suitable
on Windows
DIAM Client- *tutorial/diameter/load 1500 50% 1.1Gb
server * 10 testcases

HTTP Client- * tutorial/http/ load/get 1885 50% 535Mb

Server * 1000 bytes
* without close channel
* 10 testcases

Note: The host is a bi-processor; so the MTS process uses only one processor and that
why the CPU consumption is always 50%. If you run another MTS process in parallel
then you will have more or less performance twice better. To take this phenomenon into
account we multiply the performance above by 1.5.

5. Load figures on host #3 (Windows XP)

Hardware:
Processor: ADM Athlon 64 X2 Dual Core processor 3800+ 2Ghz
RAM: 2Gb
Network: 100Mhz

Software:
System: Windows XP (office configuration) 32bits
JRE: 1.6.0_20
MTS: V4.8.0

Version: %VERSION% 6
MTS documentation / Benchmark document

DVTFR2518: 172.16.21.32

Protocol Type Comment Flow CAPS CPU RAM

test Trans/s half
/s
SIP Uac- * tutorial/sip/load/invite 570 95 90 1064
udp Uas * 10 testcases Mb
* call duration 0s

SIP Uac- * tutorial/sip/load/invite 688 115 90 1072

tcp Uas * 10 testcases Mb
* call duration 0s

SIP Uac- * tutorial/sip/ load/invite 679 115 90 1065

tls Uas * 10 testcases Mb
* call duration 0s

SIP/ UA- * Send flow only 12135 95 1359

RTP flow Sender * 200 sessions RTP Mb
* call duration 300s

DIAM Client- *tutorial/diameter/load 969 91 390Mb

server
HTTP Client- * tutorial/http/ load/get 1600 20(net) 1Gb
server * 1000 bytes
* without close channel

HTTP Client- * tutorial/http/ load/get 280 4(net) 1Gb

server * 1000 bytes
* with close channel

IMAP Client- *tutorial/imap/load 1900 290 Mb

server
POP Client- *tutorial/pop/load 2300 225 Mb
server
RTSP Client- *tutorial/rtsp/load 2100 710 Mb
server
SMPP Client- *tutorial/smpp/load 1600 625 Mb
server
UCP Client- *tutorial/ucp/load 1300 290 Mb
server
H248 Client- *tutorial/h248/load/ 422 35 1290
text server std_call with K Mb
-xmx1200 msg incl

Version: %VERSION% 7
MTS documentation / Benchmark document
MSRP Client- tutorial/msrp/load/send 1144 12(net) 956 Mb
server * 1000 bytes
* without close channel

MSRP Client- tutorial/msrp/load/send 477 50(net) 1.200

server * 10000 bytes Mb
* without close channel

MSRP Client- tutorial/msrp/load/send 21 50(net) 1.200

server * 100000 bytes Mb
* without close channel

SIGTRAN Client- * tutorial/sigtran/iua/ 300 100 1.1gb

IUA server load/test-load.xml msg/s
* keypad_facility
* without close channel
* 20 testcases

CORE Empty *tutorial/core/load/empt 15500 100 162Mb

y/test.xml
* 100 testcases

CORE Empty *tutorial/core/load/empt 5409 24000 1.2Gb

y/test.xml threads
* 10000 testcases ?

RADIUS Client- *tutorial/radius/load/test 1200 98% 1.1Gb

server -load.xml
GTP-U Sender *tutorial/gtp/load/test.x 6500 85 800Mb
ml
UDP Sender *tutorial/udp/load/test- 5700 80 155Mb
perf.xml

6. Load figures on host #4 (Windows 7)

Hardware:
Model : HP EliteBook Folio 9470m
Processor: Intel Core i5-3427U CPU @1.8Ghz 2.3Ghz
RAM: 8Gb
Network: 100Mhz

Software:
System: Windows 7 (office configuration) 64bits
JRE: 1.7.0_75
MTS: V6.1.0

Version: %VERSION% 8
MTS documentation / Benchmark document

FR00115331: 137.58.84.39

ID Protocol Comment Flow Result CPU RAM

Resp time Net
Trans/s Proc Time % Mb
Retrans
01 SIP * tutorial/sip/load/invite 1395 Resp 8.5ms 98% 2155
udp * call duration 0s 231CAPs Proc 1.05ms 23%
* duration 38h50 Retrans 0.02%

02 DIAM *tutorial/diameter/load 2267 Resp 4.1ms 90% 673

tcp * duration 14h30 Proc 1ms 20%
Retrans 0.0%

03 RADIUS * tutorial/radius/load 2524 Resp 2.9ms 95% 1359

* duration 14h12 Retrans 0.0% 2.6%

04 SIPasync * tutorial/load/async_ 600 Resp 9.3ms 60% 1593

basic_call Proc 0.15ms 7%
* call duration 300s Retrans
* 100 CAPS 0.0009%
*62h25min
05 DIAM *tutorial/diameter/load 2343 Resp 3.9ms 90% 660
Tcp light * duration 61h47 Proc 1,9ms 21%
* 6.1.0 beta Retrans 0.0%

06 DIAM *tutorial/diameter/load 2137 Resp 4.1ms 90% 815

Tcp light * duration 61h47 Proc 1.6ms 20%
* 6.1.0 beta

07 DIAM *tutorial/diameter/load 2142 Resp 4.1ms 95% 595

Tcp light * duration 13h25 Proc 1.6ms 22%
* 6.1.0 beta
08 SIP Hybrid *tutorial/sip/load/hybrid 600 Resp 35.25ms 35% 2650
_basic_call 100 CAPS Proc 4.5ms 9%
10h00

Version: %VERSION% 9
MTS documentation / Benchmark document

7. Load test description

7.1. SIP: UAC-UAS

This test consists in setting up a call at a point between 2 servers in the networks. ie it
means that all the SIP flow is exchanged on the same transport port. In that case, we only
simulate the signaling (no RTP flow). For that we are using a testcase with 2 scenarios:
one acts as the UAC agent (caller) and other as the UAS (callee).

For the test in a synchronous way the call duration is 0s; for the asynchronous test, the
call duration is set to 300 s.

INVITE
Pick-up
Call setup 180 Ringing
200 OK
Ack ACK Accept

BYE
Hang up
Answer 200 OK Answer

UAC UAS

7.2. SIP/RTP: UA-Sender

This test consists in setting up a call at a point between 2 servers in the networks. ie it
means that all the SIP flow is exchanged on the same transport port. In that case, we
simulate both the signaling (SIP) and the data flow (RTP). For that we are using a
testcase with 2 scenarios: one acts as the UAC agent (caller) and other as the UAS
(callee). Only the UAC sends RTP flow (half duplex); and we does not receive the flow
on the other side to calculate QOS parameters.

Version: %VERSION% 10
MTS documentation / Benchmark document

INVITE
Pick-up
Call setup 180 Ringing
200 OK
Ack ACK Accept

RTP flow
Talk Talk

BYE
Hang up 200 OK Answer
Answer

UAC UAS
sender

7.3. Diameter: Client-server

The goal of this test is to simulate a Diameter client and server in the same instance of
MTS product. For that we have 2 testcases: one for the client and the other as the server.

Request MMR (303)

Answer MMA (2001 Success)

Client Server

7.4. Http: Client-server

The goal of this test is to simulate a Http client and server in the same instance of MTS
product. For that we have 2 testcases: one for the client and the other as the server. All the
HTTP traffic is supported by the same TCP connection.

Version: %VERSION% 11
MTS documentation / Benchmark document

Request GET

Response 200 OK

Client Server

7.5. SIP: register-ua

This test consists in setting up a call in the place of the endpoints (User agent) ie it means
that all the SIP flow is exchanged through one listenpoint per terminal. In that case, we
only simulate the signaling (no RTP flow). For that we are using a testcase with 2
scenarios: one acts as the UAC agent (caller) and other as the UAS (callee). On another
host, we are using antoher instance of IMslaoder in order to simulate the REGSITRAR
function.

REGISTER
REGISTER Registration
Registration
200 OK
200 OK
Pick-up
INVITE
Call setup
180 Ringing Accept
Ack 200 OK
ACK
BYE
Hang up 200 OK Answer
Answer

REGISTRAR UAS
UAC

Version: %VERSION% 12