TSM Mysteries –

TCP Window Size

Episode 1
Oxford University TSM
Symposium 2007
Michael Abel & Bruno Friess
TSM Mysteries
„ Based on personal interests,
knowledge and experience
„ Modules are part of a 2-3 day
workshop
„ Try to shed some light on the myths,
rumours and misunderstandings of
some TSM parameters

25.09.2007 (c) 2007 by eXstor GmbH 2

Warnings
„ For commands, settings, options,
values, limites, implications…
„ Check documentation valid for OS,
TSM server and TSM client level used
in your specific environment

25.09.2007 (c) 2007 by eXstor GmbH 3

Agenda
„ TCP/IP Basics
„ What is RFC1323?
„ History of TSM Defaults
„ Where can they found?
„ Measurement of the values
„ Examples

25.09.2007 (c) 2007 by eXstor GmbH 4

Why care for TCP?
„ TSM Server and client
…… communicate using TCP/IP
… … exchange large amounts of date

„ What we want to know:

… What is TCPWINDOWSIZE used for?
… Does size really matter?
… How to tune it inside TSM
… How to tune it on the OS
… How to get more out of less

25.09.2007 (c) 2007 by eXstor GmbH 5

TCP/IP Basics
„ TCP – Transmission Control Protocol
„ Positive Acknowledgement with Re-
Transmission (PAR)
„ Transmits data from one host to another as
long as the sender receives the acknowledge
„ TCP is defined in RFC 793 and has several
enhancements (RFC1323)
„ Uses a three way handshake connection

„ TCPWindowsize is part of the TCP Header

25.09.2007 (c) 2007 by eXstor GmbH 6

TCP/IP Basics
„ TCP Header

This is your windowsize.

„ 64k Limitation / IP Header 20 Bytes / TCP Header 20 Bytes

„ MTU (Maximum Transfer Unit)
„ TCPWindowsize is a „sliding window“

25.09.2007 (c) 2007 by eXstor GmbH 7

TCP/IP Basics
„ TCP three way handshake
Host A has a TCPWindowsize of 32k
and requests this from Host B
Host A Host B

Host B checks if TCPWindowsize of 32k

is ok and commits the value

Ok, all parameters are ok, Host A will start with

Transmission of the first frames

Both of them can change the TCPWindowsize whenever they want.

25.09.2007 (c) 2007 by eXstor GmbH 8

What is RFC1323?
„ RFC – Request For Comments
„ RFC1323 is from May 1992
„ What is TCP performance?
… TCP performance depends not upon the transfer rate itself, but rather
upon the product of the transfer rate and the round-trip delay. This
"bandwidth*delay product" measures the amount of data that would "fill
the pipe"; it is the buffer space required at sender and receiver to obtain
maximum throughput on the TCP connection over the path.

„ The TCP Header uses a 16bit field Æ largest

WindowSize is 64k
„ New option is a „window scale factor“

„ New recovery from losses algorithm

25.09.2007 (c) 2007 by eXstor GmbH 9

What is RFC1323?
Original WindowSize is 64k = 65535 Bytes
63k Æ 63 * 1024 = 64512 dec = fc00 hex
64k Æ 64 * 1024 = 65535 dec = ffff hex

Scale factor WindowSize

0 65535 or less
1 131.070
2 262.140
3 524.280
…
14 1.073.725.440 Bytes

Calculation is: WindowSize * 2^ScaleFactor

e.g. 65535 * 2^1 = 131.070

25.09.2007 (c) 2007 by eXstor GmbH 10

TCPWINDOW on the OS
„ RFC1323 on is not the default!!!
„ AIX, Value in tcp_sendspace
„ Linux, Value in /proc/sys/net/core/wmem_max
„ Windows
… GlobalMaxTcpWindowSize in Registry
… Key
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters

… See Microsoft Article Q224829

Very important!!

25.09.2007 (c) 2007 by eXstor GmbH 11

TSM Defaults?
„ Defaults changes
… Before TSM 5.2 it was 32 (in the dsm.opt)
… After TSM 5.2 it is 63

„ Lab recommendation
… Before TSM 5.3 it was 63 for Windows and 64
for the rest
… With TSM 5.3 and later it is 63 for all

„ TSM setting is …
„ 0 Æ ??
„ 63 Æ ??
„ More than 2048 Æ ??

25.09.2007 (c) 2007 by eXstor GmbH 12

Tool tcpdump
Information at http://www.tcpdump.org/

Tcpdump –i <interface> dst port 1500

exconsole:~ # tcpdump -i eth0 dst port 1500
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
10:43:37.035120 IP exconsolewin.intern.netarx > exconsole.exstor.de.vlsi-lm: . ack 2191904118 win 64512
10:44:03.284818 IP exconsolewin.intern.netarx > exconsole.exstor.de.vlsi-lm: . ack 77 win 64512
10:44:59.059982 IP exconsolewin.intern.gcsp > exconsole.exstor.de.vlsi-lm: S 2983155488:2983155488(0) win 64512 <mss 1460,scale 0,sackOK>

TCPWindowSize Scale factor

exconsole:~ # tcpdump -i eth0 dst port 1500

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
10:43:37.035120 IP exconsolewin.intern.netarx > exconsole.exstor.de.vlsi-lm: . ack 2191904118 win 32768
10:44:03.284818 IP exconsolewin.intern.netarx > exconsole.exstor.de.vlsi-lm: . ack 77 win 32768
10:44:59.059982 IP exconsolewin.intern.gcsp > exconsole.exstor.de.vlsi-lm: S 2983155488:2983155488(0) win 32768 <mss 1460,scale 2,sackOK>

32768 * 2^2 = 131.072 this is TSM with TCPWindowSize 128

25.09.2007 (c) 2007 by eXstor GmbH 13

Examples with 16k default
RFC1323 is off:
TCPWindowSize Request Result
0 16384 32768
32 32768 32768
63 64512 64512
64 65535 32768 or 16384

RFC1323 is on (on both sides!!)

TCPWindowSize Request Result
0 16384 32768
32 32768 32768
63 64512 64512
64 65535 32768 (Scale factor is 1)
128 131070 32768 (Scale factor is 2)

This is a window of 131.070 Bytes

25.09.2007 (c) 2007 by eXstor GmbH 14

Summary
„ TCPWINDOWSIZE is a buffer
„ The TSM param works very well
„ The value is not so important as
expected
„ Don‘t trust the default values
„ Recommendation is 63
„ RFC1323 on/off is very important
„ Talk to your network guys

25.09.2007 (c) 2007 by eXstor GmbH 15

Q&A

25.09.2007 (c) 2007 by eXstor GmbH 16

Kontaktaufnahme
„ Adresse:
eXstor GmbH
Hans-Bredow-Strasse 40
65189 Wiesbaden
„ E-Mail allgemein: info@exstor.de
„ Website: www.exstor.de
„ Telefon: +49 611 724286-0

25.09.2007 (c) 2007 by eXstor GmbH 17