Вы находитесь на странице: 1из 71

SAN

Brocade
Best Practice Planning

: 0.9
:

- 2007


.....................................................................................................................................................................................4
SAN..........................................................................................................................................5
.........................................................................................................................................................................5
SAN......................................................................................................................................................................5
..............................................................................................................................................6
...........................................................................................................................................................................7
.............................................................................................................................................................7
FC.......................................................................................................................................................8
....................................................................................................................................9
.....................................................................................................................................................................9
Fan-out Fan-in................................................................................................................................................................10
.......................................................................................................................................................................11
......................................................................................................................................................11
..................................................................................................................11
HBA ..................................................................................................................................................................................12
...................................................................................................................................................12
..........................................................................................................................................................13
........................................................................................................................................................................13
Brocade................................................................................................................................................16
.......................................................................................................................................................................16
200E ..................................................................................................................................................................................16
4100...................................................................................................................................................................................16
4900...................................................................................................................................................................................17
5000...................................................................................................................................................................................17
48000.................................................................................................................................................................................17
FC10-6...............................................................................................................................................................................17
iSCSI ........................................................................................................................................................................18
FC ............................................................................................................................................................18
...................................................................................................................................................................18
- ................................................................................................................................................................................19
...................................................................................................................................................................20
Brocade......................................................................................................................................21
CCMA ............................................................................................................................................................21
...........................................................................................................................................................26
..............................................................................................................................................................28
............................................................................................................................30
.................................................................................................................33
200E ..................................................................................................................................................................................33
5000...................................................................................................................................................................................34
4900...................................................................................................................................................................................35
48000.................................................................................................................................................................................36
FC4-16...............................................................................................................................................................................38
FC4-32...............................................................................................................................................................................40
FC4-48...............................................................................................................................................................................42
FC10-6...............................................................................................................................................................................45
FR4-18i .............................................................................................................................................................................46
7500...................................................................................................................................................................................48
FC4-16IP ...........................................................................................................................................................................49
Brocade ...............................................................................................................................................................51
Domain ID .......................................................................................................................................................51
.................................................................................................................................................52
ISL .................................................................................................................................................56
...................................................................................................................................59
FCIP .........................................................................................................................................................60
iSCSI ..........................................................................................................................................................63

SAN Brocade

2 71

.........................................................................................................................................................................65
..............................................................................................................................................66
.............................................................................................................................66
..................................................................................................................................................................67
....................................................................................................................................................................68
.........................................................................................................................................................69
..............................................................................................................................................................................71

ASIC..............................................................................................................................................................................................22
Blade Processor .............................................................................................................................................................................46
EMC Connectrix B M-series......................................................................................................................................................19
Fast Write and Tape Pipelining .....................................................................................................................................................32
LWL ELWL SFP..........................................................................................................................................................30
RSCN ............................................................................................................................................................................................65
WWN Brocade ............................................................................................................................................................45
Cisco MDS 9513 ....................................................................................................................................................25
...................................................................................................................................................22
fabric.ops.mode.longDistance ...........................................................................................................................................27
FCID...........................................................................................................................................................................40
IOD ......................................................................................................................................................................................38
, EMC..................................................................................................................15
...............................................................................................................................................................................54

SAN Brocade

3 71


EMC
. SAN Cisco MDS
SAN Fibre Channel Cisco.
, Brocade.
McData Brocade
. ,
. .

, Brocade McData.
.

. ,
.
.
. , ,
SAN .
. , ,
,
, , .
Brocade,
- .
, Fibre Channel.

SAN Brocade

4 71

SAN
, .
Brocade.
Brocade ,
, , ,
.


:
z
SAN
z

z

z

z

,
.
, , .
SAN

, . ,
SAN, .
,
,
(, , ).
, ,
? , Fibre
Channel (FC_AL P-2-P), SAN, . . .
,
. (.. 1).
, SAN,
.

(Cascade)

2-3


(Cascade Ring)

3-5

4-8

. ,
. .

4-8


(Full Mesh)

(Partial Mesh)
-
(Core-Edge)

,
. .

. 1

SAN Brocade

5 71

(Tiered fabric).
(Storage Tier),
(Host Tier) , ISL
(Connectivity Tier).
SAN :
z
, 3 ISL
( 3 hops).
z
31
z
N- 2048

(.
).
(, )
(, ).
SAN, , ,
MetaSAN. Brocade
FC .


, ISL (Inter-Switch Link) ?

. :
z
(, , ).
z
. ,
ISL (. ).
z
. ,
ISL ( ISL).
, ISL .
, , , .
, , ,
5 . . ,
.
. , ,
. , ,
?
?
,
.
, .
, ,
, .

. ,
.
.

SAN Brocade

6 71

, .
.
, , , SAN
.

(www.brocade.com/products/SAN_interop_and_compatibility.jsp) .
, ,
.
. 2.
Brocade
Interopmode0
Interopmode1
M-Series Fabric
Open Fabric

Qlogic

Standard
Brocade M-Series Fabric
Standard

Cisco MDS
Interop-2 (Core PID=0)
Interop-3 (Core PID=1)
Interop-1
Interop-4
Interop-1
. 2

Brocade

Brocade M-

Interopmode0

Interopmode1

Interopmode1

Open Fabric
Brocade M-Series Fabric
Open Fabric

SAN :
z
.
(Multipathing SW), EMC PowerPath,
Veritas DMP, Hitachi HDLM, Sun MPxIO,
.
z
SAN (Redundant SAN , , Mirrored fabrics).
.
.
.
.
, Redundant SAN , .
, .
,
, .
SAN. LWM SFP
.
FCIP, xWDM
IP SAN iSCSI.
.
, ,
, . . ,
on-line.

,
- . ,

?
SAN ,

.
.
.

SAN Brocade

7 71

.
.
, SAN
, .
FC IP SAN .
.
FC
Fibre Channel,
,
Gbps (Gbit/s). ,
, MB/s.

1,063Gbps, 2,126Gbps 4,252Gbps, Fibre Channel

- 100MB/s, 200MB/s 400MB/s.
, . . .
, HBA , 50MB/s,
2Gbps, ,
~2GHz. ,
, IDLE.
.
Fibre
Channel 3 . FC-
(autonegotiation), .
10 Gigabit Fibre Channel Standard (10GFC)
FC-.
12,75Gbps. , 10GFC
8b/10b 64b/66b .
, 1, 2 4 Gbps .
,
. . 3
.

SAN Brocade

8 71

SMF

MMF (50m)

MMF (62.5m)

400-SM-LL-I
200-SM-LL-V
200-SM-LL-I
100-SM-LL-V
100-SM-LL-L
100-SM-LL-I
400-M5-SN-I
200-M5-SN-I
850nm SWL
100-M5-SN-I
100-M5-SL-I
400-M6-SN-I
200-M6-SN-I
850nm SWL
100-M6-SN-I
100-M6-SL-I
. 3

2 - 2
2 - >50
2 - 2
2 - >50
2 - 10
2 - 2
0.5 - 150
0.5 - 300
0.5 - 500
2 - 500
0.5 - 70
0.5 - 150
0.5 - 300
2 - 175

1300nm LWL
1550nm LWL
1300nm LWL
1550nm LWL
1300nm LWL
1300nm LWL


.

. .

(. ).

FC- ,
. SAN.

(.) ,
.

:
z
(guaranteed bandwidth networks)

.
z
(statistical bandwidth networks)

.
.
. ,

. , .
, ,
. SAN .

, .
(oversubscription).
.

SAN Brocade

9 71

, ,
:
z

.
. , HBA 4Gbps,
2Gbps, 2:1.
2:1 1:4.
z
SAN .
,
ISL . ,
, 32- 4Gbps, 2
ISL. 16 4Gbps 8 2Gbps .
10:1 ( : (16*2*4+8*2*2):(2*2*4)).
SAN 6:1 - 12:1.
z
.
(FC IP)
. , FC4-48
24 FC- 4Gbps Back-end
32Gbps, 3:1 (24*4:32).
, , HBA .
ISL
.
Fan-out Fan-in
, , Fan-out
Fan-in:
z

. Fan-out HBA
.
Fan-out ,
. , EMC Clariion CX200 Fan-out
15:1, Clariion CX3-80 256:1, EMC Symmetrix DMX/DMX-2 64:1 ( Enginuity 5671), DMX-3
128:1.
z
:
. Fan-in
HBA. Fan-in
HBA. , 4:1.
Fan-out Fan-in
. Fan-out -
. - ISL,
.
, .

.
, .

SAN Brocade

10 71


( MB/s)
(. ISL)
. , , 50MB/s 2Gbps, ~25%.

. 80%.


.

. . . .
Brocade :
z
. ,
(. ),
.
.
z
.
. HBA
.
z
ASIC. ASIC
, (.
CCMA) CP.

.
, .
,
.
,
. SAN
, , SAN.

:
z
- (IOPS). IOPS
, (random) -
. OLTP, web-, .
RAID- (, hardware XOR engine)
( ).
z
(MB/s). MB/s ,
(sequential) - ,
. , , HPC (High
Performance Computing). RAID-
Front-end Back-end
, .

SAN Brocade

11 71

HBA
, HBA
. PCI.
HBA PCI 32bit/33MHz,
132MB/s (. . 4). HBA 64bit/66MHz
PCI PCI-X . ,
, 33MHz,
.
HBA PCI-X PCI-express.
. ,
HBA 4Gbps .

32 bit Sbus
64 bit Sbus
32 bit PCI
32 bit PCI
64 bit PCI
64 bit PCI-X
64 bit PCI-X
PCI-Express x1
PCI-Express x2
PCI-Express x4
PCI-Express x8
PCI-Express x16
InfiniBand


25MHz
25MHz
33MHz
66MHz
66MHz
100MHz
133MHz
2.5GHz
5 GHz
10 GHz
20 GHz
40 GHz
--. 4


100 MB/s
200 MB/s
132MB/s
264 MB/s
528MB/s
800MB/s
1064MB/s
400MB/s
800MB/s
1.6GB/s
3.2GB/s
6.4GB/s
2500, 10000 30000 MB/s

- HBA
~100000 IOPS. , (
, -),
.


. IOPS . (
) . 5.
, Fibre
Channel.

, .
.
. ,

.
. SAN .

.

SAN Brocade

12 71

GB

LTO-4
LTO-3

Ultrium 1840
Ultrium 960
Ultrium 230
Ultrium 448
Ultrium 460
Ultrium 215
DLT VS80
DLT VS160
DLT-V4
SDLT 220
SDLT 320
SDLT 600
DLT-S4
AIT-1
AIT-2
AIT-3
AIT-3Ex
AIT-4
AIT-E TURBO
AIT-1 TURBO
AIT-2 TURBO
S-AIT-1

800 /1600
400 / 800
100 / 200
200 / 400
200 / 400
100 / 200
40 / 80
80 / 160
160 / 320
110 / 220
160 / 320
300 / 600
800 /1600
25 / 50
50 / 100
100 / 200
150 / 200 / 520
20 / 52
40 / 104
80 / 208
500 /1300

LTO-2
LTO-1
DLT

SDLT
DLT-S4

AIT-4

MB/s
120 / 240
80 / 160
15 / 30
Sun
StorageTek
24 / 48
T1000
30 / 60
7,5 / 15
3/6
8 / 16
3 / 72
SLR
11 / 22
16 / 32
36 / 72
60 /120
3/6
6 / 12
IBM 3592
12 / 24
18 / 24 / 62,4
6 / 15,6
6 / 12
VXA
12 / 30 / 78
. 5

T9840A
T9840B
T9840C
T9940A
T9940B
T10000
SLR-7
SLR-75
SLR-100
SLR-140
3590 (128)
(256)
(384)
3592
TS1120
3480
3490
3490E
VXA-1
VXA-2
VXA-3

GB

MB/s

20 / 80

10 / 35

20 / 80
40 / 160
60 / 240
200 / 400
500 / 1000
20 / 40
38 / 75
50 / 100
70 / 140
10 / 30
20 / 60
30 / 90
300 / 900
500 / 1500
200 / 600
400 / 800
800 / 2400
33 / 66
80 / 160
160 / 320

19 / 70
30 / 70
10 / 35
30 / 70
120 / 35
3/6
4/8
5 / 10
6 / 12
9 / 27
14 / 42
14 / 42
4 0/ 120
104 / 210
3 / 4,5
3/6
3 / 6,5
3/6
6 / 12
12 / 24

. 5 ,
2Gbps .
LTO-4 IBM TS1120 4Gbps.

FCIP (. FCIP)
.
TCP/IP, FCIP
. .
, ,
FCIP
FCIP-TA (. Fast Write Tape Pipelining).

.
.

SAN .

. ,
.
.

SAN Brocade

13 71


.
,
.
,
.
Brocade FOS (Fabric Operating
Software) Secure Fabric OS, . ,
FOS 5.2,
.

. 1


(. . 1):
-
() SAN.
: .
:
z
.
z
VLAN
, , ssh.
- SAN .
: .
:
z
VLAN .
z
CLI- ssh
sectelnet.
z
SNMP v3.
z
Web Tools SSL.
z
( ).
z
(. ).
- .
: .
:
z
FCAP (Fibre Channel
Authentication Protocol) DHCHAP (Diffie-Hellman Challenge Handshake Authentication
Protocol).
.

SAN Brocade

14 71

z
z

,
.
ACL ( FOS 5.2.0).

- .
: .
:
z
DHCHAP (DiffieHellman Challenge Handshake Authentication Protocol).
z
RADIUS TACACS+.
z
,
.
z
ACL ( FOS 5.2.0).
z
(Persistent Binding).
- .
: .
:
z

DHCHAP (Diffie-Hellman Challenge Handshake Authentication Protocol).
z
RADIUS TACACS+.
z
,
.
z
ACL ( FOS 5.2.0).
z
LUN- (LUN-Masking).
- .
: .
:
z
(Zoning) (. ).
IP-SAN IP- .
: .
:
z
IPsec (IP Security Protocol).
, EMC
, ,
Brocade ,
. EMC Secure FOS.
IP-
IPsec. IPsec Fast Write, Tape Pipelining,
ICMP (ICMP redirect) (Jumbo frames).

SAN Brocade

15 71

Brocade
,
, Brocade (. . 2).
Brocade McData ,
. ,
.

. 2

,
.
.
. ,
.
200E
1U
SMB . 16 4Gbps .
8. Ports on Demand (.
) 12 16 .
(Switch-as-a-FRU).

.
4100
32- 4Gbps
Brocade. (EOL 12/2007)
5000.
1U FRU .
.

SAN Brocade

16 71

4900
32
4Gbps . POD
48 64 .
- 2U FRU
5000
32- 4Gbps .
(4100 M4700)
(), . 5.3
McData M-EOS.
16 . POD (. )
8 16 .
. - 1U.
48000
48000 Brocade.


( 2 8).
FC4-16, FC4-32 FC4-48
16, 32 48 4Gbps . ,
, 8 FC4-48,
382 . FC4-16IP (. iSCSI ), FR4 18i (. FC
) FA4-18 (. ).
5 6 CP (Control Processors).
. CP
Active/Standby.
14U 4 , .
FC10-6
FC10-6. 6 10GFC
(12,75Gbps) (. FC),
ISL (E-). Ficon.
SFP SWL LWL, ELWL,
40 80 , XFP.
FC10-6 (. ISL), , ,
ISL DPS (.
).

SAN Brocade

17 71

iSCSI
, IP SAN
FC SAN iSCSI (. iSCSI)
iSCSI .
BiG (Brocade iSCSI Gateway) 2 2Gbps 2 1GE 100 iSCSI
.
FC4-16IP 8 4Gbps 8 1GE . 1GE
64 iSCSI .
48000 4 .
iSCSI 4, FC- 2.
FC

,
.
,
.
.
IP , FC Brocade
Fibre Channel FCIP (.
FCIP).
7500 16 4Gbps 2 1GE. .
FR4-18i ,
48000. 4 2 .


, SAN.

SAS (Storage Application Services)
.
AP4720 16 2Gbps FC 2 1GE .
SAS,
iSCSI FC . 8 iSCSI 1 FCIP .
(EOL 12/2007)
7600 FA4-18. 16 4Gbps 2 1GE .

SAN Brocade

18 71

EMC Connectrix B M-series


EMC brocade Connectrix B-series
M-series ( McData). EMC P/N
Brocade, (General Availability, GA), (End Of Life, EOL)
(End Of Service Life, EOSL).

-
M- . 6.
M4400

M4700

M6140

EMC
McData
-

Connectrix DS-4400M
Sphereon 4400
1U
8, 12, 16

Connectrix DS-4700M
Sphereon 4700
1U
16, 24, 32

Connectrix ED-140M
Interpid 6140
12U
256

EOS, EFCM Basic

EOS, EFCM Basic

EOSc

Mi10K

z
z

z
z
z

Open Trunking
Performance
Monitoring
Event Management
Security Center
Element Manager

z
z
z
z
z
z

Open Trunking
Performance
Monitoring
Event Management
Security Center
FICON Bundle
Element Manager
. 6

z
z
z
z
z

Open Trunking
Performance
Monitoring
Event Management
Security Center
FICON Bundle

Connectrix ED-10000M
Interpid 10000
14U
140
EOSn, Open VSANs,
SANtegrity Binding,
Element Manager
z
Open Trunking
z
Performance
Monitoring
z
Event Management
z
Security Center
z
FICON Bundle

M4400 M4700 (EOL 12/2007) 5000.

SAN Brocade

19 71

M6140 Mi10K .
(EMC P/N) . 7.

2Gbps

z
z

4Gbps

z
z
z

M6140
ED-UPM-04 - 4 (SW)
ED-UPM-L10 - 4 (LW)
ED-QPM-04 - 4 (SW)
ED-QPM-L04 - 4 (4 LW)
ED-QPM-L10 - 4 (10 LW)

z
z
z
z
z
z
z

2/10Gbps

10Gbps

z
z

z
z

Mi10K
ED-10KM-320S - 32 (SW)
ED-10KM-320L - 32 (LW)
ED-10KM432SW - 32 (SW)
ED-10KM432L4 -32 (4 LW)
ED-10KM432LX - 32 (10 LW)
ED-10KM-164S - 16 2Gbps 4 10Gbps (SW)
ED-10KM-164L - 16 2Gbps 4 10Gbps (LW)
ED-10KM-242S - 24 2Gbps 2 10Gbps (SW)
ED-10KM-242L- 24 2Gbps 2 10Gbps (LW)

ED-XPM-01 - 1 (SW)
ED-XPM-L10 - 1 (10 LW)
. 7

1. Enterprise Bundle , 4 : Fabric


Watch, ISL Trunking, Performance Monitor Extended Fabric.
2. ISL Trunking (.
ISL).
3. Fabric Watch SAN (email,
SNMP) - . :
z
, ,
z
,
z
,
4. Performance Monitor
:
z

z
,
AL_PA (Arbitrated Loop Physical Address) LUN
z
IP FCP (SCSI)
z
ISL
z
Brocade ISL Trunking
z
CRC (Cyclic Redundancy Check)
5. Extended Fabric
10 (. ).
6. FICON CUP FICON CAP .
7. Ports-on-Demand (POD) - 4, 8 16
. ,
POD, .
8. FC Router
. Exchange-Based Trunking.
9. FCIP Tunneling Fibre Channel IP-. 7500
IPsec .
10. Multi Protocol Router Bundle (MPR) FC Routing FCIP
Tunneling.
11. Storage Application Services (SAS) ,
, EMC Invista.
12. IPsec Encryption Services IPsec. C PB-48K18I PB-48K-18IU .
.

SAN Brocade

20 71

EMC P/N

DS-220B

DS-4100B

DS-5000B

DS-4900B

AP7420B

MP-7500B

ED-48000B

SW-48000B-ENT

Enterprise Bundle

DS4100B-ENT

DS5000B-ENT

DS4900B-ENT

ISL Trunking

DS200B-TR

DS4100B-TR

DS5000B-TR

DS4900B-TR

MP7500B-TR

SW-48000B-TR

Fabric Watch

DS200B-FW

DS4100B-FW

DS5000B-FW

DS4900B-FW

MP7500B-FW

SW-48000B-FW

Performance Monitor

DS200B-PM

DS4100B-PM

DS4100B-PM

DS4900B-PM

SW-48000B-PM

Extended Fabric

DS-220B-XF

DS4100B-XF

DS5000B-XF

DS4900B-XF

FICON CUP

MP7500B-PM

POD Upgrade

DS200B-4U-00

DS4100B-CU
DS4100B-8U-00

DS5000B-8U

SW-48000B-XF

SW-48000B-CU

DS4900B-16U

FC Router

AP7420B-RTR

FCIP Tunneling

AP7420B-IP

MP7500B-FCIP

SW-48000B-IP

10

MPR Bundle

AP7420B-MPR

11

SAS

AP7420B-SAS

12

IPsec Encryption

SW-48K-IPS

. 8

Brocade

SAN . ,
,
.
.
CCMA
, .
(shared memory), (bus), (cell switching)
(crossbar).
, Ethernet , .
,
. Brocade
CCMA (Channeled Central Memory Architecture).
ASIC (. ASIC).
,
, (SoC,
Switch on Chip).
ASIC,
.
,
ASIC, , ,
.
Brocade ASIC 1995 .
:
z
Stitch Flannel - ASIC 1997 .
z
Loom ASIC (1999 .)
z
Bloom Bloom-II ASIC (2001 .)
z
Condor Goldeneye ASIC (2005 .)

SAN Brocade

21 71

ASIC
Application-Specific Integrated Circuits (ASIC) - ,
) , ,
- . ASIC
32- , RAM, ROM,
EEPROM Flash .
ASIC , , .
.
,

.
Condor 32 4Gbps ASIC,
256Gbps. , , Condor
(. )
(. ). (.
ISL) 8 ISL
. ,
(. ).
ASIC
(Control Processor, CP). Condor
, CP.
BIST (Build-In Self-Test)
. (parity) RAM (Random Access Memory)
CAM (Content-Addressable Memory) . ASIC Brocade
LED (Light-Emitting Diode) .
Condor 4100, 4900, 5000 48000.
Goldeneye , , Condor. 24,
Trunk 4. . Condor
4012 ( HP BladeSystem P-,
) 200E.
FC10-6 10GFC ASIC Egret.

, , ,
16- ASIC, 18
.
(SRAM)
(time-slicing). - (rx) , - (tx)
. SRAM,
(rx tx)
. (hi-z).

SAN Brocade

22 71

(stripe) .
stripe (buffer line). 32 buffer line (frame
buffer). 2304 . ,
(SOF EOF).
ASIC . ,
128 .
-, .
, (rx1) (8b/10b)
. ,
. , ,
SRAM0, , ( t+3
rx1 SRAM1).

,
(tx0). ,
VC (. ).
, , ,
(message
crossbar).

SAN Brocade

23 71

-, (tx0)
(t+20)
(SRAM1). -
, SFP.
, -
.
-,
.
ASIC
(Central Memory).
.
ASIC .
,
ASIC. ASIC, ,
Front-end (FE). ASIC Back-end (BE).
, ASIC ISL SoC.
, BE E-.
, ,
. . C-.
ASIC, , .
, , ASIC,
(). 48000
.
Fibre Channel . ,
ASIC,
, (. . 3). , ,
ASIC- ,
ASIC-. cut-through.
(latency) .
ASIC Condor 0,8s, ASIC
Goldeneye - 1,2s, 48000 2,1s (2Gbps), 3,6s
(4Gbps) 7,4s (10Gbps).
, , ASIC,
.
.

SAN Brocade

24 71

. 3

Cisco MDS 9513


,
Brocade Cisco. SAN-
Brocade 48000 Cisco MDS
9513. ,
,
Fibre Channel, FCIP iSCSI.
Brocade
ASIC. ,
SAN- Core-Edge. Cut-Through.
MDS 9513 ASIC.
. ASIC.
.
.
.
, , Fibre
Channel, . ,
, .
Store &Forward.

SAN Brocade

25 71

Brocade Cisco .
.

ISL, Brocade
, ,
, (Virtual Channels,
VC) (. . 4).
Brocade Fibre Channel.

. 4

VC . 0,
3. ,
.
F VC0 (. . 9).
3 VC.
, VC,
. ASIC (C-)
17 VC.

SAN Brocade

26 71


. ,
, .

F
FL
E
FC_SW2

-
VC
1
1

VC

VC7
VC7

2 3 ( 3)
2 3 ( 3)

0xda
0xda

VC0

0xef

VC0

0xef

VC1

0xe8

VC2
VC3
VC4
VC5
VC6
VC7

2 3 ( 2)
2 3 ( 2)
2 3 ( 2)
2 3 ( 2)
2 3 ( 3)
2 3 ( 3)

0xe4
0xe2
0xe1
0xe0
0xdc
0xda

17

AL_PA

N-
NL-
E-
(InterOp)
( F)
(F_BSY F_RJT,
2)
( 2 3)
( 2 3)
( 2 3)
( 2 3)
Multicast ( 3)
Multicast broadcast ( 3)

ASIC Condor .

. 9

VC , .
, Fibre Channel,
6
- - IDLE. VC,
ARB(),
FC_AL. 2.
VC, AL_PA. AL_PA
. 9. , AL_PA,
VC.
fabric.ops.mode.longDistance
,
WDM, .
ARB(vc).
ARB(vc) fabric.ops.mode.longDistance 1 ( Extended Fabrics
Mode WebTools).
4Gbps .
Brocade
(. ).
,
ISL,
. , VC
. ,
, ,
.
,
. VC ,
(. ).
.

SAN Brocade

27 71

ISL
.
,
VC . ,
(anti-starvation algorithm). VC ,
. ,
VC. ,
.

.
,
. , -
.
, ,
.
, ,
, .
, ,
(flow control),
. . . ,
, .
, ( , BB-credits)
, . , .

(40bit) R_RDY (Receiver Ready),
(. . 5).

. 5

,
. ,
.

SAN Brocade

28 71

, . ,
.
1 , 5 . ,
R_RDY .
, .
, , .
, SAN,
, ,
- R_RDY.
,
,
:
z
1Gbps 1 2
z
2Gbps 1
z
4Gbps 2
z
10Gbps 6
(. . 6).

. 6

,
90-95% 2KB.
5-10% - , 1KB. ,
- ,
, .
,
,
.
Brocade ,
. VC
,
(. ).
. , R_RDY ,
VC, VC_RDY.

SAN Brocade

29 71


(. ),

. , ,
, ASIC, (. . 10).

200E
4100
4900
5000
48000
M4400
M4700
M6140
Mi10K

ASIC
Goldeneye
Condor
Condor
Condor
Condor/Egret
Pegasus/Teton
Pegasus/Teton
Viper / Fuji-Shasta
Sanera
. 10

. . 11
. . 11
. . 11
. . 11
. . 11
120
120
60
1373

LWL ELWL SFP


LWL ELWL SFP,
, .

Brocade CWDM DWDM.


,
.
Brocade : VC_RDY
R_RDY.
VC_RDY, ASIC
(Long distance mode), (.
. 11):
z
L0 .
. .
z
LE 10.
z
LM 25.
z
L1 50.
z
L2 100.
z
LD .
z
LS .
, 4Gbps
.
.

SAN Brocade

30 71

1Gbps

2Gbps

L0
LE
L0.5
L1
L2
LD
LS

3
11
18
31
56

3
16
31
56
106

L0
LE
L0.5
L1
L2

5
11
18
31
56

5
16
31
56
106

LD
LS

L0
LS

4Gbps

12,75Gbps

ASIC Golgeneye
3

31

56

106

ASIC Condor
5

26

56

106

206

Extended Fabric?

Fabric OS

3.x, 4.x
5.1.0
5.1.0
5.1.0
5.1.0
5.1.0

3.x, 4.x
3.1.0, 4.1.0, 4.x, 5.x

3.1.0, 4.1.0, 4.x, 5.x



05.01.2000

5.3.0
5.3.0

ASIC Egret
720

. 11

ASIC , ,
, (. . 12).

Gbps

10.

25.

1
2
3

16
16
8

12
6
3

1
2
3

32
32
32

32
32
15

1
2
3

64
64
64

64
64
48

1
2
3

16
16
16

16
16
16

1
2
3
10

16
16
16
-

16
16
12
-

50. 100.
200E
6
3
3
1
1
0
4100
32
15
15
7
7
3
4900
64
44
48
20
20
8
7500
16
16
16
8
8
4
FC4-16
16
12
12
5
5
2
-

250.

500.

10.

25.

1
-

32
32
32

32
32
26

6
3
-

48
48
32

48
48
22

16
8
-

8
-

16
16
16

16
16
16

6
2
-

2
-

8
8
8

8
8
8

4
2
0
-

0
0
0
. 12

SAN Brocade

50. 100.
FC4-32
32
26
26
12
12
6
FC4-48
48
22
22
10
10
4
FR4-18i
16
16
16
8
8
4
FC4-16IP
8
8
8
6
6
3
FC10-6
6
6

250.

500.

10
4
0

4
0
0

8
4
0

4
0
0

16
2
-

2
-

5
2
0

2
0
0

31 71

R_RDY,
, . ASIC Condor
R_RDY 1024 ( 255 ),
Goldeneye 288 (128 ).
Long distance R_RDY.
Fast Write and Tape Pipelining
SCSI .
1.
SCSI-initiator (FCP_CMD)
2.
SCSI-target (FCP_XFR_RDY)
3.
SCSI-initiator (FCP_DATA)
4.
SCSI-target (FCP_RSP)

.
2 FCP_XFR_RDY ,
SCSI-target.
(spoofing) Fast Write.

,
,
(sequence) .
FCP_XFR_RDY

, - . FCP_XFR_RDY
spoofing Tape Pipelining.

SAN Brocade

32 71

Fast Write Tape Pipelining


FCIP.
.
Acceleration :
z
Acceleration , SCSI initiator SCSI target
(. )
z
Acceleration
Cisco Write Tape
Acceleration.

Brocade.
.
200E
, 200E SAN
. .
, (. . 7).
ASIC Goldeneye. 24
FE 16.
SMB .
ASIC ,
.

SAN Brocade

33 71

Control Processor, :
z
IBM PowerPC 440GP
z
128MB DDR
z
4MB flash,
z
256MB CompactFlash
z
RS-232 (DB-9) Ethernet 10/100Mbps (RJ-45)
z
JTAG

. 7

ASIC PCI (33MHz, 32bit).


I2C ( ) .
. ,
, ,
. .
. 8. 0-7
. POD 8-11 12-15.

. 8

5000
5000 ASIC Condor.
.
FE 32 . , 200E:
z
CPU IBM PowerPC 440GP 333MHz
z
256MB DDR
z
4MB boot flash
z
1GB CompactFlash
z
RS-232 (RJ-45) Ethernet 10/100Mbps (RJ-45)
z
JTAG

SAN Brocade

34 71

. 9

. 10. 0-15.
POD 8-11 12-15.

. 10

4900
4900 Core-Edge (. . 11). 4 ASIC Condor
(Edge) SoC. 16 FE.
8 BE (Core) SoC. ,
, SAN Core-Edge CCMA
. , BE ASIC
ISL.
4 . 0-15, 16-31, 32-47 48-63.
, 16 FE ASIC 16 BE .
, - 1:1.

. 11

:
z
CPU IBM PowerPC 440GP 400MHz
z
256MB DDR
z
4MB boot flash,
z
512MB CompactFlash
z
RS-232 (RJ-45) Ethernet 10/100Mbps (RJ-45)
z
JTAG
.

SAN Brocade

35 71

. 12. 0-15.
POD 8-11 12-15.

. 12

48000
48000 .

. Fibre Channel,
iSCSI FCIP. 10
(Control Processor, CP). , CP
5 6.
backplane.
Fabric OS (FOS). ,
MontaVista, Linux.
Core-Edge (. . 13). ASIC Condor,
Edge . 32- Core
CP. 128 BE . 8
Core ASIC 4Gbps . ,
.

. 13
.

SAN Brocade

36 71

Control Processor 48000 CP4.


CP (. . 14):
z
CPU IBM PowerPC 440GP 800MHz
z
512MB DDR
z
4MB flash,
z
1GB CompactFlash
z
RS-232 (RJ-45) Ethernet 10/100Mbps (RJ-45)

. 14

CP .
(Mean Time Between Replacement, MTBR)
38 . ,
Control Processor.
CP active-passive.
CP. ,
active-active, ASIC Control Processor.
CP .
, CP
. .
CP .
Core ASIC CP. ,
Core ASIC 50%.
. ,
CP
, 16Gbps ( full duplex, ).

. .
.

SAN Brocade

37 71

IOD
CP, Core ASIC .
ASIC
. ,
, (Out-of-Order delivery, OOD).
IOD (InOrder Delivery), .
, .
, , OOD,
: HBA, FC SCSI ,
.
, IOD

FC4-16
FC4-16 ( Proton) ASIC Condor
(. . 15).

. 15

16 FE . 4 BE
Core ASIC (. . 16). ASIC ,
. 1:1 (16:16).
FC4-16, FC4-32 FC4-48 ,
.

SAN Brocade

38 71

. 16

200E, 500 4900


, 48000 .
:
z
. , 4 7 - 4/7.
z
Area ID. Area ID . ,
, Area ID .
FCID
FCID Fibre Channel.
24- 8- :
z
Domain ID - , ,
z
Area ID - ,
z
Node ID - FC_AL, P-2-P 0x00.

.
,
, , .
Area ID FC4-16 . 17.

SAN Brocade

39 71

. 17
z

. , ,
ACL DCC Domain ID,
Area ID.
384 . , Area ID 8 , . .
255. (Port Index, PI),
, ,
(. . 18).

. 18

FC4-32
32- FC4-32 ( Electron) ASIC (. . 19).

SAN Brocade

40 71

. 19

16 ASIC FE . ,
Core ASIC 16 (128Gbps full duplex).
ASIC FC4-32 8 BE (. . 20). 8
. .
2:1 (16:8).

. 20

Area ID FC4-32 . 21.

SAN Brocade

41 71

. 21

. 22.
( ASIC).

. 22

FC4-48
48- FC4-48 ( Positron)
FC4-32. ASIC Condor.

SAN Brocade

42 71

. 23

, 8 ASIC
FE (. . 24). , 24 FE 8 BE .
. 3:1 (24:8).

. 24

, 8-
Area ID. 255. FC4-48
: FC_AL.
, FCID
8 Node ID. , 8 ,
0-31 Node ID 0x00, 32-47
0x80 (. . 25).

SAN Brocade

43 71

. 25

Shared Area ID. , 8 32


(share) Area ID. Node ID.
Shared Area Node ID . 26.

. 26

FC4-48 . 27.
.

. 27

SAN Brocade

44 71

WWN Brocade
Brocade OUI 00:05:1e 00:60:69. WWN
.

32-47 Shared Area WWPN.

FC10-6
6 12,75Gbps . E.
FC10-6 .
ASIC Egret (. . 28).

. 28

3 FE ASIC Egret 8 BE (. . 29).


( 0-2 3-5). 1,18:1 (~37,5:32).
.

SAN Brocade

45 71

. 29

. 30.

. 30

FR4-18i
FR4-18i ( Marathon) 16 FC 2 1GE .
FC (.
). Fibre Channel , . .
(fabric-to-fabric), FC
IP FCIP (. FCIP).
FCIP (. )
1GE 8 FCIP.
(Virtual E_Port,
VE_Port). GE0 VE- 16 23, GE1 24 31.
,
(traffic shaping). FC
FCIP (VEX_Port).
Blade Processor
, FC4-16/32/48 .
CP. Control Processors
, ,
, ,
Blade Processor (BP).
Blade FOS (BFOS), ,
Application Processor (AP). FOS
. CP (auto-leveling) AP Blade
FOS. Ethernet TFTP.
.

SAN Brocade

46 71

FCIP
Fast Write.
FR-18i :
z
CPU IBM PowerPC 440GP 800MHz
z
512MB DDR
z
4MB boot flash
z
512MB CompactFlash
z
JTAG
FCIP :
z
Broadcom BCM1250
z
Hifn9630
z
FPGA Xilinx ( Flipper)
Xilinx FPGA ( Pinball).
FC . ASIC Condor
(. . 31).

. 31

ASIC Condor 8 FE . 8 BE
Core ASIC (. . 32).
( 0-7 8-15). 1:1 (8:8).
Flipper ASIC Condor FC. Condor
Pinball , 8 4Gbps
.

SAN Brocade

47 71

. 32

(. . 33). ,
(disabled) .

. 33

AP7420 FC4-18i 7500 .


7500
7500 , , FR4-18i
. (. . 34).
, , ,
Ethernet .

. 34
.

SAN Brocade

48 71

, FR4-18i ASIC Condor


Core ASIC CP. 7500 Core ASIC .
Condor 8
(. . 35).
CP256 .
(FOS)
(BFOS).

. 35

. 36.

. 36

( 0-7 8-15).
1:1 (8:8).
FC4-16IP
FC4-16IP ( Ironman) 8 4Gbps FC 2 1GE .
iSCSI . (.
iSCSI). 64 iSCSI 1GE .
FICON .
FC-16IP :
z
CPU IBM PowerPC 440GP 800MHz
z
512MB DDR
z
4MB boot flash
z
512MB CompactFlash
z
JTAG
iSCSI (. . 37):
z
Broadcom BCM1125H .
256MB DDR SDRAM.
z
FPGA Xilinx ( iFlipper) iSCSI

SAN Brocade

49 71

. 37

Condor (. . 38). iFlipper


FC-. .

. 38

(. . 39).

. 39

SAN Brocade

50 71

Brocade
SAN .

.
.
Brocade .
.
Domain ID
Domain ID 1-239.
:
z
(insistent) Domain ID.
, , ,
, FCID . HP-UX
AIX. , Domain
ID, .
z
Domain Manager, ,
Domain ID , .
(preferred) , .
- , Domain ID.
Domain ID .
.

(Principal Switch). , Domain ID
. , Principal Switch
Principal Selection Mode. ,
, WWN. ,
WWN ASCII . , , WWN
10:00:00:0d:ec:0a:00:00, 10:00:00:0d:ec:01:00:00.
, Principal.
.

Principal Selection Mode. Principal
, , . .
hops /
.
( F) ,
Principal ISL (. 40). ISL . ,
Principal Switch, Downstream, Principal Switch Upstream.

SAN Brocade

51 71

. 40

.
RCF (Re -Configure Fabric) BF (Build Fabric). RCF
. BF - ,
,
.
Principal ISL .
BF Principal Switch.
15 (3x F_S_TOV, F_S_TOV 5.). ,
ISL, ,
.


. Brocade :
1. (Intra-fabric routing) .
2. (Inter-fabric routing) .
FC-SW-2 FSPF (Fabric Shortest Path First).
, (Dijkstra),
(cost). .
, ISL,
(hop). ISL, , .
, ISL ,
1Gbps 1000. , , , ,
, 2Gbps, - 4Gbps 750 (1000/2+1000/4).
, FOS ( 4.4), 4Gbps
500 (. . 41). , 2Gbps 4Gbps ISL
.

SAN Brocade

52 71

. 41

FSPF .
20- .
ISL 64- HELLO
(HLO). , . ,

ISL .
.
- FSPF .
, FSPF
.
.
,
,
:
z
Port-based Domain ID
(. . 42).
,
.

. 42

SAN Brocade

53 71


- ,
. (-)
,
. ,
. ,
. -
.

.
(round-robin) .
,
(. . 43).

. 43

DLS (Dynamic Load Sharing)


,
. ,
(in-order delivery).
z

Device-based - FCID
(Source ID, S_ID) FCID (Destination ID,
D_ID) (. . 44). 4Gbps FOS 5.1.

. 44
.

SAN Brocade

54 71

Exchange-based FCID
(Source ID, S_ID), FCID (Destination ID,
D_ID) FCP (Originator eXchange ID, OXID Receiver eXchange ID, RXID)
(. . 45). DLS .
. 4Gbps Exchanged-based

( FC10-6). DPS (Dynamic Path Selection).
DLS
, (in-order delivery), .
. in-order,
IOD (. IOD).

. 45


.
-
round-robin. , . -
ISL , - .
DPS, host-based, . FCP
OLTP
, .
,
(load-balancing). SAN
(load-sharing).
Brocade
, Brocade, . . .
,
.
,
, PowerPath, HDLM, VxDMP MPxIO.

SAN Brocade

55 71

ISL
,
(load sharing). Brocade
,

ISL. ISL
(Trunking).
,
, . SAN,
.
ISL
.
,
z
E EX .
z
ASIC ( ).
z
2Gbps, 4Gbps ASIC Goldeneye,
(-) 4.
Condor 8 . FC10-6 ASIC Egret
.
z
- .
(auto-negotiation), .
, 4Gbps
-, .
z
Long Distance Mode.
z
(InterOp Mode).
z
, , ISL
Trunking.
z
, ,
, (disable-enable).
z
,
(in-order), , ,
, ISL. (deskew)
. .
deskew . -
4Gbps deskew=200, 2000
400 .
deskew=15, 150
30 .
, ISL
.
2Gbps
(. . 46). (in-order)
. (Master Link).
Master Link BF (.
Domain ID), Principal 15 .
. ,
.

SAN Brocade

56 71

.
(FCP sequence).

. 46

4Gbps ,
. . (. . 47).
. .
Pseudo-master Link.
, ,
. Pseudo-master Link
, Pseudo-master .

. 47

FSPF ,
Master Pseudo-master Link.

SAN Brocade

57 71

, , ISL, .
. 48 Host-based load sharing.

. 48

. 49 Exchange-based load sharing.

. 49


. (. )
(. ).
:
z
(Core/Edge, Mesh, Cascade, Ring)
.
z
- Core
.
z
, ,
.

SAN Brocade

58 71



.
FCRP (Fibre Channel Routing Protocol).
, , Edge. ,
Backbone. backbone
FR4-18i. FR4-18i,
, 48000, backbone .
edge (E2E routing), backbone
edge (B2E routing).
, edge
, EX-. E-, EX- (.
ISL).
(edge backbone) Fabric
ID (FID) (. . 50). , backbone
FID. backbone ,
edge FID backbone FID. edge backbone
IFL (InterFabric Link).

. 50

, .
, , .
backbone edge . , EX-,
, Front (Front Domain, FD).
Translate (Xlate Domain, XD),
-. FCID - (Proxy PID).
XD IFL (. . 51).
.
B2E , backbone FD .
XD.

SAN Brocade

59 71

. 51

,
LSAN (Logical SAN). ,
(. ). LSAN ,
LSAN_ ( ).
.
FCID ,
LSAN WWN aliases.
WWN Front
WWN - Front WWN
EX-.

FCIP
FCIP (FC over IP) Fibre Channel IP-.
, FC- ()
IP- (. . 52) ,
(. . 53).
.

SAN Brocade

60 71

. 52


. , FCIP .
, FCIP
IP SAN.

. 53

FCIP GE - (Blade
FOS, Broadcom, Hifn Flipper), FCIP_DE (FCIP Data Engine).
GE 8 FCIP .
(Virtual E-port, VE) (. . 54).

. 54

SAN Brocade

61 71

FCIP
FCIP_LEP (FCIP link end-point).
FCIP_LEP:
z
IP-
z
TCP. TCP- 3225
F, 3226 .
z
WWN
z
(QoS).
, FCIP .
VE- .
.
IP
, Principal (.
Domain ID), . .
.
.
,
SAN (. ).
VE- EX-
(Virtual EX_Port, VEX) (. . 55).

. 55

. ,
, .
FCIP
IP ,
. .
, , MTU (Maximum Transfer Unit) 2250
. Ethernet Jumbo Frames. TCP
MWS (Maximum Windows Size).
.
FCIP
. ,
FCIP
.

SAN Brocade

62 71

iSCSI
Fibre Channel iSCSI (SCSI over IP).
, , FCIP, SCSI FC-, SCSI IP- (. . 56).

. 56

FC
, ,
. SAN
iSCSI, IP-
1Gbps. , FCIP,
TCP/IP- .
iSCSI
(Network Interface Card, NIC) iSCSI
. NIC
. ,

,
.
CPU ( 1%)
TCP (TCP Offload Engine NIC, TNIC),
.
iSCSI HBA (iSCSI accelerator). Ethernet
iSCSI HBA , TCP/IP
iSCSI. iSCSI HBA FC HBA.
, SAN iSCSI
. , ,
iSCSI.
iSCSI HBA TOI
iSCSI FC.
SAN Enterprise, , iSCSI
.

. 57

Brocade ,
FC SAN, , IP (.. 57).
Fibre Channel iSCSI. , IP FC
iSCSI targets (VT) FC
initiators (VI) (. . 58).
.

SAN Brocade

63 71

. 58

TCP/IP iSCSI ,
, (portals). IP ,
IP ( 3260). iSCSI
EUI IQN.
TCP/IP Fibre Channel . 59.

. 59

(iSCSI ) iSCSI
VI, FC N- WWN
.
, VT ( )
LUN FC , iSCSI SCSI LUN
iSCSI Qualified Names (IQN) (. . 60).

SAN Brocade

64 71

. 60

iSCSI , iSCSI VI
(. ) LUN-
.
iSCSI Brocade
iFCS (IP Fabric Configuration Server). iFCS ,
. ,
FC-iSCSI. -
iSCSI
. , , IP
LUN.
iSCSI-FC
CHAP, , , .


. :
z
.
z
- RSCN.
RSCN
- RSCN (Registered State Change Notifications)
(, )
. RSCN . RSCN ,
,
.
:
z

z

SAN Brocade

65 71

, ,
WWW- , ,
. .

Brocade
:
z
Domain ID .
Insistent Domain ID (. Domain ID).
z
WWNN WWPN .
FCID .

. 61

. 13.

WWN
. .

WWN (WWN

spoofing).
. WWN
.
.
WWN .
.
.

,
.
.
.
.
. 13


- (FCP-initiator
HBA ) (SNS)
. ,
- .
, SAN,
.

SAN Brocade

66 71

-
.
z

(soft zoning) SNS


. ,
FCID. HBA
(, Emulex)
.
(hard zoning)
.
ASIC.

Brocade .
,
WWN -.
.
-
, hard. soft
.
. FC4-48 16-47
-Domain ID, , , soft.

:
z
(Alias) . Alias
(, HBA
) (,
, ).
Aliases, , , . 64-
WWN 24- FCID . -
. SAN.
z
. Aliases .
WWPN.
.
, Zone_1 qwerty, .
z
ZoneSet. .
ZoneSet.
.
z
ZoneSet. ,
.
.
.
, ZoneSet
SAN.
Alias, Zone ZoneSet .
. ,
. , Zone_Solaris_1 ORACLE_ProductionTop.
,
.
.

SAN Brocade

67 71

, :
z
. , HBA

.
z
ISL .
z
, , ,
( no access).
,
HBA (FCP-initiator).


. ,
, .
,
, .
.
, SAN (hanging devices).
(dead) .

3.2 4.4 Fabric OS (default user
accounts): root, factory, admin user.
(user-defined accounts).
15 .
(Admin User), .
RBAC (Role-Based Access Control). 15
(login) .
FOS 5.2 256,
32. ,
Admin. 4
. (. . 14). root
factory Root Factory.
Brocade.

Admin
BasicSwitchAdmin
FabricAdmin
Operator
SecurityAdmin

FOS

5.2.0+
5.2.0+
5.2.0+
5.3.0+

SwitchAdmin
User
ZoneAdmin

5.0.0+

5.2.0+


AD
SAN
,
,
AD


. 14

SAN Brocade

68 71


(Virtual Fabrics, VF)
SAN.
,
(, , . .) VF.
, VF RBAC.
VF
. FOS 5.2.0.
,
SAN. Cisco VSAN
. ,
(Administrative Domain, AD)
VF :
Ficon
Fibre Channel .
z

.
z
.
z
EMC Invista.
z

SAN ,
.
: AD0 AD255.
AD0 , .
,
. , AD0 (implicit)
.
AD255 .
. AD255
. ,
. AD255 SNS .

(. . 62):
z
(Domain ID: ).
,
. Insistent Domain ID (.
Domain ID).
z
WWPN WWNN .
WWN
z
WWN Domain ID .
.
.
AD. ,
, iSCSI VI (. iSCSI)
AD0.

SAN Brocade

69 71

. 62

, - , AD0.
, .
AD0, (explicit)
.
,
(AD_List), (Home AD)
. Home AD ,
AD_List .

(. ).
AD_List, LSAN.
WWN
WWN
.

SAN Brocade

70 71


,
Brocade,
.
? .
.

.

Solutions Architect
TSG - EMEA Delivery
EMC Russia & CIS
pantyukhin_vasily@emc.com

SAN Brocade

71 71

Оценить