ZS44HardwareOverview

Items:17Steps(IncludingIntroduction)

Duration:51seconds

ZS44HardwareOverview
5seconds

StepText
Overview of Major Components
ZS4-4 is part of the Oracle ZFS Storage Appliance family running on the Oracle X4-4 3U
Server with Solaris OS based Fishworks software providing the appliance interface to the
Solaris OS. The ZS4-4 has four CPUs, built-in four 10Gb/s Front-end GigE ports, redundant
power supplies, up to eleven PCIe HBA Card slots. The ZS4-4 ZFS Storage System uses the
G3 chassis containing a four socket CPU Motherboard.

ZS44HardwareOverview 1
The unit is equipped with a large memory capacity that supports eight Memory Risers, each
supporting twelve DIMMs. This allows for a total system memory maximum capacity of up
to 1.5TB.
The system contains two 2.5 900GB SAS drives with Silver Marlin Brackets used for OS
storage and booting. There are two OS Boot drives per server such that the drives will reside
in two of the six Server right front drive slots: 0 and 1; in a RAID 1 (mirror) configuration.
Up to four optional 1.6TB SAS Flash SSDs can be added to the 4 remaining right front drive
slots 2 through 5 in that order. The SSDs are used for read cache from the Adaptive
Replacement Cache (ARC) resident on main memory (Readzilla) solid state drives (SSDs).
The unit contains nine x8 and two x16 PCIe Gen3 HBA card slots. These HBA Card option
slots will be used for Host attach InfiniBand, Fibre Channel, 10G Ethernet, as well as back-
end attach SAS-2 to our Storage Arrays.
Data storage is provided by the Performance DE2-24P or the Capacity DE2-24C SAS-2
interface JBODs.

Appliance Specific Details

The ZS4-4 incorporates one or two 4 socket Intel Ivy Bridge-EX based controllers and 1 to 36
disk enclosures (DE2-24P & DE2-24C) in homogenous or mixed disk enclosure variants.
Unlike the ZS3-4 predecessor, there is only one controller option, which is a fully populated
4-socket configuration. All 4 sockets are required to meet the memory density and PCIe slot
count requirements of the storage appliance. Memory is configurable in a limited manner in
both the number of slots utilized as well as the size of the DIMM (16GB). The minimum
memory configuration will be 1.5TB based on 96 x16GB DIMMs.
Each controller has 9 x8 external PCIe3 expansion slots in addition to 2 x16 PCIe3 slots.
Unlike the previous generation, there is no dedicated slot for the Fishworks Cluster
Controller (Clustron V2). Instead the Clustron card occupies a general-purpose x8 Gen3 slot.
While this is expensive in the case of the storage appliance usage model, there is no
reduction in overall PCIe slot count versus the previous product as there are now a total of
11 general-purpose slots rather than 10. While this is still one slot less than the desired
minimum, it is not a step backward from the current family offering.
Contrary to previous Oracle ZFS Storage Appliance controllers, ZS4-4 will NOT incorporate a
USB 2.0 thumb-drive internal to the system motherboard USB Slot. This option has been
removed for new and future platforms until inclusion of this device can be solidly justified.
The controller includes 6x 2.5 SAS-2 front accessible disk slots, which will accommodate 2
mirrored system boot disks. The additional 4 slots may be used for Readzilla devices or will
contain filler panels if optional Readzilla SSDs are not installed in any given slot.
Back-end I/O connectivity is provided by the Thebe-2 SAS-2 HBA. Up to 4 total external HBAs
are allowed in the configuration providing up to 16 x4 SAS-2 expansion ports. Disk chains
are to remain at maximum 6 deep as in previous products.
Front-end (client) connectivity options are shared with the rest of the product line and are
described in the configuration section later in this document. At product release, the IO
option cards are a subset of those supported on the existing ZS3-4 with the exception of the
Thebe3 SAS HBA and the CX3 Infiniband adapter. Overall the number of IO options has
been reduced.
Each drive enclosure holds either 24 3.5 or 2.5 SAS-2 disks or 20 3.5 or 2.5 SAS-2 disks
and from 0 to 4 special use SAS-2 SSDs utilized as ZFS write intent log devices. These are
special purpose Write Intensive (WI) devices and are not to be confused as general-purpose
flash pools.

ZS44HardwareOverview 2
As with previous versions of ZFSSA appliances, disks with different capacities or with
different spindle rotation rates may not be mixed within a single storage enclosure.
However different types of enclosures may be used in conjunction in a ZS4-4 stack freely.

ZS44HardwareOverview 3
ControllerOverview
3seconds

StepText
The ZS4-4 controller can be configured as a single controller or two controllers to create a
high-availability cluster configuration. The ZS4-4 controller chassis fits in a standard
equipment rack, and occupies three rack units (3RU) in height. Click on the figure to view
ZS4-4 controller specifications and chassis dimensions.

ZS44HardwareOverview 4
FrontPanel
2seconds

StepText
The ZS4-4 controller front panel components are described in the following figure and
legend.

ZS44HardwareOverview 5
SystemDrives
2seconds

StepText
The ZS4-4 controller has two 2.5-inch 1.2TB or 900GB SAS-2 system boot drives in slots 0 and
1, configured as a mirrored pair. Up to four 1.6TB SAS-2 Read flash SSDs can fill slots 2
through 5, in that order. A filler panel must be installed in empty drive slots. The system
drive LEDs are shown in the following figure.

ZS44HardwareOverview 6
RearPanel
2seconds

StepText
The ZS4-4 rear panel is shown in the following figure. Base configuration PCIe cards are not
depicted in this illustration.

ZS44HardwareOverview 7
EthernetPorts
2seconds

StepText
The ZS4-4 has four RJ-45 10-Gigabit Ethernet (10GbE) network connectors on the rear panel,
labeled NET 0, NET 1, NET 2, and NET 3 (bottom left to top right), as shown in the following
figure. Use these ports to connect the appliance to the network.

ZS44HardwareOverview 8
NetworkManagementPort
2seconds

StepText
The network management connector (NET MGT), shown in the following figure, is an RJ-45
port and provides an alternate terminal interface to the service processor (SP) console.

ZS44HardwareOverview 9
SerialManagementPort
3seconds

StepText
The serial management connector (SER MGT), shown in the following figure, is an RJ-45 port
and provides a terminal connection to the SP console.

ZS44HardwareOverview 10
4x4SAS2HBA
4seconds

StepText
The 4x4 SAS-2 HBA provides connectivity to external DE2-24 and Sun Disk Shelves. The HBA
ports are numbered 3-0, top to bottom, as shown in the following figure.

ZS44HardwareOverview 11
PCIeI/OSlotNumbering
4seconds

StepText
The ZS4-4 base configuration contains the following PCIe cards:
One 8-port SAS-2 internal HBA (slot 2)
Two 4-port (4x4) SAS-2 external HBAs (slot 6 and slot 7)
One cluster interface card (slot 4)

The following figure shows the PCIe I/O slot numbers.

Additional client-facing cards can be installed in the remaining PCIe slots. See the PCIe Slot
Order tab.

ZS44HardwareOverview 12
PCIeSlotOrder
3seconds

StepText
Install optional PCIe cards in the following order:

1. Install additional 4x4 SAS-2 HBAs in slot 9, then slot 3.

2. Install InfiniBand CX3 HCAs into the first available client-option slot starting with slot 11,
then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.
3. Install 16Gb FC HBAs into the first available client-option slot starting with slot 11, then
slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.
4. Install 10Gb Ethernet Optical NICs into the first available client-option slot starting with
slot 11, then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.
5. Install 10Gb Ethernet Copper NICs into the first available client-option slot starting with
slot 11, then slot 8, slot 5, slot 1, slot 10, slot 3, slot 9.

ZS44HardwareOverview 13
PCIeBaseandOptionalConfigurations
4seconds

StepText
The following table describes the PCIe base and optional slot assignments for ZS4-4
standalone and cluster configurations. PCIe slot numbering begins with slot 1.

Click on tab 12 to see the legend for a description of the interconnect types and option
codes shown in the Type column.

ZS44HardwareOverview 14
LegendforInterconnectTypesandOptions
4seconds

StepText
See the legend for a description of the interconnect types and option codes shown in the
Type column.

ZS44HardwareOverview 15
InternalComponents
2seconds

StepText
The ZS4-4 controller chassis contains both customer-replaceable units (CRUs) and field-
replaceable units (FRUs) as shown in the following figure. FRUs must be replaced by trained
Oracle service technicians.

ZS44HardwareOverview 16
CPUandMemory
3seconds

StepText
The ZS4-4 controller has four Intel Xeon E7-8895 v2 15-core 2.8 GHz CPUs and eight memory
riser cards as shown in the following figure. The memory configuration is 16GB DDR3
DIMMs to accommodate 1.5TB (ninety-six 16GB). All ZS4-4 DIMM risers are fully populated
to accomodate this offering.

Each memory riser card contains twelve DIMM slots, four DDR3 channels, and two memory
buffer ASICs. Each each memory buffer has two channels (A and B) and links to three DIMM
slots per channel. Each memory buffer is connected to the processor's built-in memory
controller by an SMI-2 link.

DIMM names in appliance logs and the Maintenance > Hardware view are displayed with
the full name, such as /SYS/MB/P0/D7.

ZS44HardwareOverview 17
CoolingSubsystem
3seconds

StepText
The ZS4-4 internal components are cooled by air that is pulled in through the front of the
controller and exhausted out the back of the controller. Cooling occurs in two areas of the
chassis: the power supply area and the motherboard area.

The following figure shows the cooling zones and the approximate location of the
temperature sensors. The accompanying legend table provides sensor NAC names and
sensor motherboard designations.

ZS44HardwareOverview 18
AttachedStorage
3seconds

StepText
The ZS4-4 single and cluster controller configurations allow up to 36 disk shelves, consisting
of up to six chains of one to six disk shelves. Any combination of disk-only and Write flash
disk shelves can be combined within the chain in any order.

ZS44HardwareOverview 19