Teradata 2580d BYNET Configuration

Nov 4, 2009

RESTRICTED PROGRAM

Teradata 2580 Knowledge Transfer Session

BYNET Agenda

Overview
Switch Configuration Node Configuration

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Overview Hardware

For 2580, BYNET switching is provided by redundant, dual-active Dell PowerConnect 6248 48-Port Gigabit Ethernet Switches.
> The Dell PowerConnect 6248 is a managed switch > It is managed by SMWeb. In the base cabinet, SMWeb shares the same switch with BYNET

BYNET Gigabit Ethernet Switches are installed in positions 1 and 20 in the 2580 base cabinet.
> chassis 1 = BYNET 0 > chassis 20 = BYNET 1

2580 DELL processing node on-board Ethernet interfaces are connected to the BYNET Gigabit Ethernet Switches.
> Port 4 eth3 is connected to BYNET 0. > Port 3 eth2 is connected to BYNET 1.

All connections are copper (1000BASE-T). The following cable lengths have been certified for BYNET: 5, 10, 20, and 30 meters.

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Overview Dell R710 node

1 -Size PCIe X4 slot and 1 -Size PCIe X4 slot on the Center PCI Riser 2 -Size PCIe X8 slots on the Left PCI Riser

Rear View

1 2 3 4

Server Management connections (1GbE) Port 1 = eth0 Port 2 = eth1

BYNET connections (1GbE) Port 3 = eth2 = BYNET 1 Port 4 = eth3 = BYNET 0

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Overview Cabling for 1-2 Cliques

BYNET 1

BYNET 0

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Overview IP Subnets

BYNET IP Subnet
> Used by PUT, Reconcile, pcl, psh, and other common Teradata network utilities. > The BYNET IP subnet is a virtual subnet. Packets are routed by BYNET Protocol Emulation with the BYNET Gigabit Ethernet Switches providing the physical transport.
Traffic is load-balanced between available switches. In the event of a cable or switch failure, traffic is automatically rerouted.

BYNET 0 Gigabit Ethernet IP Subnet

> Used internally by BYNET Protocol Emulation to poll the BYNET 0 Gigabit Ethernet Switchs management port. > Should not normally be used by external clients or utilities. > Can be used to access the BYNET 0 Gigabit Ethernet switch to configure flow control, update firmware, etc

BYNET 1 Gigabit Ethernet IP Subnet

> Same story as BYNET 0.

Note that these three subnets are logically independent!

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Overview IP Address Conventions

All IP Subnets are based on the systems SM3G IP configuration. BYNET IP Subnet:
> > > Default: Class A Network ID 39.0.0.0 Configured by Server Management via /etc/opt/teradata/sm3g/smainfo. Example:

SMP001-8:~ # cat /etc/opt/teradata/sm3g/smainfo PMA 40 BYNETIP 39.<x>.<y>.<z> BYNETMASK 255.240.0.0

<x> = 64 + (node collective number / 32) <y> = 8 * (node collective number % 32) <z> = node chassis number

BYNET 0 Gigabit Ethernet IP Subnet:

> > > > > Configured manually Uses the otherwise unused System 0 SM3G Primary IP Subnet Subnet mask: 255.240.0.0 Switch management address: 39.16.0.1 Node addresses: 39.<x>.<y>.<z> <x> = 16 + (node collective number / 32) <y> = 8 * (node collective number % 32) <z> = node chassis number Configured manually Uses the otherwise unused System 0 SM3G Secondary IP Subnet Subnet mask: 255.240.0.0 Switch management address: 39.32.0.1 Node addresses: 39.<x>.<y>.<z> <x> = 32 + (node collective number / 32) <y> = 8 * (node collective number % 32) <z> = node chassis number

BYNET 1 Gigabit Ethernet IP Subnet:

> > > > >

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Agenda

Overview
Switch Configuration Node Configuration

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Switch Configuration

The switch configuration for the 2580 is identical to the 2550. Use the 2550 configuration procedure for configuring the two Dell 6248 switches.

Nov 4, 2009

Teradata and Restricted

Teradata 2580 Knowledge Transfer Session

BYNET Agenda

Overview
Switch Configuration Node Configuration

Nov 4, 2009

Teradata and Restricted

10

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration

The Broadcom Gigabit Ethernet driver (bnx2) is used on the 2580 for the BYNET ports. BYNET driver version 3.1.2.2 is required for supporting this configuration The following steps are required to configure BYNET on a 2580 node:
> Install BYNET 3.1.2.2 (or later) > Configure eth2 and eth3 > > > > Make eth2 and eth3 mandatory Create /etc/opt/teradata/bynet/netdev Add BYNET host file entries Install BYNET software

> Reboot > Verify that BYNET is online > Verify access to the Ethernet Management Interface

Nov 4, 2009

Teradata and Restricted

11

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Configure eth2 and eth3

eth2 and eth3 need to be configured with static IP addresses. To configure eth3, create an /etc/sysconfig/network/ifcfg-eth3 file:
STARTMODE=onboot MTU= BOOTPROTO=static IPADDR=39.<x>.<y>.<z> NETMASK=255.240.0.0 <x> = 16 + (node collective number / 32) <y> = 8 * (node collective number % 32) <z> = node chassis number
For example, if the node collective number =100 and the node chassis number =5, then: x = 16 + (100/32) = 16 + 3 = 19 y = 8 * (100%32) = 8 * 4 = 32 z=5 Therefore, the IPADDR = 39.19.32.5

To configure eth2, create an /etc/sysconfig/network/ifcfg-eth2 file:

STARTMODE=onboot MTU= BOOTPROTO=static IPADDR=39.<x>.<y>.<z> NETMASK=255.240.0.0 <x> = 32 + (node collective number / 32) <y> = 8 * (node collective number % 32) <z> = node chassis number

The configuration updates will be picked up on the next reboot.

Nov 4, 2009

Teradata and Restricted

12

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Make eth2 and eth3 mandatory devices The eth2 and eth3 devices must be marked as mandatory to guarantee that they are available when the BYNET startup script (/etc/init.d/bynet) runs. To mark eth2 and eth3 as mandatory, edit /etc/sysconfig/network/config and add eth2 and eth3 to the list of mandatory devices:
MANDATORY_DEVICES=eth2 eth3

The mandatory attribute will be picked up on the next reboot.

Nov 4, 2009

Teradata and Restricted

13

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Create /etc/opt/teradata/bynet/netdev

The BYNET driver needs to be configured to operate in protocol emulation mode, it needs to be told which interfaces to use, and it needs to be given the IP addresses of the Gigabit Ethernet switch management ports.
> If an /etc/opt/teradata/bynet/netdev file is present, the BYNET driver will operate in protocol emulation mode. If no /etc/opt/teradata/bynet/netdev file is present, the BYNET driver will operate in native BYNET mode. The 2580 uses protocol emulation and thus requires this netdev file.

Create the /etc/opt/teradata/bynet directory if it doesnt already exist:

# mkdir p /etc/opt/teradata/bynet

Create an /etc/opt/teradata/bynet/netdev file containing the interface and switch IP information. Note that the order of these lines is important. eth3 must appear first since it is used for BYNET 0.
eth3 dell6248 39.16.0.1
eth2 dell6248 39.32.0.1

The updated configuration will be picked up on the next reboot.

Nov 4, 2009

Teradata and Restricted

14

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Add BYNET host file entries For convenience, a BYNET entry should be added to /etc/hosts for each node in the system. Here are example BYNET host file entries for an eight node system:
39.64.8.8 39.64.8.9 39.64.16.8 39.64.16.9 39.64.24.8 39.64.24.9 39.64.32.8 39.64.32.9 byn001-8 byn001-9 byn002-8 byn002-9 byn003-8 byn003-9 byn004-8 byn004-9

You may also want to add cop entries for Teradata. /etc/hosts changes take effect immediately.

Nov 4, 2009

Teradata and Restricted

15

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Install BYNET Software The SM3G node software must be installed before the BYNET software to ensure proper PMA ID and node type assignment. The latest x86_64 BYNET rpm should be downloaded from TSS. Use PUT's Install/Upgrade Software operation to install the rpm.

After BYNET has been installed, exit any active shells and start fresh ones to ensure that the BYNET user utilities will be found in your PATH.

Nov 4, 2009

Teradata and Restricted

16

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Reboot and verify that BYNET is online

After all the configuration steps have been completed, reboot the node so configuration changes can take effect. After the node reboots, check to make sure that the BYNET is functional:
> Run bam s and verify that the correct number of interfaces and nodes are online. For example:
# bam -s Node is running in protocol emulation mode. Node state: attached Net # 0 1 Nodes 4 4 Routes 4 4 Net name eth3 eth2

> Run netecho and verify that all nodes respond. For example:
# netecho byn001-8 byn001-9 byn002-8 byn002-9

Nov 4, 2009

Teradata and Restricted

17

Teradata 2580 Knowledge Transfer Session

BYNET Node Configuration Verify access to the Ethernet Management Interface

BYNET accesses the management interface of the Dell switches to determine the state of the switches. After configuration is complete, verify that the nodes can ping each switch
> Run ping <switch IP> to confirm that the switches are configured with the correct IP address and can be reached from the node. For example:
# ping 39.16.0.1 PING 39.16.0.1 (39.16.0.1) 56(84) bytes of data. 64 bytes from 39.16.0.1: icmp_seq=1 ttl=64 time=0.667 ms 64 bytes from 39.16.0.1: icmp_seq=2 ttl=64 time=0.678 ms # ping 39.32.0.1 PING 39.32.0.1 (39.32.0.1) 56(84) bytes of data. 64 bytes from 39.32.0.1: icmp_seq=1 ttl=64 time=0.667 ms 64 bytes from 39.32.0.1: icmp_seq=2 ttl=64 time=0.678 ms

Nov 4, 2009

Teradata and Restricted

18