Академический Документы
Профессиональный Документы
Культура Документы
_____________________________________________________________________________
_
Components of a Cluster
The main components of a cluster are the Personal
Computer and the interconnection network. The computer can
be built out of Commercial off the shelf components (COTS)
and is available economically.
The interconnection network can be either an ATM ring
(Asynchronous Transfer Mode), which guarantees a fast and
effective connection, or a Fast Ethernet connection, which is
commonly available now. Gigabit Ethernet which provides
speeds up to 1000Mbps,or Myrinet a commercial
interconnection network with high speed and reduced latency
are viable options.
But for high-end scientific clustering, there are a variety of
network interface cards designed specifically for clustering.
Those include Myricom's Myrinet, Giganet's cLAN and the
IEEE 1596 standard Scalable Coherent Interface (SCI). Those
4 Cluster Computing
_____________________________________________________________________________
_
cards' function is not only to provide high bandwidth between
the nodes of the cluster but also to reduce the latency (the time it
takes to send messages). Those latencies are crucial to
exchanging state information between the nodes to keep their
operations synchronized.
INTERCONNECTION NETWORKS
Myricom
Myricom offers cards and switches that interconnect at
speeds of up to 1.28 Gbps in each direction. The cards come in
two different forms, copper-based and optical. The copper
version for LANs can communicate at full speed at a distance of
10 feet but can operate at half that speed at distances of up to 60
feet. Myrinet on fiber can operate at full speed up to 6.25 miles
on single-mode fiber, or about 340 feet on multimode fiber.
Myrinet offers only direct point-to-point, hub-based, or switch-
based network configurations, but it is not limited in the number
of switch fabrics that can be connected together. Adding switch
fabrics simply increases the latency between nodes. The average
latency between two directly connected nodes is 5 to 18
microseconds, a magnitude or more faster than Ethernet.
5 Cluster Computing
_____________________________________________________________________________
_
Giganet
Giganet is the first vendor of Virtual Interface (VI)
architecture cards for the Linux platform, in their cLAN cards
and switches. The VI architecture is a platform-neutral software
and hardware system that Intel has been promoting to create
clusters. It uses its own network communications protocol rather
than IP to exchange data directly between the servers, and it is
not intended to be a WAN routable system. The future of VI now
lies in the ongoing work of the System I/O Group, which in
itself is a merger of the Next-Generation I/O group led by Intel,
and the Future I/O Group led by IBM and Compaq. Giganet's
products can currently offer 1 Gbps unidirectional
communications between the nodes at minimum latencies of 7
microseconds.
6 Cluster Computing
_____________________________________________________________________________
_
IEEE SCI
The IEEE standard SCI has even lower latencies (under 2.5
microseconds), and it can run at 400 MB per second (3.2 Gbps)
in each direction. SCI is a ring-topology-based networking
system unlike the star topology of Ethernet. That makes it faster
to communicate between the nodes on a larger scale. Even more
useful is a torus topology network, with many rings between the
nodes. A two-dimensional torus can be pictured as a grid of n by
m nodes with a ring network at every row and every column. A
three-dimensional torus is similar, with a 3D cubic grid of nodes
that also has rings at every level. Supercomputing massively
parallel systems use those to provide the relatively quickest path
for communications between hundreds or thousands of nodes.
The limiting factor in most of those systems is not the operating
system or the network interfaces but the server's internal PCI
bus system. Basic 32-bit, 33-MHz PCI common in nearly all
desktop PCs and most low-end servers offers only 133 MB per
second (1 Gbps), stunting the power of those cards. Some costly
high-end servers such as the Compaq Proliant 6500 and IBM
7 Cluster Computing
_____________________________________________________________________________
_
Netfinity 7000 series have 64-bit, 66-MHz cards that run at four
times that speed. Unfortunately, the paradox arises that more
organizations use the systems on the low end, and thus most
vendors end up building and selling more of the low-end PCI
cards. Specialized network cards for 64-bit, 66-MHz PCI also
exist, but they come at a much higher price. For example, Intel
offers a Fast Ethernet card of that sort for about $400 to $500,
almost five times the price of a regular PCI version
8 Cluster Computing
_____________________________________________________________________________
_
o Close Clusters
o Open Clusters
Close Clusters
They hide most of the cluster behind the gateway node.
Consequently they need less IP addresses and provide better
security. They are good for computing task
s.
Open Clusters
Close Cluster
High Speed Network
Service Network
gateway Front-end
gateway
node
External Network
Open Cluster
High Speed Network
External Network
Front-end
10 Cluster Computing
_____________________________________________________________________________
_
Security Considerations
Special considerations are involved when completing the
implementation of a cluster. Even with the queue system and
parallel environment, extra services are required for a cluster to
function as a multi-user computational platform. These services
include the well-known network services NFS, NIS and rsh.
NFS allows cluster nodes to share user home directories as well
as installation files for the queue system and parallel
environment. NIS provides correct file and process ownership
across all the cluster nodes from the single source on the master
machine. Although these services are significant components of
a cluster, such services create numerous vulnerabilities. Thus, it
would be insecure to have cluster nodes function on an open
network. For these reasons, computational cluster nodes usually
reside on private networks, often accessible for users only
12 Cluster Computing
_____________________________________________________________________________
_
through a firewall gateway. In most cases, the firewall is
configured on the master node using ipchains or iptables.
Having all cluster machines on the same private network
requires them to be connected to the same switch (or linked
switches) and, therefore, localized at the same proximity. This
situation creates a severe limitation in terms of cluster
scalability. It is impossible to combine private network machines
in different geographic locations into one joint cluster, because
private networks are not routable with the standard Internet
Protocol (IP).
Combining cluster resources on different locations, so that users
from various departments would be able to take advantage of
available computational nodes, however, is possible.
Theoretically, merging clusters is not only desirable but also
advantageous, in the sense that different clusters are not
localized at one place but are, rather, centralized. This setup
provides higher availability and efficiency to clusters, and such a
proposition is highly attractive. But in order to merge clusters,
all the machines would have to be on a public network instead of
a private one, because every single node on every cluster needs
to be directly accessible from the others. If we were to do this,
13 Cluster Computing
_____________________________________________________________________________
_
however, it might create insurmountable problems because of
the potential--the inevitable--security breaches. We can see then
that to serve scalability, we severely compromise security, but
where we satisfy security concerns, scalability becomes
significantly limited. Faced with such a problem, how can we
make clusters scalable and, at the same time, establish a rock-
solid security on the cluster networks? Enter the Virtual Private
Network (VPN).
VPNs often are heralded as one of the most cutting-edge,
cost-saving solutions to various applications, and they are
widely deployed in the areas of security, infrastructure
expansion and inter-networking. A VPN adds more dimension to
networking and infrastructure because it enables private
networks to be connected in secure and robust ways. Private
networks generally are not accessible from the Internet and are
networked only within confined locations.
The technology behind VPNs, however, changes what we
have previously known about private networks. Through
effective use of a VPN, we are able to connect previously
unrelated private networks or individual hosts both securely and
transparently. Being able to connect private networks opens a
14 Cluster Computing
_____________________________________________________________________________
_
whole slew of new possibilities. With a VPN, we are not limited
to resources in only one location (a single private network). We
can finally take advantage of resources and information from all
other private networks connected via VPN gateways, without
having to largely change what we already have in our networks.
In many cases, a VPN is an invaluable solution to integrate and
better utilize fragmented resources.
In our environment, the VPN plays a significant role in
combining high performance Linux computational clusters
located on separate private networks into one large cluster. The
VPN, with its power to transparently combine two private
networks through an existing open network, enabled us to
connect seamlessly two unrelated clusters in different physical
locations. The VPN connection creates a tunnel between
gateways that allows hosts on two different subnets (e.g.,
192.168.1.0/24 and 192.168.5.0/24) to see each other as if they
are on the same network. Thus, we were able to operate critical
network services such as NFS, NIS, rsh and the queue system
over two different private networks, without compromising
security over the open network. Furthermore, the VPN encrypts
all the data being passed through the established tunnel and
15 Cluster Computing
_____________________________________________________________________________
_
makes the network more secure and less prone to malicious
exploits.
The VPN solved not only the previously discussed
problems with security, but it also opened a new door for
scalability. Since all the cluster nodes can reside in private
networks and operate through the VPN, the entire infrastructure
can be better organized and the IP addresses can be efficiently
managed, resulting in a more scalable and much cleaner
network. Before VPNs, it was a pending problem to assign
public IP addresses to every single node on the cluster, which
limited the maximum number of nodes that can be added to the
cluster. Now, with a VPN, our cluster can expand in greater
magnitude and scale in an organized manner. As can be seen, we
have successfully integrated the VPN technology to our
networks and have addressed important issues of scalability,
accessibility and security in cluster computing.
16 Cluster Computing
_____________________________________________________________________________
_
Beowulf Cluster
Basically, the Beowulf architecture is a multi-computer
architecture that is used for parallel computation applications.
Therefore, Beowulf clusters are primarily meant only for
processor-intensive and number crunching applications and
definitely not for storage applications. Primarily, a Beowulf
cluster consists of a server computer that controls the
functioning of many client nodes that are connected together
with Ethernet or any other network comprising of a network of
switches or hubs. One good feature of Beowulf is that all the
system's components are available from off-the-shelf component
and there is no special hardware that is required to implement it.
It also uses commodity software - most often Linux - and other
commonly available components like Parallel Virtual Machine
(PVM) and Messaging Passing Interface (MPI).
Besides serving all the client nodes in the Beowulf cluster,
the server node also acts as a gateway to external users and
passes files to the Beowulf system. The server is also used to
drive the console of the system from where the various
17 Cluster Computing
_____________________________________________________________________________
_
parameters and configuration can be monitored. In some cases,
especially in very large Beowulf configurations, there is
sometimes more than one server node with other specialized
nodes that perform tasks like monitoring stations and additional
consoles. In disk-less configurations, very often, the individual
client nodes do not even know their own addresses until the
server node informs them.
The major difference between the Beowulf clustering system
and the more commonly implemented Cluster of Workstations
(CoW) is the fact that Beowulf systems tend to appear as an
entire unit to the external world and not as individual
workstations. In most cases, the individual workstations do not
even have a keyboard, mouse or monitor and are accessed only
by remote login or through a console terminal. In fact, a
Beowulf node can be conceptualized as a CPU+memory
package that can be plugged into the Beowulf system - much
like would be done with a motherboard.
It's important to realize that Beowulf is not a specific set of
components or a networking topology or even a specialized
kernel. Instead, it's simply a technology for clustering together
Linux computers to form a parallel, virtual supercomputer.
18 Cluster Computing
_____________________________________________________________________________
_
PLAPACK
PLAPACK is an MPI-based Parallel Linear Algebra
Package that provides an infrastructure for building parallel
dense linear algebra libraries. PLAPACK provides 3 unique
features.
o Physically based matrix distribution
o API to query matrices and vectors
o Programming interface that allows object oriented
programming
26 Cluster Computing
_____________________________________________________________________________
_
ScaLAPACK
ScaLAPACK is a library of high performance linear
algebra routines for distributed memory MIMD computers. It
contains routines for solving systems of linear equations .Most
machine dependencies are limited to two standard libraries
called the PBLAS, or Parallel Basic Linear Algebra Subroutines,
and the BLACS ,or the BLACS, or Basic Linear Algebra
Communication Subroutines. LAPACK and ScaLAPACK will
run on any system where the PBLAS and the BLACS are
available.
27 Cluster Computing
_____________________________________________________________________________
_
Diskless Cluster
Server assign IP
TFTP load
OS
Server Supply OS