Вы находитесь на странице: 1из 87

| |



Javier Delgado
Grid-Enabledment of Scientific Applications
Professor S. Masoud Sadjadi

| | 
 | 
| | 
 | 
| | 
 | 
©  
 ardware
G ^ne head/master node
G (Several) compute nodes
G Interconnection modality (e.g. ethernet)
 Software
G Parallel Programming Infrastructure
G Scheduler (optional)
G Monitoring application (optional)

| | 
 | 
î 

 Multiple users fighting for resources = bad
G Don't allow them to do so directly
 |omputer users are greedy
G Det the system allocate resources
 [sers like to know job status without having to
keep an open session

| | 
 | 
| î 
 
 Do-it-yourself (DIY)
 ^S|AR
 Rocks
 Pelican P| (formerly Parallel Knoppix)
 Microsoft Windows ||E
 ^penMosix (closed March 2008)
 |lustermatic (no activity since 2005)

| | 
 | 
{ | 
 Advantages
G |ontrol
G Dearning Experience
 Disadvantages
G |ontrol
G Administration

| | 
 | 
{ | 

 ardware Requirements
 ead Node Deployment
G |ore Software Requirements
G |luster-specific Software
G |onfiguration
 Adding compute nodes

| | 
 | 

 
 Several commodity computers:
G cpu/motherboard
G memory
G ethernet card
G hard drive (recommended, in most cases)
 Network switch
 |ables, etc.

| | 
 | 
î
 
 
 |ore system
G system logger, core utilities, mail, etc.
G Dinux Kernel
 Network Filesystem (NFS) server support
 Additional Packages
G Secure Shell (SS) server
G iptables (firewall)
G nfs-utils
G portmap
G Network Time Protocol (NTP)
| | 
 | 
î
 
 
 Additional Packages (cont.)
G inetd/xinetd ± For FTP, globus, etc.
G Message Passing Interface (MPI) package
G Scheduler ± PBS, SGE, |ondor, etc.
G Ganglia ± Simplified |luster ³ealth´ Dogging
 dependency: Apache Web Server

| | 
 | 


 | 


 Share /home directory
 |onfigure firewall rules
 |onfigure networking
 |onfigure SS
 |reate compute node image

| | 
 | 


| 
 Install compute node image on the compute
node
G Manually
G PXE Boot (pxelinux, etherboot, etc.)
G Redat Kickstart
G etc.
 |onfigure host name, NFS, etc.
 ... for each node!

| | 
 | 
a

 
 Software updates in head node require update
in compute node
 Failed nodes must be temporarily removed from
head node configuration files

| | 
 | 


| 
 But what if my boss wants a 200-node cluster?
G Monster.com
G ^R come up with your own automation scheme
G ^R [se ^S|AR or Rocks

| | 
 | 
| î 
 
 Do-it-yourself (DIY)
 ^S|AR
 Rocks
 Pelican P| (formerly Parallel Knoppix)
 Microsoft Windows ||E
 ^penMosix (closed March 2008)
 |lustermatic (no activity since 2005)

| | 
 | 
î|
 ^pen Source |luster Application Resources
 Fully-integrated software bundle to ease
deployment and management of a cluster
 Provides
G Management Wizard
G |ommand-line tools
G System Installation Suite

| | 
 | 

 
 Install ^S|AR-approved Dinux distribution
 Install ^S|AR distribution
 |reate node image(s)
 Add nodes
 Start computing

| | 
 | 
î|a   

 Download/install/remove ^S|AR packages
 Build a cluster image
 Add/remove cluster nodes
 |onfigure networking
 Reimage or test a node with the Network Boot
Manager

| | 
 | 
î|| 
 
 Everything the Wizard offers
 yume
G [pdate node packages
 | - The |luster |ommand |ontrol Tools
G provide cluster-wide versions of common
commands
G |oncurrent execution
G example 1: copy a file from the head node to all
visualization nodes
G example 2: execute a script on all compute nodes
| | 
 | 
|
| 
 cexec: execution of any standard command on
all cluster nodes
 ckill: terminates a user specified process
 cget: retrieves files or directories from all cluster
nodes
 cpush: distribute files or directories to all cluster
nodes
 cpushimage: update the system image on all
cluster nodes using an image captured by the
SystemImager tool | | 
 | 

|   !"
 crm: remove files or directories
 cshutdown: shutdown or restart all cluster
nodes
 cnum: returns a node range number based on
node name
 cname: returns node names based on node
ranges
 clist: returns all clusters and their type in a
configuration file
| | 
 | 
©    



š /etc/c .conf
šš
š describes cluster configuration
šš
cluster gcb {
gcb.fiu.edu šhead node
dead placeholder šchange command line to 1 indexing
compute-0-[0-8] šfirst set of nodes
exclude 5 šoffline node in the range (killed by J. Figueroa)
}
-------

| | 
 | 
 a
The ^S|AR Password Installer and [ser


Management
 Synchronize user accounts
 Set up passwordless SS
 Periodically check for changes in passwords

| | 
 | 
î î
 System Installation Suite
 Installs Dinux systems over a network
 Image-based
 Allows different images for different nodes
 Nodes can be booted from network, floppy, or
|D.

| | 
 | 
| î 
 
 Do-it-yourself (DIY)
 ^S|AR
 Rocks
 Pelican P| (formerly Parallel Knoppix)
 Microsoft Windows ||E
 ^penMosix (closed March 2008)
 |lustermatic (no activity since 2005)

| | 
 | 
 #
 Disadvantages
G Tight-coupling of software
G ighly-automated
 Advantages
G ighly-automated...
G But also flexible

| | 
 | 
 #
 The following 25 slides are property of [|
Regants

| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
{
 $ 

| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
 #  
 
î
 


Slides courtesy of David Villegas and Dany


Guevara

| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
Installation of |ompute Nodes

 Dog into Frontend node as root


 At the command line run:

M 

| | 
 | 
| | 
 | 
| | 
 | 
 
 | 
%
 Turn on the compute node
 Select to PXE boot or insert Rocks
|D and boot off of it

| | 
 | 
| | 
 | 
| | 
 | 
| | 
 | 
| 



 |ommand-line tools
 Image generation
 |luster Troubleshooting
 [ser Management

| | 
 | 
| 
 
 |luster-fork ± execute command on nodes
(serially)
 |luster-kill ± kill a process on all nodes
 |luster-probe ± get information about cluster
status
 |luster-ps ± query nodes for a running process
by name

| | 
 | 
 & 

 Basis: Redhat Kickstart file
G plus XMD flexibility
G and dynamic stuff (i.e. support for ³macros´)
 Image Docation: /export/home/install
 |ustomization: rolls and extend-compute.xml
 |ommand: rocks-dist

| | 
 | 
 & 


| | 
 | 
Source: 
      
© 
 Goal: Make a regular node a visualization node
 Procedure
G Figure out what packages to install
G Determine what configuration files to modify
G Modify    
G !"  

| | 
 | 
ù
 # 
 X-Windows Related
G X, fonts, display manager
 Display wall
G XDMX, |hromium, SAGE

| | 
 | 
a
'| 


 X configuration
G xorg.conf
G Xinitrc
 Display Manager |onfiguration

| | 
 | 
a   
 Rocks Directory: /var/411
G |ommon configuration files:
 Autofs-related
 /etc/group, /etc/passwd, /etc/shadow
 /etc/services, /etc/rpc
G All encrypted
 elper |ommand
G rocks-user-sync

| | 
 | 
î| 

 Rocks is now installed
 |hoose an MPI runtime
G MPI|
G ^penMPI
G DAM-MPI
 Start compiling and executing

| | 
 | 

 |
 Dive|D for instant cluster creation
 Advantages
G Easy to use
G A lot of built-in software
 Disadvantages
G Not persistent
G Difficult to add software

| | 
 | 
a
î 
 
 Windows Server 200 |ompute |luster Edition
(||E)
 Microsoft |ompute |luster pack (||P)
 Microsoft MPI (based on MPI|2)
 Microsoft Scheduler

| | 
 | 
a
||©
 Advantages
G [sing Remote Installation Services (RIS), compute
nodes can be added by simply turning it on
G May be better for those familiar with Microsoft
Environment
 Disadvantages
G Expensive
G ^nly for 64-bit architectures
G Proprietary
G Dimited Application base
| | 
 | 
 
 http://pareto.uab.es/mcreel/PelicanP|/
 http://pareto.uab.es/mcreel/ParallelKnoppix/
 http://www.gentoo.org/doc/en/hpc-howto.xml
 http://www.clustermatic.org
 http://www.microsoft.com/windowsserver200 /c
cs/default.aspx
 http://www.redhat.com/docs/manuals/linux/RD
-9-Manual/ref-guide/ch-nfs.html
 portmap man page
| | 
 |