Simulation of Missile Aerodynamics

using Parallel Computing

FASTRAN Tutorial 9

1
 Objectives
This tutorial covers the following subjects:
Setting up a model for running CFD-FASTRAN solver in parallel
mode.
You would require a UNIX/LINUX Operating System to run
Structured Grid cases in Parallel.

2
 Problem Description
A missile staging problem is modeled as shown in Fig1 and Fig 2 in
the next slides.
This tutorial is the same as Tutorial 5.
Please refer to Tutorial 5 for the procedure on setting up the
problem using CFD-FASTRAN-GUI.
In this tutorial, the steady state part of Tutorial 5 is set up for the
parallel version of CFD-FASTRAN solver.

3
 Problem Description

Fig 1: Overview of the payload and booster

vehicles grid boundaries
4
 Problem Description

Fig 2: The 6DOF motion of the booster.

5
 Procedure:Model
To start CFD-FASTRAN,
type “CFD-FASTRAN” on
the screen. This action
launches the graphical
user interface.
Load the model
Select File/Open from the
Menu bar
The Open dialog panel
opens.

6
 Procedure:Model
Select the file named
“staging.DTF” and press
the “Accept” button to
read the file.
The DTF file is read into
CFD-FASTRAN and a
model properties window
appears on the screen as
shown in the figure.
Wireframe outline of the
model appears in the
viewing window after
press OK on the model
properties panel

7
 Procedure:Model

Fig 3: Model as shown in CFD-FASTRAN-GUI

8
 Procedure: Parallel Setup
Create fastran.hosts file for the parallel-solver.
This step is involved only when a user is setting up the
parallel solver for the first time.
This steps creates a fastran.hosts file in the .mdice
directory under user’s home directory
($HOME/.mdice/fastran.hosts).
If fastran.hosts is already present, this step is skipped by
the GUI. The user may modify a pre-existing file
manually by opening it in a text editor. The user may
recreate the file by deleting it and repeating this step.
The fastran.hosts file is used to obtain the default values
for the various inputs associated with the processors
such as the default working directory, relative speed,
total number of processors on the machine and whether
the processor is on a NFS or distributed network.
9
 Procedure:Parallel Setup
Press the Configure
Parallel button to
activate the Parallel
Setup page.
The CFD-FASTRAN
Hosts Setup Page
opens. Please note that
if you have already
performed this step
before then you will not
see the Hosts Setup,
instead the GUI will
continue with Step
shown on Slide Number

10
 Procedure:Parallel Setup

Fig 4: Input panel for creating fastran.hosts file.

11
 Procedure:Parallel Setup
An input panel for creating fastran.hosts file
appears as shown in fig 4.
Click on Add New Host to add a host and
provide default values for the working directory,
the number of processors, relative speed of the
processor and a value for NFS (Y/N indicating
whether the processor is NFS mounted or not).
To delete a host, select the host and click on
Delete Host.
Click OK to save the settings to the
fastran.hosts file.

12
 Procedure:Parallel Setup
After clicking OK, the GUI continues with the Parallel
Setup panel when the user clicks on the “Configure
Parallel” push button again. However, the fastran.hosts
file can be edited manually in a text editor or through the
GUI by deleting it from the $HOME/.mdice/fastran.hosts
file and repeating this step.
In this tutorial assume that the parallel job will be run as
2 processes on the same machine edison.esi-cfd.com
(figure 5 on the next slide).
Therefore these two hosts must be added in the
fastran.hosts file. The user must substitute the machine
names with his/her own set of machine names.

13
 Procedure:Parallel Setup

Fig 5: List of hosts for the parallel simulation

14
 Procedure:Parallel Setup
Press the Configure Parallel tab to activate the Parallel
Setup Page.
Select a host from the list of hosts under fastran.hosts.
Click on Add/Delete to add/delete the selected host. For
a multi-processor machine, the Add button may need to
be clicked multiple times.
The user must add the machine for each instance. For
example if a machine has four processors and the user
wants to use three processor of the machine and the
machine should be added three times.
The GUI obtains the default values from the
fastran.hosts file. If necessary, modify the default values
for Working directory, Relative Speed per Processor,
NFS and Display.

15
 Procedure:Parallel Setup

Fig 6: Input panel for parallel setup

16
 Procedure:Parallel Setup
Important Notes:
A separate working directory (e.g. /proj/fastran/staging/cpu1
and /proj/fastran/staging/cpu2 etc.) must be provided by the
user for each processor listed under Hostname as shown in
Fig 7.Also, a separate working directory (e.g.
/proj/fastran/staging) should contain the original input files.
Please note that for NFS mounted systems, it is often
convenient to have all the directories (one for each processor)
on one machine. In that case the processors need to access
their assigned directories across the network. Therefore the
directory pathnames must be prepended by the NFS mount
name and the machine name where the directory is located.
For example the pathname may be similar to
/hosts/”machine_name”/proj/fastran/staging/cpu1 etc.), where
the NFS mount name is /hosts (/net is also commonly used).
Please consult your system administrator if you do not know
the NFS mount name.
17
 Procedure:Parallel Setup
Important Notes:
The user must also make sure that the directory
permissions are proper, so that the processor can
read/write their assigned directories and the files in the
directories. One way of achieving this is to add the
following line to your .cshrc or .bashrc file: umask 000.
Please note that this will cause all the directories to be
created with read, write, execute permissions for all
users and all files to be created with read and write
permission for all users.
The first processor in the list is always the Master
Processor. This is indicated by the letter “M” in the list
on the left to Hostname list.
Value of Display can be Y (yes) or N (No). If the value is
Y then an xterm window is displayed showing the
screen output for that processor.
18
 Procedure:Parallel Setup
Similarly, add other processors from the hosts
name listed in the fastran.hosts list by first
selecting the host and then clicking on Add as
shown in figure 7 on the next slide.
Enter “Y” in the Display (Y/N) field.

19
 Procedure:Parallel Setup

Fig 7: Adding hosts to the parallel setup panel.

20
 Procedure:Parallel Setup
It is the users responsibility to perform domain
assignments to different processors. This can be done
automatically by clicking on Distribute Zones button.
This step will assign the zones to different processors
depending on the domain sizes and the processors
speed so that the overall computation time can be
optimized. However, the user may modify this
distribution using Add/Delete button on the extreme right
of the panel shown in fig 8 in the next slide.
Click on Distribute Zones.
The Domain Numbers list will show the list of zones
assigned to the highlighted processor. Highlight other
processors to examine which domains have been
assigned to that particular processor.

21
 Procedure:Parallel Setup

Fig 8: Distribute domains to the processors.

22
 Procedure:Parallel Setup
For problems involving chimera/overset grids,
one processor must be assigned exclusively for
performing chimera computations. This
processor must not have any domains
assigned to it.
Add a chimera processor by first selecting the
machine and then clicking on Add. Then
highlight the newly added processor and click
on Chimera to mark this processor as chimera
as shown in fig 8. The chimera processor is
marked by the letter “C”.

23
 Procedure:Parallel Setup
Select NFS or Distributed toggle button
depending on whether the network system is
NFS or distributed.
In the Display (Y?N) field, enter “Y”.
Click OK to save the settings to the staging.DTF
file.

24
 Procedure:Saving the Model
Save CFD-FASTRAN model.
The CFD-FASTRAN model
must be saved in DTF (Data
Transfer Facility) format to be
able to run the parallel
version of CFD-FASTRAN
solver.
Select Save from the File
menu.
Select the filename
staging.DTF and click OK.
Exit from the GUI by
selecting Quit.

25
 Procedure:Run Parallel CFD-FASTRAN
Parallel version of CFD-FASTRAN solver
must be invoked from the operating system
command prompt.
It is required to have an mdice registry running
with mdice daemons running on all the
machines list in figure 8. There are three steps:
Start mdice registry.
Start mdice daemons.
Launch the parallel job.

26
 Procedure:Run Parallel CFD-FASTRAN
Let’s assume that the user wants to use the
following account to run parallel jobs.
Hostname Account
Edison.esi-cfd.com fastran
Newton.esi-cfd.com fastran

27
 Procedure:Start MDICE registry
The first step is to start a mdice registry on a
machine connected to network. This machine could
be any of the machine listed above but it is not
necessary. Let’s assume this machine name is
edison.esi-cfd.com and the account on that
machine is fastran. The first step is to log into
edison.esi-cfd.com as user fastran and type
>> mdicer –nogui &
This will create an mdice registry on machine
edison.esi-cfd.com as follows:

28
 Procedure:Start MDICE daemons
The second step is to start mdice daemons on
all the accounts as listed in the previous slide
(edison and newton in this example).
So in this example, log into machine edison.esi-
cfd.com as user fastran and type:
>> mdiced -r edison.esi-cfd.com:2008 &
The previous step will start a daemon for
registry edison.esi-cfd.com:2008 on machine
edison.esi-cfd.com.

29
 Procedure:Start MDICE daemons
Similarly, type the following to log into newton.esi-
cfd.com as fastran and start mdice daemon:
>> mdiced -r edison.esi-cfd.com:2008 &
You can now log out of the machine newton.esi-cfd.com.
The daemons will keep running in the background.
At the command prompt you will see the following output
when launching the MDICE daemons.

30
 Procedure:Launch the parallel job
The user must log into the machine (e.g., on
edison.esi-cfd.com as user fastran) from which
(s)he wants to submit the job. Then change
directory to the location where all the DTF file
(staging.DTF)is located.
Note: Please make sure that the DISPLAY
environment variable is set correctly.

31
 Procedure:Launch the parallel job
For NFS mounted network system, type the following to
invoke the parallel solver:
>> CFD-FASTRAN-SOLVERP –r mdice_registry_name -dtf
modelname > model.out
mdice_registry_name is the mdice registry where the user
wants to submit the job.
modelname is the CFD-FASTRAN model name which will be
“staging” for this tutorial.
Note: ALL OUTPUT GOES TO THE SCREEN BY DEFAULT.
If you wish to capture the solver output, you can redirect it to a
file, for example: model.out
Assuming that the mdice registry name is edison.esi-
cfd.com:2008
Therefore type the following:
>> CFD-FASTRAN-SOLVERP –r edison.esi-cfd.com:2008
-dtf staging > staging.out
32
 Procedure:Launch the parallel job
The user can also type CFD-FASTRAN-SOLVERP
at the command prompt to get the usage with
description of all the options as shown in the figure
below.

33
 Procedure:Launch the parallel job
For Distributed network, first make sure that the
File System button Figure 8 is set to Distributed.
Save the DTF file in CFD-FASTRAN-GUI and you
can then Quit GUI.
Then run the parallel solver by typing the following,
CFD-FASTRAN-SOLVERP –r registry_name –dtf
modelname –rcp –nonfs > model.out
If the user does a ssh then he/she can use –scp
option for the file transfer.
This tutorial is now finished.

34