FDTD S olu tion s

Linux Installation

Release 8.7

Contents

Table of Contents
Part I Linux
1
2
3
4
5

Installation
............................................................................................................
instructions
2
Configure
............................................................................................................
engine
7
Cluster ............................................................................................................
setup
9
Running
............................................................................................................
simulations
12
FAQ ............................................................................................................ 19
Starting
.................................................................................................................................................
the program
19
Crashes
.................................................................................................................................................
and graphics problem s
20
Parallel
.................................................................................................................................................
engine issues
20
USB.................................................................................................................................................
hardw are key not found error
23
Installing
.................................................................................................................................................
w ithout adm inistrator access
23
Installing
.................................................................................................................................................
m ultiple versions
24
Installing
.................................................................................................................................................
on Ubuntu
25
Trial.................................................................................................................................................
license validation
27

6 Next Steps
............................................................................................................ 27

2003 - 2013 Lumerical Solutions, Inc

Linux Installation

Linux
Welcome to the FDTD Solutions Installation Manual for Linux. Please contact
support@lumerical.com if you have any questions.

1.1

Installation instructions
The following section describes the standard installation process for FDTD Solutions. In
order to use our software, you would need at least two install packages: The Lumerical
FlexNet License Manager and FDTD Solutions.
Lumerical provides a floating license; which means that the license can be shared between
multiple machines in the same broadcast domain or subnet. The following figures illustrates
two possible scenarios on how to setup your Lumerical FlexNet License Manager and any
Lumerical Products.

Figure 1: License setup for two users with

non-dedicated license server

Figure 2: License setup for two users with a

dedicated license server

Please ensure that you have an Administrator privilege before continuing with the install
process.

Installation Procedure
Pre-Step: Check for the system requirements and supported platforms
View details
For the complete list of system requirements and supported platforms, please consult
the Technical Specifications

1. Download the Installation Packages of FDTD Solutions and Lumerical FlexNet

License Manager
View details

2003 - 2013 Lumerical Solutions, Inc

Linux

The installation packages are available from Lumerical Download Center page (registration
is required)

2. Extract the Compressed Installation Packages of FDTD Solutions and Lumerical

FlexNet License Manager
View details
Upon successful download, unpack both installation packages with the following
command:
tar -zxf <name of downloaded file>

3. Run the installer of FlexNet License Manager

View details
Go to the FlexNet License Manager directory that you have extracted, and run the
installer:
sudo yum install lumerical-flexlm-<version>.i386.rpm

2003 - 2013 Lumerical Solutions, Inc

Linux Installation

If you don't have yum installed, you can use the following:
sudo rpm -ivh lumerical-flexlm-<version>.i386.rpm
For a more detailed instructions of installing FlexNet License Manager, please see "Part
1: Install License Manager" from the following link: Getting Started
4. Activate your License Activation Code
View details
Upon successful installation, you would need to launch the Lumerical Activation Utility to
activate your license.
sudo /opt/lumerical/lumerical-flexlm/lumerical-activationutility
In general, there are two activation modes that are available: Online Mode and Offline
Mode.
The online activation would be faster and easier, so this is the recommended method
for both customers and trial users.
The offline activation is a backup solution for customer who is unable to activate license
through the internet due to high security network environment. License activation will be
done by sending a "request XML file" to Lumerical Support, and a "response XML file"
will be provided. Please note: Offline mode activation is only available for customer.
Note: If you are using non-graphical Linux, please see "Part 2: Activate License" link
below for further information on how to activate your license with a Console Mode.
Please enter the 8 characters Activation Code that you have received (the dash is also
required), and click Activate. It may take a few minutes before the license activation
completes.

2003 - 2013 Lumerical Solutions, Inc

Linux

Please keep your Activation Code for your records. If you need to change your license
server, you can "de-activate" the license and re-use the Activation Code on the new
server.
For a more detailed instructions for activating your license, please see "Part 2: Activate
License" from the following link: Getting Started
5. Run the installer of FDTD Solutions
View details
Go to the FDTD Solutions directory that you have extracted from Step 2, and then run the
install utility script with the command
sudo sh ./install.sh
Follow the steps and instructions on the script to complete the installation.
Tip: Resolving dependencies
If required system libraries are missing from your computer, the install script will fail.
These dependencies must be resolved before the install script will run successfully.
One option is to manually install the missing libraries, then re-run the install script.
Another option is to install Lumerical's software with the yum command (if it is available
on your computer), rather than with the install script. The yum command will attempt to
automatically resolve dependency problems by downloading and installing the missing
libraries. The command to install with yum will be something like the following:
yum install ./rpm_install_files/FDTD-x.x.x-1.rhel5.x86_64.rpm

2003 - 2013 Lumerical Solutions, Inc

Linux Installation

6. Launch FDTD Solutions software

View details
This can be done from your terminal by using the following command
/opt/lumerical/fdtd/bin/fdtd-solutions&

Note: If you have installed the Lumerical FlexNet License Manager on a different
machine, you would be required to configure your FDTD Solutions License
Configuration.
1. Open your FDTD Solutions License Configuration
/opt/lumerical/fdtd/bin/fdtd-config-license
2. On the "Server" box, enter the host name or the IP address of your license server.
3. Click OK to save your changes
4. Re-launch FDTD Solutions
For a more detailed instructions of accessing your license, please see "Part 3: Access
the License" from the following link: Getting Started
This completes the FDTD Solutions installation for Linux.
Optional-Step: Add FDTD Solutions to your system path
View details
Run the config script with the following command:
sh /opt/lumerical/fdtd/bin/fdtd-config.sh

2003 - 2013 Lumerical Solutions, Inc

Linux

This script will add the install directory of FDTD Solutions to your system path and add
an icon on your desktop. After running the script, you can start FDTD Solutions by
typing fdtd-solutions.

1.2

Configure engine
This page describes how to configure the simulation engine so you can run jobs on
computers within your local area network. FDTD Solutions must be installed on each
computer, as described in the main Installation instructions 2 section, before you attempt
to configure the engine.
Concurrent computing refers to running multiple simultaneous simulations, usually on
different computers. Distributed computing refers to running a single simulation using
multiple cores/processors/workstations, giving access to a greater total amount of memory
and reducing computation time. Only some simulation engines support distributed
computing (eg. the FDTD engine).
Note: Users with only 1 computer
If you have only one computer, skip steps 1-3.
1. Configure your firewall
Many Linux clusters communicate across a private Ethernet network and therefore firewall
security may not be required. If no firewall is in use in your network, this step may be
skipped.
The MPI processes communicate using a range of ports. It's easiest to simply disable
the firewall on all nodes. An alternate solution is to configure MPI to use a specific range
of ports, then create exceptions for those ports. See the MPI documentation for details.
If you want to leave the firewall turned on, two additional firewall exceptions are required:
In some configurations, MPI requires the use of the ssh programs to start remote
processes on the compute nodes during parallel execution. Ensure that the ssh port 22
is allowed to accept incoming TCP/IP connections on all of your compute nodes.
The USB Network Keys use ports 1947:udp and 1947:tcp (port 1947 for both the UDP
and TCP protocols) for licensing.
2. Setup a network directory
To launch simulations on an arbitrary node, it's best to setup a network file system. The
network file system must be accessible to all nodes under the same name. This will allow
any node to access the simulation files.
On many Linux networks, each user's home directory is a network file system and is

2003 - 2013 Lumerical Solutions, Inc

Linux Installation
common to all nodes. If this is the case you may use your home directory to store your
simulation files. For more information on creating a network file systems, see your
operating system documentation.
3. Configure your compute nodes to allow remote login without a password, as the version
of MPICH included with the installation package uses ssh to start remote jobs. If this is not
setup, the user will have to type their password many times. On your primary computer,
enter the following commands to create a set of ssh keys.
ssh-keygen -t rsa
Press enter several times to accept all the defaults and an empty passphrase. This creates
your public/private keys and saves them in your home directory under the $HOME/.ssh
folder. Next you must place your public key in the text file $HOME/.ssh/
authorized_keys on each compute node. This can be accomplished using the following
commands for each compute node:
ssh <node name> "mkdir -p ~/.ssh; chmod 700 ~/.ssh"
cat ~/.ssh/id_rsa.pub | ssh <node name> "cat - >> ~/.ssh/
authorized_keys"
ssh <node name> "chmod 600 ~/.ssh/authorized_keys"
Note that if your home directory is on a network file system and is the same directory for all
compute nodes, then you will only have to run the above command one time. Once you
have completed this step, you should be able to login to any of the compute nodes using
the ssh <node name> command without entering a password.
Note: Determining the Name / IP address of your computer
If you need to find the IP address of a computer, use the command /sbin/ifconfig
4. Check resources
1. Open FDTD Solutions
2. Open the Resource configuration
window, (in the Simulation ->
Configure resources menu)
3. Add additional resources to the
list (Optional)
4. Click the Run tests button
To configure your system to use a
different version of MPI, see FAQ Parallel engine issues 20 .

2003 - 2013 Lumerical Solutions, Inc

Linux

Copy the file /opt/lumerical/fdtd/examples/paralleltest to your working

directory. Open paralleltest, then switch to the Optimization and Sweeps window.
Select the parameter sweep named test_sweep and click the Run button. If the sweep
runs properly, your system is configured properly.
Note: Log files
The simulation engine creates a log file named filename_p0.log. The information in
this file can be helpful when debugging problems with the parallel engine.
This completes the simulation engine configuration on a local area network.

1.3

Cluster setup
To install FDTD Solutions on a computer cluster or network of workstations, follow these
instructions. On a computer cluster, the computers are generally referred to as compute
nodes rather than workstations. Often one or more computers in a cluster are designated
as management nodes and are reserved for launching jobs and/or storing data files. For the
purpose of these instructions, we will use the terminology of compute nodes and
management nodes.
1. Install software
Install FDTD Solutions on each node of your cluster (or group of workstations) as described
in the main installation 2 section.
2. Flexnet license configuration
The FlexNet licence manager should be installed on the management node. FDTD
Solutions can be installed on both the management node and the compute nodes.
Note for large computer cluster installs:
If you have a large number of compute nodes, you may find that manually running the
install script is time consuming. One way to avoid this is to install on a network file
system that is visible to all the compute nodes. This can be accomplished using the
install utility and specifying an alternate install directory instead of /opt/lumerical/
fdtd.
3. Configure your firewall
Many Linux clusters communicate across a private Ethernet network and therefore firewall
security may not be required. If no firewall is in use on your cluster nodes then this step
may be skipped.
The MPI software requires the use of the ssh programs to start remote processes on the
compute nodes during parallel execution. If you are using a firewall, you should ensure

2003 - 2013 Lumerical Solutions, Inc

10

Linux Installation
that the ssh port 22 is allowed to accept incoming TCP/IP connections on all of your
compute nodes. Additional firewall exceptions may be required, depending on your
MPICH version (e.x. MPICH ch_p4 uses ports 1024-65535 by default).
The USB Network Keys use ports 1947:udp and 1947:tcp (port 1947 for both the UDP
and TCP protocols) for licensing.
4. Setup a network directory
To launch simulations on an arbitrary compute node, it's best to setup a network file
system on one of the nodes (typically the management node). The network file system
should be accessible to all nodes in the cluster under the same mount point. This will allow
any node to access the simulation tile.
On many Linux clusters each user's home directory is a network file system and is
common to all nodes. If this is the case you may use your home directory to store your fsp
files for parallel simulation. Otherwise, you can create a network file share on your
management node. For more information on creating a network file systems, see your
Linux operating system documentation.
5. Determine which executables to run
When running parallel simulations, inter-process communication is accomplished using a
Message Passing Interface (MPI) library. Different MPI library implementations allow
parallel execution on a variety of different types of hardware. Several versions of MPICH and
MPICH2 are included with the standard product installation. You should select the MPI
version that is most appropriate for your system, and select the corresponding engine
executable. The installDir/bin/fdtd-mpi-status.sh script determines which
options are available on your system. The default option MPI version is MPICH2 Nemesis, located at installDir/mpich2/nemesis/bin/mpiexec.
The corresponding simulation engine is installDir/bin/fdtd-engine-mpich2nem.
For a complete list of MPI and engine versions, see the FAQ - FDTD parallel config 20
page.
6. Login without passwords
If you are using a scheduler, or are linking to other local copies of MPI libraries that are
already configured and running parallel jobs, this step may be unnecessary.
Your compute nodes must be configure to allow remote login without a password. This will
allow jobs to start without the user having to type a password for each compute node. By
default, the version of MPICH2 that is included will use the ssh command to start remote
jobs. The ssh command can be configured to use public keys instead of passwords for
login. To set this up, run the following command on the management node where you plan
to launch your jobs:
ssh-keygen -t rsa

2003 - 2013 Lumerical Solutions, Inc

Linux

11

Press enter several times to accept all the defaults and an empty passphrase. This creates
your public/private keys and saves them in your home directory under the $HOME/.ssh
folder. Next you must place your public key in the text file $HOME/.ssh/
authorized_keys on each compute node. This can be accomplished using the following
commands for each compute node in your cluster:
ssh <node name> "mkdir -p ~/.ssh; chmod 700 ~/.ssh"
cat ~/.ssh/id_rsa.pub | ssh <node name> "cat - >>\ ~/.ssh/
authorized_keys"
ssh <node name> "chmod 600 ~/.ssh/authorized_keys"
Note that if your home directory is on a network file system and is the same directory for all
compute nodes, then you will only have to run the above command one time for one of the
nodes. Once you have completed this step, you should be able to login to any of the
compute nodes using the ssh <node name> command without entering a password.
7. Run a simple MPI test program
Lumerical includes a simple test program to test your MPI installation. If CPi runs
successfully, you will see the value of pi (3.1415....) printed on your screen as well as a
message from each of the computers that participates in the calculation. To run CPi, use a
command like this:
installDir/mpich2/nemesis/bin/mpiexec -n 4 installDir/mpitest/
cpi-mpich2nem
The option -n specifies the number of processes to use in the computation. You may
change the number 4 to match the number of compute nodes or processors in your
system. There are also other advanced options for mpiexec/mpirun that can be listed with
the command mpiexec -h. You should try to run CPI with the same arguments that will be
used to run the actual simulation engine.
8. Run the simulation engine
Copy the file installDir/examples/paralleltest.fsp to your working directory.
The command required to run the simulation should be similar to what was required for the
cpi test.
installDir/mpich2/nemesis/bin/mpiexec -n 4 installDir/bin/fdtdengine-mpich2nem paralleltest.fsp
For more information, see the FAQ - Parallel Issues 20 section of this installation manual,
and the User Guide - Running simulations section of the online help.

2003 - 2013 Lumerical Solutions, Inc

12

Linux Installation

1.4

Running simulations
In this section we will explain how to configure your system to launch jobs in two common
ways:
From the Graphical User Interface (GUI) using the Run button, script commands and the
optimization and parameter sweep windows.
From the command line with no GUI interface.
The list below explains how to configure your system, depending on your hardware and how
you intend to launch simulation:
1. Launch simulations from the GUI using your local workstation for the
simulations.
It's likely that this will work after installation immediately without further configuration.
Please see Resource configuration in the User Guide if you want to change the default
number of cores used for each simulation.
2. Launch simulations from the GUI using several computers on your LAN or on a
cluster without a scheduler.
This may take some configuration initially. Please see Configure engine 7 for the
necessary steps and then Resource configuration in the User Guide.
3. Launch simulations from the command line on your local workstation
The complete command to start the simulation engine in parallel using MPI is quite lengthy
and complex. The fdtd-run-local.sh shell file automatically generates the complete
command, making it much easier to start simulations from the command line. If you want
to use non-default MPI options, you can edit this file as needed. To run a simulation with
this shell file, use
/opt/lumerical/fdtd/bin/fdtd-run-local.sh -n 8 file1.fsp
to run the file using 8 cores. If the bin directory is in your path, the command can be
shorted to
fdtd-run-local.sh -n 8 file1.fsp
You can also use this command to run multiple fsp files. For example
fdtd-run-local.sh -n 8 file1.fsp file2.fsp
will run both files sequentially using 8 cores and
fdtd-run-local.sh -n 8 *.fsp
will run all the simulation files in the current directory.
4. Launch jobs by submitting to a scheduler from the command line
We provide a template submission script for a linux based PBS scheduling system. Some
modifications may be required depending on the configuration of your cluster and it may be
necessary to involve your System Administrator for this configuration.

2003 - 2013 Lumerical Solutions, Inc

Linux

13

The template of submission scripts composed of 3 parts for easier customization and reuse
/opt/lumerical/fdtd/bin/fdtd-run-pbs.sh: The master script file used to
submit your simulations to the scheduler. It takes fsp files as arguments, produces the
submission script and runs the qsub command. It is documented with comments in case
it needs to be customized. If you modify this file, we recommend that you make a copy
of it under a different name, such as fdtd-run.sh so that your changes will not be
overwritten when you upgrade the software.
/opt/lumerical/fdtd/bin/fdtd-process-template.sh: this script is
reference by fdtd-run-pbs.sh but should not have to be modified.
/opt/lumerical/fdtd/templates/fdtd-pbs-template.sh: This is the
template script that is used to create a job submission script for your scheduler. it must
be configured once to ensure that we create the correct submission script as required by
your System Administrator. When modifying this file, we recommend that you make a
copy of this file (to avoid overwriting when you upgrade the software). You will also then
need to modify the TEMPLATE variable in fdtd-run-pbs.sh script to specify the
correct template file.
Once configured, you can submit jobs to the scheduler with a command like
sh ./fdtd-run-pbs.sh [-n <procs>] fsp1 [fsp2 ... [fspN]]
For example, to submit all the fsp files in a directory to the scheduler, asking for 8
processes per simulation, you can use
sh ./fdtd-run-pbs.sh *.fsp
5. Launch jobs by submitting to a scheduler from the GUI
This is not an officially supported feature of FDTD Solutions, but it can be configured to
work on some systems. Some of the usual features, such as being able to monitor job
progress in the job manager window, or pausing jobs, will not be possible. However, if this
can be configured for your system, it can enable you to run optimization tasks from the
head node of cluster with a scheduler.
First, it is important to be sure that you can correctly submit jobs to the scheduler form the
command line using step 4. Once that is working, there are 2 additional steps
Modify your fdtd-pbs-template.sh template or fdtd-pbs-run.sh script so that
it will wait after submission of the job until the job is completed. Some schedulers have a
simple blocking options, such as the -K option with lsf, or the "-W block=true" option
with some versions of qsub. If there is not a simple blocking option for your scheduler,
please contact Lumerical support for assistance.
Configure your resources as shown in the screenshots below. Please note that you will
likely not get any information about the job progress in the job manager. It will simply
indicate 0% complete until the job is finished. To see job progress, you will have to

2003 - 2013 Lumerical Solutions, Inc

14

Linux Installation
monitor the log files. Furthermore, you should not try to pause or terminate jobs. Job
cancellation or deletion should be done as directed by your System Administrator to
correctly terminate jobs that have been submitted to a scheduler.
Add a new Cluster resource. Don't worry about the number of Processes listed or the IP/
Hostname.

Edit the advanced options and set it up as below. Please note that we use -n 16 to use 16
processes per simulation. You can change this as desired.

2003 - 2013 Lumerical Solutions, Inc

Linux

15

It is likely that your Job Manager window will not display any progress at all until the job is
100% complete. To try and make it update progress, you can try adding the -remote flag
to the fdtd-engine. However, you may only see something like the screenshot below.
Please look at the log file from the job to monitor progress. Please do not try to use the
Quit buttons, but cancel your jobs if necessary as directed by your System Administrator.

Once everything is working, duplicate your cluster resource as many times as you have
licenses on your cluster. Your Resource Configuration Window might look like the following
if you want to run 10 concurrent simulations on your cluster.

2003 - 2013 Lumerical Solutions, Inc

16

Linux Installation

6. Launch jobs from Windows on a Linux machine

This is not an officially supported feature of FDTD Solutions, but it is possible to launch
simulation on a Linux machine from Windows. The fsp file must be saved in a network
folder that can be read and written by both the Windows and the Linux machine. Also,
there must be no spaces in the path or file name. The steps are:
a. Download and install an ssh client for Windows such as PuTTY.
b. Ensure that you can login using ssh from Windows without having to type a password.
For example, using PuTTY, you can generate a set of public/private keys. The public key
must be placed in the your home the text file $HOME/.ssh/authorized_keys on
the Linux machine.
c. Create and save a session with PuTTY. For example, if you are connecting to a Linux
machine called wkst1, you can create a PuTTY session called wkst1.
d. Add a resource and configure the basic and advanced options as shown below. Note that
the IP/Hostname is only listed as PuTTY as a reminder and the number of processes
specified is only a reminder. The key part of the setup is done in the Resource
advanced options windows. Please verify carefully that the Command to execute
displayed at the bottom of the window is correct, and note that the number of processors
to be used is actually controlled with the option -n 8 (for 8 processes) in the Extra
FDTD command line options field.

2003 - 2013 Lumerical Solutions, Inc

Linux

2003 - 2013 Lumerical Solutions, Inc

17

18

Linux Installation

e. Save the file fdtd_launch_from_windows.sh in /opt/lumerical/fdtd/bin and ensure that it is

executable.
f. Modify the lines in fdtd_launch_from_windows.sh that translate the Windows PATH to
the Linux PATH. For example, if you have access a file in Windows from \\network1
\shared\fdtd_files and on Linux from /wkst1/fdtd_files, then you need could modify the
lines to be
# replace the file path
FSPFILE=$(echo $FSPFILE | sed 's_//network1/shared_/wkst1_')
Please note that both the Windows and the Linux path should be specified with only
forward slashes ('/') because the lines immediately before this in the script automatically
convert all backslashes ('\') to forward slashes ('/').
Tips: You can also use echo to see on how your command is being interpreted by the
command line. For e.g.:
echo \\123.123.123.123\fdtd\file.fsp
g. Test the setup. We recommend first trying to use the file
fdtd_launch_from_windows.sh directly from Linux while typing a Windows style
path to the fsp file. Next, please try running a simple fsp file. (Please note that the "Run
tests" button on the resource configuration may not indicate success even if everything
is set up correctly.) Once running you should get a progress update for your simulations,
and you can right click on the job to display additional job information, as shown in the
windows below.

2003 - 2013 Lumerical Solutions, Inc

Linux

1.5

19

FAQ

1.5.1 Starting the program

If I type the "fdtd-solutions" command at the prompt Linux reports
"command not found"
This indicates that the FDTD Solutions program directory is not in your path environment
variable. You can add the following line to your .bashrc file in your home directory to fix this
problem:
export PATH=$PATH:/opt/lumerical/fdtd/bin

When I run FDTD Solutions I get an error about a missing shared

library.
FDTD Solutions depends on several Linux system libraries. It is possible that your Linux
system is missing the correct version of one or more libraries. Please contact Lumerical
support at support@lumerical.com for further assistance.
For example, the following section shows how to fix a "could not load libmat" link error.

"Could not load libmat" error

If everything was installed successfully, but upon entering fdtd-solutions in the command
line, the following error could appear:
could not load libmat
This is likely a result of missing dependencies in the requisite libraries, which can occur for
certain Linux distributions. To find the missing dependencies, use the ldd command to
print all the shared library dependencies:
ldd -v /opt/lumerical/fdtd/lib/matlab/libmat.so
You should be able to locate the missing links from the output.
As an example, libz.so is a common case of a missing dependency:
libz.so => not found
To resolve this missing dependency, locate the version of libz.so in your machine (often
located in /usr/lib64/ or /usr/lib/) and create a corresponding symbolic link. For example,
ln -s /usr/lib64/libz.so.1 /opt/lumerical/fdtd/lib/matlab/libz.
so
if your version of libz.so is called libz.so.1 and is located in the /usr/lib64/ directory. Please
note that the version and location of the libz.so library can differ between machines.
You can then use the command:
ldd -v /opt/lumerical/fdtd/lib/matlab/libmat.so
to verify that the libz.so not found error has been resolved, or if there are any other

2003 - 2013 Lumerical Solutions, Inc

20

Linux Installation
missing dependencies. Once all the dependencies are linked, you should be able to run
FDTD Solutions. Please contact support@lumerical.com if the problem persists.

1.5.2 Crashes and graphics problems

If the software crashes at startup or unexpectedly while using the layout editor, your
graphics card driver is probably to blame. Try the following steps to fix the problem:

Update graphics card drivers.

The majority of graphics problems are related to the graphics card driver. Installing the
latest drivers from the graphics card vendors website will fix most problems.
If you still have problems after installing the latest drivers, see the additional suggestions
below. Always remember to restart FDTD Solutions after making changes to the graphics
card settings.

Disable hardware acceleration

If you have the most recent graphics card drivers and are still experiencing problems,
disabling hardware OpenGL acceleration can sometimes help. Running the software with
the command fdtd-solutions-noaccel will automatically disable the acceleration.

NVIDIA graphic cards

Disabling the vertical synchronization can fix some graphics problems. The vertical
synchronization setting can be accessed through the NVIDIA control panel.

I've tried your suggestions, but there are still problems.

Please contact Lumerical support at support@lumerical.com for assistance. Please
include as much information as possible, including:
The version of your software (i.e. FDTD 7.0.1, x64 for Linux)
Your Operating System details, including name, version, x32 or x64 bit. (i.e. Windows
Vista Business x64 edition with Service Pack 2)
Screenshots of all error messages
A detailed description of exactly what steps are required to reproduce the problem
Details of your graphics card
Any other information that you think is relevant

1.5.3 Parallel engine issues

MPI and Engine versions 21
How do I specify which MPI distribution to use in the GUI? 22
How do I use my Myrinet network hardware? 22
Can I use a remote shell other than SSH to launch jobs? 23
Where can I find more information about the mpirun/mpiexec command?

23

2003 - 2013 Lumerical Solutions, Inc

Linux

21

MPI and Engine versions

Lumerical includes several versions of the parallel engine for different implementations of
MPI. The following table lists the various options. "fdtd-engine-mpich2nem" is the default
option, which should be used with the nemesis implementation of MPICH2.
Command Name

MPI
implementation

MPI library used

Suitable for

fdtd-enginempich2nem

MPICH2 (nemesis)

MPICH2 library
included with
installation

Multiprocessor
computers and
Clusters using
Ethernet network
hardware

fdtd-enginempichp4

MPICH (ch_p4)

MPICH library
included with
installation

Clusters using
Ethernet network
hardware

fdtd-enginempichshmem

MPICH (ch_shmem)

MPICH library
included with
installation

Multiprocessor
computers

fdtd-enginempichgm

MPICH-GM (ch_gm)

MPICH-GM library
included with
installation

Clusters using
Myrinet hardware

fdtd-engineimpi-lcl

Intel MPI

Intel MPI library

installed by user

Any parallel system

with Intel MPI
libraries installed

fdtd-enginehpmpi-lcl

HP-MPI

HP-MPI library
installed by user

Clusters with
Ethernet, Myrinet
and Quadrics
hardware

fdtd-engineinfinipath-lcl

Infinipath MPI

Infinipath MPI
installed by user

Clusters with
Infinipath hardware

fdtd-enginempichp4-lcl

MPICH

MPICH library
installed by user

Any parallel system

with a variant of the
MPICH libraries
installed

fdtd-enginescali-lcl

Scali MPIConnect

Scali MPIConnect
installed by user

Systems with Scalis

MPIConnect product
installed

2003 - 2013 Lumerical Solutions, Inc

22

Linux Installation
fdtd-engineompi-lcl

OpenMPI 1.3, 1.4

OpenMPI library
installed by user

Clusters using TCP,

Myrinet or
OpenFabrics
hardware

fdtd-enginempich2-lcl

MPICH2

MPICH2 library
installed by user

Any parallel system

with a variant of the
MPICH2 libraries
installed

You should select the command from the above table which matches your MPI distribution.
The fdtd-mpi-status.sh utility (in the bin directory) can be used to analyze the fdtdengine-* and determine which ones have their dependencies satisfied on your system.

How do I specify which MPI distribution to use in the GUI?

First, the desired MPI version must be installed on your system.
Determine which engine binary is
compatible with your MPI distribution. The
fdtd-mpi-status.sh utility (in the bin
directory) can be used to analyze the
engine binaries listed above to determine
which ones have their dependencies
satisfied on your system.
You must also locate the mpirun/
mpiexec executable for your MPI
distribution.
Finally, use the Resource configuration
utility to configure the simulation engine to
use the desired version of mpirun/
mpiexec and the simulation engine.

How do I use my Myrinet network hardware?

Parallel simulations can be run on clusters that use Myricoms Myrinet network hardware
by using the MPICH for Myrinet (ch_gm device) distributed by Myrinet. This is commonly
known as MPICH-GM. The installation package includes a MPICH-GM library, or you may
wish to obtain the software directly from Myricom and install it on your cluster separately.
The MPICH-GM library depends on Myrinets GM hardware drivers to also be installed on
your cluster system. These drivers are not distributed in the product installation package.
You will have to obtain them from Myrinet, if you have not done so already. Myrinet also

2003 - 2013 Lumerical Solutions, Inc

Linux

23

supports a second version of their drivers known as the MX drivers. They also have a
version of MPICH known as MPICH-MX that works with the MX drivers. If you wish to use
Myrinet MX drivers you will have to obtain both the MX drivers and MPICH-MX from
Myricom.

Can I use a remote shell other than SSH to launch jobs?

If you are using the MPICH1 MPI environment all you have to do is set the environment
variable P4_RSHCOMMAND to rsh on your cluster nodes. If you only launch jobs from
specific management nodes, you only have to set this variable on those machines.

Where can I find more information about the mpirun/mpiexec

command?
A description of the options for the mpirun/mpiexec command can be printed using the
command mpirun -h or mpiexec -h.

1.5.4 USB hardware key not found error

Problem description
The software gives a "Hardware key not found" error, and will not start.
The USB key is connected to the computer
The software ran properly in the past
You now receive a "Hardware key not found" error

Solution
The problem is that the USB hardware key daemon has stopped for some reason. You can
restart the daemon by restarting the computer, or with the following command (which
requires root access).
/etc/init.d/aksusbd restart
You can check the status of the USB hardware key daemon using the command (which
only runs properly with root access).
/etc/init.d/aksusbd status

1.5.5 Installing without administrator access

Can I install Lumerical's software on Linux without administrator
(root) access?
We do not recommend installing without administrator (root) access on Linux. When
possible, ask your system administrator to install the software. In some circumstances,
such as installing on a large, shared cluster, it may be necessary to install the software
without root access.
Note: The license manager (FlexNet) always requires administrator access.

2003 - 2013 Lumerical Solutions, Inc

24

Linux Installation

Installing without administrator access can be accomplished using the RPMs for FDTD
Solutions with the following steps.
Identify the correct rpm for your installation from the rpm_install_files directory of
the installation CD or download package. The RPM name will be something like FDTDx.y.z-n.dist.*.rpm.
Select a directory for installation, for example $HOME. Copy the rpm files to $HOME.
From the directory $HOME, unpack the rpm archives in this directory with the commands
rpm2cpio < FDTD-x.y.z-n.dist.*.rpm | cpio -i --make-directories
This will unpack the archived files into the directory $HOME/opt/lumerical/fdtd
instead of the usual /opt/lumerical/fdtd.
Add the directory $HOME/opt/lumerical/fdtd/bin to your PATH environment
variable, by adding this line to the appropriate file. The file you should modify depends on
the shell you use. For example, if you use the bash shell, you can set environment
variables by adding the above lines to .bashrc in your home directory. If you would like to
set environment variables for all users you can add the lines to /etc/profile. If you are
unsure of how to modify set environment variables in Linux, please consult your system
administrator.
export PATH=$PATH:$HOME/opt/lumerical/fdtd/bin
Run the license configuration utility
fdtd-config-license
If you get a 'could not load libmat' error, see this page: Starting the program

19

If you want to use a version of MPICH to run parallel simulations, you may need to edit
the appropriate mpirun or mpiexec file and search/replace all instances of "/opt/
lumerical/fdtd" with "$HOME/opt/lumerical/fdtd". If you selected a version of MPICH2,
this step is not required.

1.5.6 Installing multiple versions

Can I install multiple versions of Lumerical's software on the same
system?
We do not recommend installing multiple versions on the same system. The standard
installation instructions, which involve running the install.sh script, will automatically
un-install all other versions of our software before installing the new version.
If you need to install multiple versions, the install.sh script can't be used. Instead, the

2003 - 2013 Lumerical Solutions, Inc

Linux

25

RPM's must be manually installed. Use the RPM command, with the --force and --prefix
options, to install the software. Use the prefix option to specify the installation directory.
Obviously the installation directory of each version must be different. The commands will
look something like this:
cd /home/username/Desktop/FDTD_Solutions-x.y.z/
rpm_install_files/
rpm -i --force --prefix=/opt/lumerical/fdtd_xyz FDTD-x.y.z-n.
dist.*.rpm
When multiple versions are installed, users must take care to run the correct version of the
executable. In particular, it is important to ensure that the graphical CAD application runs
the matching version of the simulation engine. Use the Advanced tab of the Resource
configuration tool to specify the simulation engine location.

1.5.7 Installing on Ubuntu

Can I install Lumerical's software on Ubuntu?
Ubuntu is not an officially supported Linux distribution. The standard installation
instructions will not work, and our abilities to provide technical support for such installations
are very limited. Nonetheless, it is generally possible to install the software on Ubuntu,
with a little extra effort. The following notes describe some of the important steps required
to install on Ubuntu. The example commands assume you are trying to install FDTD
Solutions, but the same approach can be used for other Lumerical products. These
commands must be run as the root user.
Please note that these instructions are tested for Ubuntu 12.10. If you are using a
different version, the steps that are required may varies.
Ensure that the alien command is installed on your system. If you need to install this
command, use
apt-get install alien
Convert our .rpm packages to a Debian (Ubuntu) .dev package using the alien
command with the -k option. The RPM to convert are found in the installation directory in
the sub-folder rpm_install_files. For example, if you are trying to install FDTD
Solutions x64, the file will be named something like:
64 bit OS - FDTD_Solutions-x.x.x-1.rhel4.x86_64.rpm
To convert the RPM file, the use the command
alien -k FDTD_Solutions-x.x.x-1.rhel4.x86_64.rpm
This will create deb packages with the same file names but with the extension .deb
instead of .rpm.

2003 - 2013 Lumerical Solutions, Inc

26

Linux Installation
Install the resulting .deb package as usual on a Ubuntu distribution. For example, on a
64 bit system, you might do
dpkg -i FDTD_Solutions-x.x.x-1.rhel4.x86_64.deb
Note: If you don't have 32-bit libraries compatibility installed, you will need to install it using
the following command:
apt-get install ia32-libs
The software may require libraries that are not installed on all Ubuntu distributions.
When this happens, you will encounter missing dependencies error. To resolve it, first you
will need to search what package do you need, for example if you are libXss
apt-cache search libXss
The result will display some package that you can install, choose the one that fits your
need. In our example, we will want to do:
apt-get install libxss1
You should fix all the dependencies using the above 2 commands in order to be able to
install the software successfully.
On some occasions a recent Ubuntu may have a newer version of the dependencies that
we need. When this happen try to create a symbolic link, such as:
ln -s /usr/lib/x86_64-linux-gnu/libtiff.so.4 /opt/lumerical/
fdtd/lib/libtiff.so.3
Note that this solution may not work if there are major version difference between the
libraries.
Continue with the usual configuration for your system. This typically involves: configuring
the installation by running fdtd-config.sh (in /opt/lumerical/fdtd/bin) which
will add the installation directory to your PATH and configuring the license by running
fdtd-config-license.
Drivers for the HASP USB key can be downloaded directly from SafeNet Website if
necessary.
If the installation has completed successfully, and "fdtd-solutions" still fails to launch,
please see Running FDTD 19 .
Installing multiple products
When trying to install multiple products, you may see something like the following error
message:
dpkg: error processing mode_5.0.3-1.rhel5_amd64.deb (--install):
trying to overwrite '/var/hasplm/vendors/65620.xml', which is
also in package fdtd 7.0.1-1.rhel5

2003 - 2013 Lumerical Solutions, Inc

Linux

27

In this case, uninstall both products. Re-install the first product as per the above
instructions. Next, install the additional products using the force argument for the dpkg
command:
dpkg -i --force-overwrite FDTD*.deb mode_5.0.3-1.rhel5_amd64.deb
This solution assumes that you can successfully install each product individually and the
only problem is the 'trying to overwrite' file conflict problem.

1.5.8 Trial license validation

Connecting to the internet through a firewall
Many personal computers use firewall software to improve security. Most firewalls are
configured to allow all outgoing connections to the internet but to strictly control incoming
connections. The trial license validation process only requires an outgoing connection to
the internet, so normally no firewall configuration is required. If you are using very strict
firewall settings you may have to add exceptions for to your firewall settings to allow
outgoing connections. The best way to test whether your firewall is interfering with the
license validation process is to temporarily disable it. If disabling the firewall allows the
license validation process to complete, then you should re-enable the firewall and add the
following firewall exceptions:
fdtd-solutions
fdtd-engine

1.6

Next Steps
How do I run FDTD Solutions?
You can start FDTD Solutions by typing fdtd-solutions.

Is there a Getting Started Guide?

The manual can be found under the Help menu. It is highly recommended that you start by
reading the Getting Started manual, which provides a simple introduction to FDTD
Solutions.

Is there a Reference Guide?

The full Reference Guide manual can be found under the Help menu. Additional information
can be found in the online Knowledge Base.

2003 - 2013 Lumerical Solutions, Inc