Running Gaussian with Linda on Ada

31-Jan-2008

Introduction

Linda will run Gaussian calculations on multiple processors and nodes while Gaussian (without Linda) is restricted to a single node. The following instructions will explain how to set up your environment and run Gaussian with Linda on ada.rice.edu.

Set up Gaussian Environment and PBS Batch Script

In order to set up the environment to run Gaussian with Linda, perform the following steps:

Change your default shell to tcsh. This is required. Instructions for changing your default shell are in our FAQ.
Linda will spawn jobs on compute nodes via SSH, so enable passwordless SSH for your account.
Create a file in your home directory called .tsnet.config with the following contents:

Tsnet.Node.lindarsharg: ssh

Write a Linda startup script such as the following linda.csh script:

#!/usr/bin/tcsh
cd /home/adauser/workingdirectory
umask 0077

setenv g03root /opt/apps/gaussian/g03-d1/     #required
source $g03root/g03/bsd/g03.login              #required

if $?PBS_NODEFILE then
foreach node ( `cat $PBS_NODEFILE | sort | uniq` )
if $?GAUSS_LFLAGS then                      #GAUSS_LFLAGS is required
    setenv GAUSS_LFLAGS "${GAUSS_LFLAGS} ${node}"
else
    setenv GAUSS_LFLAGS ${node}
endif
end
setenv GAUSS_LFLAGS "-nodelist '${GAUSS_LFLAGS}'"
endif

g03l yourlindajob.com                          #run the job

rm -rf $GAUSS_SCRDIR                           #remove scratch directory

NOTE: The $g03root/g03/bsd/g03.login script will automatically define your Gaussian scratch directory ($GAUSS_SCRDIR) to be /lustre/adauser/gauss-scrdir-jobID if you have not already defined this variable. This will create a scratch directory for every job that you are running. Due to the large size of these scratch files, it is important for you to remove your scratch files from /lustre when your job is finished, especially if the job crashes or exceeds its maximum allowed walltime before the linda.csh script is able to remove the directory for you as its last step. If you define $GAUSS_SCRDIR yourself, please write your scratch files to /lustre, not /home or /projects.

Submit your job using a batch script similar to the following:

#PBS -N Linda
#PBS -l nodes=2:ppn=4
#PBS -M riceuser@rice.edu
#PBS -S /usr/bin/tcsh
#PBS -m abe
#PBS -o /home/adauser/workingdirectory
#PBS -e /home/adauser/workingdirectory
#
echo "I ran on:"
cat $PBS_NODEFILE
#
cd /home/adauser/workingdirectory

./linda.csh #run the linda.csh script as listed above

NOTE: This example is for a job that contains 2 nodes (one master node, one slave node) with 4 processors per node.

Configuration Tips

More information about configuring Gaussian for Linda can be found on Gaussian's website. Much of this has already been done for you using the above examples. However, it is important to note the definition of some of Linda's variables as listed on Gaussian's website:

nproclinda (nprocl) - number of Linda nodes, not processors.
nprocshared (nprocs) - number of processors per node using smp.

Also it is important to set the Int=FMMNAtoms= n value to around 300 instead of the default value. The default value will cause Linda to use its linear scaling algorithm which is not a parallel calculation. This is described on Gaussian's website.

Getting Help

If you need assistance with Gaussian, please contact the Help Desk at 713-348-4357.