LSF Autosubmission or How can I send large numbers of jobs to LSF?

Author: Michael Reuter

LSF on RCAS is a wonderful tool for processing many jobs at once. By using the standard PHOBOS logon

to the RCAS nodes you can already use it. See George's webpage. I created, about a year ago, a method to send

large numbers of jobs to any LSF queue (exception being phcas_hi which only allows five jobs). This document will

outline the steps to setup this large autosubmission system.

1. You must setup George's Nasty LSF Hack.

2. Create a .forward file in your home directory that contains the line username@rmail.rhic.bnl.gov. This will ensure that the

    email the job sends when it finishes will be available to be read (otherwise is gets sent to the submitting node).

2. In your .cshrc or .login (which ever you prefer), set up an environmental variable called LSFLOGDIR. I point mine to

    a directory in my $SCR directory because the LSF log files can pile up quickly. You will need to source whichever file

    you put the environmental variable in.

3. In your home directory, create the following directory structure:

            lsf, lsf/lsfcontrol, lsf/submit, lsf/submitted

4. From my lsf directory (~reuter/lsf/lsfcontrol) copy autosubmit.pl to your lsf/lsfcontrol directory

5. Create scripts for running your Phat or ROOT jobs. Examples can be found in the directories inside ~reuter/lsf/submitted.

    I deviate somewhat from what George put on his web page by not calling the evals to the setup. By logging onto a node

    this already is taken care of for you (thanks to George).

6. Place the scripts into your ~/lsf/submit directory.

7. Goto ~/lsf/lsfcontrol and execute autosubmit.pl (you may have to type ./autosubmit.pl).

8. Choose the corresponding queue that you wish to submit to from the list and the script does the rest.

What happens is it takes the scripts from the submit directory, moves them to the submitted directory and executes them

through LSF. This is to ensure that if you want to submit more scripts you don't resend to ones that are already executing. Also

if you edit the scripts inside the submit directory (this is especially true for emacs) remove any backup files. They will also be

sent by the submission script. You can check to see if your jobs are running by using the command bjobs. When the job

completes, it will send you an email and write a log file to wherever $LSFLOGDIR is pointing.