BlueGene Homepage BlueGene Homepage BlueGene Homepage BlueGene Homepage

Back to Batch Job Submission

How to Specify Nodes Desired in Your Batch Script

  1. Do not specify a partition when submitting a batch job on NY Blue/P.

    Instead, specify the size of the dynamic partition that LoadLeveler should allocate for your batch job. Specify between 64 and 2048 nodes.

    For example for 64 nodes, you would include the following statement in your batch script (instead of specifying #@ bg_partition):

    # @ bg_size = 64

    # @ arguments = -mode VN -exe /path/to/your/executable/my.exe

    Recall that there are four processors per node on NY Blue/P (not two as on NY Blue/L). This job would use 256 processors. See execution process modes .

  2. If you wish to use fewer processors than the allocated partition possesses, use -np. For example, if you wanted to use only 4 of 256 allocated processors (wasteful unless necessary for some reason), you would include these two statements in your batch script:

    # @ bg_size = 64

    # @ arguments = -mode VN -np 4 -exe /path/to/your/executable/my.exe

  3. Important Note: You must use

    # @ bg_size

    and not -np to specify the number of nodes to allocate.

    For instance the following job which attempts to allocate 512 processors using -np rather than allocating nodes using # @ bg_size will abort:

    # @ arguments = -np 512 -exe /path/to/your/executable/my.exe

  4. In testing we noticed for example that when 1536 nodes were requested, 2048 nodes were instead allocated. Note the following excerpts from the Loadleveler documentation indicating that your job could be allocated more nodes than the number of nodes you requested:

    bg_size is an integer indicating the size of the job in units of compute nodes... The size of the allocated shape will be no smaller than the requested size and as close to the request size as possible... Not all values given for bg_size are representable. For example, consider an 8x4x4 Blue Gene system in units of base partitions and a requested bg_size of 5632 (equivalent to 11 base partitions). Since 11 is a prime number, it cannot be decomposed. Futhermore, it is greater than any one dimension of the system. In this case, a 3x4x1 partition is allocated, since it is the smallest number of base partitions larger than the requested size.

Back to Batch Job Submission

This site maintained by: bgwebmaster@bnl.gov

One of ten national laboratories overseen and primarily funded by the Office of Science of the U.S. Department of Energy (DOE), Brookhaven National Laboratory conducts research in the physical, biomedical, and environmental sciences, as well as in energy technologies and national security. Brookhaven Lab also builds and operates major scientific facilities available to university, industry and government researchers. Brookhaven is operated and managed for DOE's Office of Science by Brookhaven Science Associates, a limited-liability company founded by Stony Brook University, the largest academic user of Laboratory facilities, and Battelle, a nonprofit, applied science and technology organization.
Privacy and Security Notice