Running Threaded Programs on NY Blue/P

• Introduction
• Invocation Names of the Thread-Safe Compilers
• Execution Process Modes

Introduction

For those unfamiliar with threads and pthreads it is suggested that you read this Pthreads tutorial as an introduction before reading the rest of this page.

In reading it, note that the main program counts as one of the threads.

Another reference is the Hybrid/Multi-core Programming section of the June 5, 2009 NYCCS Presentation.

Invocation Names of the Thread-Safe Compilers

If a thread-safe version of a particular compiler on NY Blue/P is offered, it's invocation name has suffix

_r

For example bgxlc_r is the thread-safe version of the unwrapped IBM C cross compiler bgxlc.

If you are unfamiliar with all the compilers (thread-safe or not) offered on NY Blue/P, see the NY Blue/P Compiler Invocation web page.

Execution Process Modes

On NY Blue/L there are two execution process modes, CO and VN.

On NY Blue/P there are three modes:

• SMP : Each compute node executes one mpi task and a maximum of four threads. There is one thread per processor.
• DUAL: Each compute node executes two mpi tasks. There is a maximum of two threads per mpi task. There is one thread per processor.
• VN: Each compute node executes four mpi tasks. There is one thread per mpi task. There is one mpi task per processor.

See the NY Blue/P Sample Batch Job Control File for an example as to how to specify the mode for your batch job; that example specifies mode VN. The default mode is SMP.

See Execution Process Modes (chapter 4) of the IBM System Blue Gene Solution: Blue Gene/P Application Development redbook for a better understanding of these modes.

Note the following excerpt:

"The choice of the node mode largely depends on the type of application and the parallel paradigm that has been implemented for a particular application. The obvious case involves applications where a hybrid paradigm between MPI and OpenMP has been implemented. In this case, it is beneficial to use the SMP node mode. If you are writing single-threaded applications, you should consider Virtual Node Mode.

I/O-intensive tasks that require a relatively large amount of data interchange between Compute Nodes benefit more by using Virtual Node Mode. Those applications that are primarily CPU bound and do not have large working memory requirements (the application gets only half of the node memory), run more quickly in Virtual Node Mode."