Note: There is an applications web page listing applications available on NY Blue/P beyond the libraries described here.

So also see that applications page.

IBM's ESSL for SLES on POWER (v4.4) that supports Blue Gene/P has been installed
on the

Front-End Node. ESSL provides over 150 math subroutines (Linear Algebra, Matrix operations,
Sorting and Searching, FFT, Random Number Generator, etc.) that have been specifically tuned
for performance on BG/P.

ESSL subroutines are for Serial code. Parallel ESSL (PESSL) in not available on NY Blue/P.

The ESSL routines have been installed on the Front-End host under **/opt/ibmmath/lib**
and the header files under **/opt/ibmmath/include**. Thus, to link your
code to the ESSL library that contains single-threaded routines you should use (Fortran):

>mpixlf90_r example.f -L/opt/ibmmath/lib -lesslbg

The ESSL SMP Library (contains multi-threaded routines) is **libesslsmpbg.a** in the same directory.
To compile SMP code, you must specify the **-qsmp** compiler option.

Information on the ESSL subroutines is available on the command line of the
Front-End host using the
installed man pages. For example, to obtain a description of
the subroutine *sasum*, type: ** man sasum **

Additional Information:

- The ESSL
**Guide and Reference**is available (in**pdf**) at:

http://publib.boulder.ibm.com/epubs/pdf/am501403.pdf - For more information about ESSL, refer to the following web address:

http://www-03.ibm.com/systems/p/software/essl.html.

Mathematical Acceleration Subsystem (MASS) for Blue Gene/P consists of libraries of mathematical intrinsic functions tuned specifically for optimum performance on Blue Gene/P. These libraries offer improved performance over the standard mathematical library routines, are thread-safe, and support compilations in C, C++, and Fortran applications.

MASS v4.4 (scalar and vector) libraries have been installed on the Front-End node under

**/opt/ibmcmp/xlmass/bg/4.4**. The subdirectory **bglib** contains the
"cross-compiled"

versions of the libraries. The **lib** and **lib64** subdirectories contain
the "native" versions

(for executables that will run on the Front-End node, not on the Blue Gene Compute

Nodes).

Add linker option **-Wl,--allow-multiple-definition** to allow multiple definitions for the math routines.

Additional Information:

- Mathematical Acceleration Subsystem
- Mathematical Acceleration Subsystem Features and Benefits
- Using the MASS Libraries on Blue Gene/L and Blue Gene/P
- MASS Library slides of Blue Gene/P System and Optimization Tips by Bob Walkup of IBM
- MASS C/C++ function prototypes for Blue Gene/L and Blue Gene/P
- MASS Fortran Interface blocks for Blue Gene/L and Blue Gene/P
- Performance information for the MASS libraries on Blue Gene/L and Blue Gene/P
- Accuracy information for the MASS libraries for Blue Gene/L and Blue Gene/P

The Basic Linear Algebra Communication Subprograms (BLACS) provide a linear algebra

oriented message passing interface.
The archives have been compiled from the version 1.1 source code downloaded from
http://www.netlib.org/blacs/mpiblacs.tgz
and http://www.netlib.org/blacs/mpiblacs-patch03.tgz

For Fortran, to link with the BLACS libraries add the following at your link step:

/bgsys/apps/blacs-1.1/BLACS/LIB/blacsF77init_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacs_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacsF77init_MPI-BGP-0.a

For C, to link with the BLACS libraries add the following at your link step:

/bgsys/apps/blacs-1.1/BLACS/LIB/blacsCinit_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacs_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacsCinit_MPI-BGP-0.a

The LAPACK library contains routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.

On NY Blue/P it was built from the NETLIB Fortran source code.

To link against the lapack library, add the following at the link step:

**-L/bgsys/apps/lapack-3.2/lapack-3.2 -llapack_BGP.a**

Additional Information:

- Many LAPACK routines invoke BLAS routines. To link against the LAPACK library and the BLAS routines in ESSL (using the XL Fortran compiler):

**-L/bgsys/apps/lapack-3.2/lapack-3.2****-L/opt/ibmmath/lib****-llapack_BGP -lesslbg** - To link against the LAPACK library and the BLAS routines in the LAPACK distribution (using the XL Fortran compiler):

**-L/bgsys/apps/lapack-3.2/lapack-3.2****-llapack_BGP -lblas_BGP** - The ESSl library contains BLAS routines, is tuned specifically for Blue Gene, and can help significantly in improving performance.
- But be aware that ESSL does not contain all of the standard BLAS routines.
- Using the GNU fortran, or GNU or XL C compiler, to link against the LAPACK library and the BLAS routines in the LAPACK distribution :

**-L/bgsys/apps/lapack-3.2/lapack-3.2****-llapack_BGP -lblas_BGP****-L/opt/ibmcmp/xlf/bg/11.1/bglib****-lxlf90_r -lxl -lxlfmath****-L/opt/ibmcmp/xlsmp/bg/1.7/bglib****-lxlomp_ser****-lm** - Using the GNU fortran, or GNU or XL C compiler, to link against the LAPACK library, and the BLAS routines
in the LAPACK distribution, then against the BLAS routines in ESSL:
**-L/bgsys/apps/lapack-3.2/lapack-3.2 -L/opt/ibmmath/lib****-llapack_BGP -lblas_BGP -lesslbg****-L/opt/ibmcmp/xlf/bg/11.1/bglib****-lxlf90_r -lxl -lxlfmath****-L/opt/ibmcmp/xlsmp/bg/1.7/bglib****-lxlomp_ser****-lm** - Information about LAPACK including a hyperlink to LAPACK USER'S GUIDE

ScaLAPACK is a library of high-performance linear algebra routines for distributed-memory message-passing MIMD computers and networks of workstations supporting MPI . It is a continuation of the LAPACK project, which designed and produced analogous software for workstations, vector supercomputers, and shared-memory parallel computers. Both libraries contain routines for solving systems of linear equations, least squares problems, and eigenvalue problems. The goals of both projects are efficiency (to run as fast as possible), scalability (as the problem size and number of processors grow), reliability (including error bounds), portability (across all important parallel machines), flexibility (so users can construct new routines from well-designed parts), and ease of use (by making the interface to LAPACK and ScaLAPACK look as similar as possible).

The library is currently written in Fortran (with the exception of a few symmetric eigenproblem auxiliary routines written in C to exploit IEEE arithmetic) in a Single Program Multiple Data (SPMD) style using explicit message passing for interprocessor communication.

On NY Blue/P it was built from the NETLIB Fortran source code at http://www.netlib.org/scalapack.

Additional Information:

- To use SCALAPACK, one must link the BLACS and LAPACK libraries. Additionally, many SCALAPACK routines invoke BLAS routines.
- To link against the SCALAPACK library and the netlib BLAS routines in the lapack distribution:
mpixlf77_r -o myprog myprog.c -L/bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 -lscalapack -L/bgsys/apps/blacs-1.1/BLACS/LIB -lblacsF77init_MPI-BGP-0 -lblacs_MPI-BGP-0 -lblacsF77init_MPI-BGP-0 -L/bgsys/apps/lapack-3.2/lapack-3.2 -llapack_BGP -lblas_BGP

- To link against the SCALAPACK library and the BLAS routines in ESSL:
mpixlf77_r -o myprog myprog.c -L/bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 -lscalapack -L/bgsys/apps/blacs-1.1/BLACS/LIB -lblacsF77init_MPI-BGP-0 -lblacs_MPI-BGP-0 -lblacsF77init_MPI-BGP-0 -L/bgsys/apps/lapack-3.2/lapack-3.2 -llapack_BGP -L/opt/ibmmath/lib -lesslbg

- To link against the SCALAPACK library, and the BLAS routines in the lapack distribution then the BLAS in ESSL, using the XL C compiler,
or using the GNU C or Fortran compiler (this example is using the GNU C compiler).

Replace**blacsF77init_MPI-BGP-0**with**blacsCinit_MPI-BGP-0**if using a C compiler:mpicc -o myprog myprog.c -L/bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 -lscalapack -L/bgsys/apps/blacs-1.1/BLACS/LIB -lblacsCinit_MPI-BGP-0 -lblacs_MPI-BGP-0 -lblacsCinit_MPI-BGP-0 -L/bgsys/apps/lapack-3.2/lapack-3.2 -L/opt/ibmmath/lib -llapack_BGP -lblas_BGP -lesslbg -L/opt/ibmcmp/xlf/bg/11.1/bglib -lxlf90_r -lxl -lxlfmath -L/opt/ibmcmp/xlsmp/bg/1.7/bglib -lxlomp_ser -lm

- You may find after linking to the three blacs libraries above that you have to add another
**-lblacs_MPI-BGP-0** - If you are calling a scalapack routine from a C code, in the C code you need to prototype the fortran scalapack routine it is calling. See pages 4 to 7 of the IBM XL C/C++ Programming Guide , especially the example on pages 7 and 8.
- The above links will link to scalapack built with the -O3 -qstrict -qarch=450d -qtune=450

compiler option. - To instead link to scalapack built with the -O4 -qarch=450d -qtune=450

compiler option, link as above except replace all occurrences of O3qstrict with**O4** - Scalapack User Guide
- Scalapack Homepage
- Also see the README file in /bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 or /bgsys/apps/scalapack-1.8.0/O4/scalapack-1.8.0.