Site Navigation
General Information
NY Blue/P Libraries
Note: There is an applications web page listing applications available on NY Blue/P beyond the libraries described here.
So also see that applications page.
Engineering and Scientific Subroutine Library (ESSL)
IBM's ESSL for SLES on POWER (v4.4) that supports Blue Gene/P has been installed
on the
Front-End Node. ESSL provides over 150 math subroutines (Linear Algebra, Matrix operations,
Sorting and Searching, FFT, Random Number Generator, etc.) that have been specifically tuned
for performance on BG/P.
ESSL subroutines are for Serial code. Parallel ESSL (PESSL) in not available on NY Blue/P.
The ESSL routines have been installed on the Front-End host under /opt/ibmmath/lib and the header files under /opt/ibmmath/include. Thus, to link your code to the ESSL library that contains single-threaded routines you should use (Fortran):
>mpixlf90_r example.f -L/opt/ibmmath/lib -lesslbg
The ESSL SMP Library (contains multi-threaded routines) is libesslsmpbg.a in the same directory. To compile SMP code, you must specify the -qsmp compiler option.
Information on the ESSL subroutines is available on the command line of the Front-End host using the installed man pages. For example, to obtain a description of the subroutine sasum, type: man sasum
Additional Information:
- The ESSL Guide and Reference is available (in pdf) at:
http://publib.boulder.ibm.com/epubs/pdf/am501403.pdf - For more information about ESSL, refer to the following web address:
http://www-03.ibm.com/systems/p/software/essl.html.
Mathematical Acceleration Subsystem (MASS)
Mathematical Acceleration Subsystem (MASS) for Blue Gene/P consists of libraries of mathematical intrinsic functions tuned specifically for optimum performance on Blue Gene/P. These libraries offer improved performance over the standard mathematical library routines, are thread-safe, and support compilations in C, C++, and Fortran applications.
MASS v4.4 (scalar and vector) libraries have been installed on the Front-End node under
/opt/ibmcmp/xlmass/bg/4.4. The subdirectory bglib contains the
"cross-compiled"
versions of the libraries. The lib and lib64 subdirectories contain
the "native" versions
(for executables that will run on the Front-End node, not on the Blue Gene Compute
Nodes).
Add linker option -Wl,--allow-multiple-definition to allow multiple definitions for the math routines.
Additional Information:
- Mathematical Acceleration Subsystem
- Mathematical Acceleration Subsystem Features and Benefits
- Using the MASS Libraries on Blue Gene/L and Blue Gene/P
- MASS Library slides of Blue Gene/P System and Optimization Tips by Bob Walkup of IBM
- MASS C/C++ function prototypes for Blue Gene/L and Blue Gene/P
- MASS Fortran Interface blocks for Blue Gene/L and Blue Gene/P
- Performance information for the MASS libraries on Blue Gene/L and Blue Gene/P
- Accuracy information for the MASS libraries for Blue Gene/L and Blue Gene/P
Basic Linear Algebra Communication Subprograms (BLACS)
The Basic Linear Algebra Communication Subprograms (BLACS) provide a linear algebra
oriented message passing interface.
The archives have been compiled from the version 1.1 source code downloaded from
http://www.netlib.org/blacs/mpiblacs.tgz
and http://www.netlib.org/blacs/mpiblacs-patch03.tgz
For Fortran, to link with the BLACS libraries add the following at your link step:
/bgsys/apps/blacs-1.1/BLACS/LIB/blacsF77init_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacs_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacsF77init_MPI-BGP-0.a
For C, to link with the BLACS libraries add the following at your link step:
/bgsys/apps/blacs-1.1/BLACS/LIB/blacsCinit_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacs_MPI-BGP-0.a /bgsys/apps/blacs-1.1/BLACS/LIB/blacsCinit_MPI-BGP-0.a
Linear Algebra Package (LAPACK)
The LAPACK library contains routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems. The associated matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are also provided, as are related computations such as reordering of the Schur factorizations and estimating condition numbers. Dense and banded matrices are handled, but not general sparse matrices. In all areas, similar functionality is provided for real and complex matrices, in both single and double precision.
On NY Blue/P it was built from the NETLIB Fortran source code.
To link against the lapack library, add the following at the link step:
-L/bgsys/apps/lapack-3.2/lapack-3.2 -llapack_BGP.a
Additional Information:
- Many LAPACK routines invoke BLAS routines. To link against the LAPACK library and the BLAS routines in ESSL (using the XL Fortran compiler):
-L/bgsys/apps/lapack-3.2/lapack-3.2
-L/opt/ibmmath/lib
-llapack_BGP -lesslbg - To link against the LAPACK library and the BLAS routines in the LAPACK distribution (using the XL Fortran compiler):
-L/bgsys/apps/lapack-3.2/lapack-3.2
-llapack_BGP -lblas_BGP - The ESSl library contains BLAS routines, is tuned specifically for Blue Gene, and can help significantly in improving performance.
- But be aware that ESSL does not contain all of the standard BLAS routines.
- Using the GNU fortran, or GNU or XL C compiler, to link against the LAPACK library and the BLAS routines in the LAPACK distribution :
-L/bgsys/apps/lapack-3.2/lapack-3.2
-llapack_BGP -lblas_BGP
-L/opt/ibmcmp/xlf/bg/11.1/bglib
-lxlf90_r -lxl -lxlfmath
-L/opt/ibmcmp/xlsmp/bg/1.7/bglib
-lxlomp_ser
-lm - Using the GNU fortran, or GNU or XL C compiler, to link against the LAPACK library, and the BLAS routines
in the LAPACK distribution, then against the BLAS routines in ESSL:
-L/bgsys/apps/lapack-3.2/lapack-3.2 -L/opt/ibmmath/lib
-llapack_BGP -lblas_BGP -lesslbg
-L/opt/ibmcmp/xlf/bg/11.1/bglib
-lxlf90_r -lxl -lxlfmath
-L/opt/ibmcmp/xlsmp/bg/1.7/bglib
-lxlomp_ser
-lm - Information about LAPACK including a hyperlink to LAPACK USER'S GUIDE
Scalable Linear Algebra Package (SCALAPACK)
ScaLAPACK is a library of high-performance linear algebra routines for distributed-memory message-passing MIMD computers and networks of workstations supporting MPI . It is a continuation of the LAPACK project, which designed and produced analogous software for workstations, vector supercomputers, and shared-memory parallel computers. Both libraries contain routines for solving systems of linear equations, least squares problems, and eigenvalue problems. The goals of both projects are efficiency (to run as fast as possible), scalability (as the problem size and number of processors grow), reliability (including error bounds), portability (across all important parallel machines), flexibility (so users can construct new routines from well-designed parts), and ease of use (by making the interface to LAPACK and ScaLAPACK look as similar as possible).
The library is currently written in Fortran (with the exception of a few symmetric eigenproblem auxiliary routines written in C to exploit IEEE arithmetic) in a Single Program Multiple Data (SPMD) style using explicit message passing for interprocessor communication.
On NY Blue/P it was built from the NETLIB Fortran source code at http://www.netlib.org/scalapack.
Additional Information:
- To use SCALAPACK, one must link the BLACS and LAPACK libraries. Additionally, many SCALAPACK routines invoke BLAS routines.
- To link against the SCALAPACK library and the netlib BLAS routines in the lapack distribution:
mpixlf77_r -o myprog myprog.c -L/bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 -lscalapack -L/bgsys/apps/blacs-1.1/BLACS/LIB -lblacsF77init_MPI-BGP-0 -lblacs_MPI-BGP-0 -lblacsF77init_MPI-BGP-0 -L/bgsys/apps/lapack-3.2/lapack-3.2 -llapack_BGP -lblas_BGP
- To link against the SCALAPACK library and the BLAS routines in ESSL:
mpixlf77_r -o myprog myprog.c -L/bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 -lscalapack -L/bgsys/apps/blacs-1.1/BLACS/LIB -lblacsF77init_MPI-BGP-0 -lblacs_MPI-BGP-0 -lblacsF77init_MPI-BGP-0 -L/bgsys/apps/lapack-3.2/lapack-3.2 -llapack_BGP -L/opt/ibmmath/lib -lesslbg
- To link against the SCALAPACK library, and the BLAS routines in the lapack distribution then the BLAS in ESSL, using the XL C compiler,
or using the GNU C or Fortran compiler (this example is using the GNU C compiler).
Replace blacsF77init_MPI-BGP-0 with blacsCinit_MPI-BGP-0 if using a C compiler:mpicc -o myprog myprog.c -L/bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 -lscalapack -L/bgsys/apps/blacs-1.1/BLACS/LIB -lblacsCinit_MPI-BGP-0 -lblacs_MPI-BGP-0 -lblacsCinit_MPI-BGP-0 -L/bgsys/apps/lapack-3.2/lapack-3.2 -L/opt/ibmmath/lib -llapack_BGP -lblas_BGP -lesslbg -L/opt/ibmcmp/xlf/bg/11.1/bglib -lxlf90_r -lxl -lxlfmath -L/opt/ibmcmp/xlsmp/bg/1.7/bglib -lxlomp_ser -lm
- You may find after linking to the three blacs libraries above that you have to add another -lblacs_MPI-BGP-0
- If you are calling a scalapack routine from a C code, in the C code you need to prototype the fortran scalapack routine it is calling. See pages 4 to 7 of the IBM XL C/C++ Programming Guide , especially the example on pages 7 and 8.
- The above links will link to scalapack built with the -O3 -qstrict -qarch=450d -qtune=450
compiler option. - To instead link to scalapack built with the -O4 -qarch=450d -qtune=450
compiler option, link as above except replace all occurrences of O3qstrict with O4 - Scalapack User Guide
- Scalapack Homepage
- Also see the README file in /bgsys/apps/scalapack-1.8.0/O3qstrict/scalapack-1.8.0 or /bgsys/apps/scalapack-1.8.0/O4/scalapack-1.8.0.
