Compiling OpenMP Code at NERSC
OpenMP Flags for Different PrgEnv Modules
Depending on the compiler suite used, a different flag is needed to use OpenMP:
| Vendor | PrgEnv | Language(s) | OpenMP flag |
|---|---|---|---|
| GNU | PrgEnv-gnu |
C/C++/Fortran | -fopenmp |
| NVIDIA | PrgEnv-nvidia |
C/C++/Fortran | -mp |
| HPE | PrgEnv-cray |
C/C++ | -fopenmp |
| HPE | PrgEnv-cray |
Fortran | -homp |
GPU Offload
!!! warning “GPU target must be set”
Either load the cudatoolkit and craype-accel-nvidia80
modules or use the “target” flags below.
PrgEnv-nvidia
Use the -mp=gpu option.
Details of OpenMP support and programming recommendation are available at in the OpenMP section of the Perlmutter Readiness Guide.
PrgEnv-cray
| Language(s) | OpenMP flag | target |
|---|---|---|
| C/C++ | -fopenmp |
-fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64 -march=sm_80 |
| Fortran | -homp -hnoacc |
-haccel=nvidia80 |
!!! bug “linker warning” The following linker warning can be safely ignored:
```
warning: linking module '@': linking module flags 'SDK Version': IDs have conflicting values ('[2 x i32] [i32 11, i32 1]' from /opt/cray/pe/cce/14.0.1/cce-clang/x86_64/lib/libcrayomptarget-nvptx-sm_80.bc with '[2 x i32] [i32 11, i32 5]' from ../benchmarks/babelStream/main.cpp) [-Wlinker-warnings]
```
!!! bug “Target selection error with Fortran”
Target selection error With PrgEnv-cray Fortran applications with
OpenMP offload can be avoided by
1. unset the `CRAYPE_ACCEL_TARGET` variable
2. manually specify the target architecture `-homp -hnoacc -haccel=nvidia80`
!!! tip “mixed C/C++ and Fortran applications”
Use -fopenmp -fopenmp-targets=nvptx64 -Xopenmp-target=nvptx64 -march=sm_80
to manually specify the target architecture for C/C++ code.
PrgEnv-gnu
PrgEnv-gnu is not recommended for OpenMP offloading.
To access it:
ml use /global/cfs/cdirs/m1759/wwei/Modules/perlmutter/modulefiles/
ml gcc/13.1.0
An example to compile your code:
gcc -Ofast -fopenmp -foffload=nvptx-none="-Ofast -lm -latomic -misa=sm_80" main.c -o main