NWChem

NWChem is an open-source quantum chemistry code developed by the Environmental Molecular Sciences Laboratory at Pacific Northwest National Laboratory. The code is hosted on a GitHub repository and official documentation is provided.

Using NWChem at NERSC

First Option: Shifter images for release 7.2.0

The following scripts use an image created from the 7.2.0 NWChem release using the following Dockerfile to run NWChem on Perlmutter via Shifter.

As far as the execution on GPUs is concerned, the user should read the related documentation on the NWChem website https://nwchemgit.github.io/TCE.html#ccsdt-method-with-cuda

Slurm script for NWChem Shifter image on Perlmutter CPUs

```bash

!/bin/bash

SBATCH -C cpu

SBATCH -q regular

SBATCH -t 1:00:00

SBATCH -N 2

SBATCH –ntasks-per-node=64

SBATCH –cpus-per-task=2

SBATCH -J my_nwchem_job

SBATCH -o my_nwchem_job.%j.out

SBATCH -e my_nwchem_job.%j.err

SBATCH –image=ghcr.io/nwchemgit/nwchem-720.nersc.mpich4.mpi-pr:latest

export OMP_NUM_THREADS=1 export COMEX_MAX_NB_OUTSTANDING=6 export FI_CXI_RX_MATCH_MODE=hybrid export COMEX_EAGER_THRESHOLD=16384 export FI_CXI_RDZV_THRESHOLD=16384 export FI_CXI_OFLOW_BUF_COUNT=6 export MPICH_SMP_SINGLE_COPY_MODE=CMA srun -N $SLURM_NNODES –cpu-bind=cores shifter –module=mpich nwchem input.nw ```

Slurm script for NWChem Shifter image on Perlmutter GPUs

```bash

!/bin/bash

SBATCH -C gpu

SBATCH -q regular

SBATCH -t 1:00:00

SBATCH -N 2

SBATCH –ntasks-per-node=16

SBATCH –cpus-per-task=8

SBATCH –gpus-per-node=4

SBATCH –gpu-bind=none

SBATCH -J my_nwchem_job

SBATCH -o my_nwchem_job.%j.out

SBATCH -e my_nwchem_job.%j.err

SBATCH –image=ghcr.io/nwchemgit/nwchem-720.nersc.mpich4.mpi-pr:latest

export OMP_NUM_THREADS=1 export COMEX_MAX_NB_OUTSTANDING=6 export FI_CXI_RX_MATCH_MODE=hybrid export COMEX_EAGER_THRESHOLD=16384 export FI_CXI_RDZV_THRESHOLD=16384 export FI_CXI_OFLOW_BUF_COUNT=6 export MPICH_SMP_SINGLE_COPY_MODE=CMA srun -N 1 -n 2 –cpu-bind=cores shifter –module=gpu,mpich nvidia-smi srun -N $SLURM_NNODES –cpu-bind=cores shifter –module=gpu,mpich nwchem input.nw ```

Second Option: Shifter images for the master branch

The following scripts use an image created from the master branch of the NWChem GitHub repository using the following Dockerfile to run NWChem on Perlmutter via Shifter.

Slurm script for NWChem Shifter image on Perlmutter CPUs

```bash

!/bin/bash

SBATCH -C cpu

SBATCH -q regular

SBATCH -t 1:00:00

SBATCH -N 2

SBATCH –ntasks-per-node=64

SBATCH –cpus-per-task=2

SBATCH -J my_nwchem_job

SBATCH -o my_nwchem_job.%j.out

SBATCH -e my_nwchem_job.%j.err

SBATCH –image=ghcr.io/nwchemgit/nwchem-dev.nersc.mpich4.mpi-pr:latest

export OMP_NUM_THREADS=1 export COMEX_MAX_NB_OUTSTANDING=6 export FI_CXI_RX_MATCH_MODE=hybrid export COMEX_EAGER_THRESHOLD=16384 export FI_CXI_RDZV_THRESHOLD=16384 export FI_CXI_OFLOW_BUF_COUNT=6 export MPICH_SMP_SINGLE_COPY_MODE=CMA srun -N $SLURM_NNODES –cpu-bind=cores shifter –module=mpich nwchem input.nw ```

Slurm script for NWChem Shifter image on Perlmutter GPUs

```bash

!/bin/bash

SBATCH -C gpu

SBATCH -q regular

SBATCH -t 1:00:00

SBATCH -N 2

SBATCH –ntasks-per-node=16

SBATCH –cpus-per-task=8

SBATCH –gpus-per-node=4

SBATCH –gpu-bind=none

SBATCH -J my_nwchem_job

SBATCH -o my_nwchem_job.%j.out

SBATCH -e my_nwchem_job.%j.err

SBATCH –image=ghcr.io/nwchemgit/nwchem-dev.nersc.mpich4.mpi-pr:latest

export OMP_NUM_THREADS=1 export COMEX_MAX_NB_OUTSTANDING=6 export FI_CXI_RX_MATCH_MODE=hybrid export COMEX_EAGER_THRESHOLD=16384 export FI_CXI_RDZV_THRESHOLD=16384 export FI_CXI_OFLOW_BUF_COUNT=6 export MPICH_SMP_SINGLE_COPY_MODE=CMA srun -N 1 -n 2 –cpu-bind=cores shifter –module=gpu,mpich nvidia-smi srun -N $SLURM_NNODES –cpu-bind=cores shifter –module=gpu,mpich nwchem input.nw ```