Julia¶

Julia is a high-level, high-performance dynamic programming language for technical computing. It has syntax that is familiar to users of many other technical computing environments. Designed at MIT to tackle large-scale partial-differential equation simulation and distributed linear algebra, Julia features a robust ecosystem of tools for optimization, statistics, parallel programming, and data visualization. Julia is actively developed by teams at MIT and in industry, along with hundreds of domain-expert scientists and programmers from around the world.

Using Julia at NERSC¶

Multiple Julia versions are available at NERSC. You can see which ones we currently support by running module avail julia which will print out a list of julia versions -- for example:

$ module avail julia

--------------------------- NERSC-provided Software ----------------------------
   julia/1.8.5    julia/1.9.2    julia/1.9.3    julia/1.9.4 (D)

  Where:
   D:  Default Module

...

You can then load a specific version of julia by specifying julia/<version> -- for example:

module load julia/1.9.4

Or you can load the default version available by omitting the version -- eg:

module load julia

NERSC-Specific Preferences¶

Our Julia modules also configure the JULIA_LOAD_PATH to include NERSC-specific prferences. Note that the Julia modules append the Julia preferences path to the end of JULIA_LOAD_PATH. Therefore, you can always override NERSC-specific preferences using a LocalPreferences.toml in a project at an earlier location in the JULIA_LOAD_PATH.

Using NERSC-specific settings with your own version of Julia¶

Sometimes it is desirable to bring your own version of Julia. For example, you might have a well-established bootstrapping process on new systems. It is still possible to use your own version of Julia in combination with the NERSC-specific settings. It is for this reason that we do not provide a global JULIA_DEPOT as any packages therein would be tied to specific Julia versions. Instead we recommend that you use our Juliaup install that is included with every Julia module.

For example, this installs julia version 1.10.2 to $SCRATCH and makes use of NERSC-specific settings.

$ module load julia
$ export JULIAUP_DEPOT_PATH=$SCRATCH/julia
$ juliaup add 1.10.2
Installing Julia 1.10.2+0.x64.linux.gnu
$ export PATH=$JULIAUP_DEPOT_PATH/juliaup/julia-1.10.2+0.x64.linux.gnu/bin:$PATH
$ which julia
/pscratch/sd/e/elvis/julia/juliaup/julia-1.10.2+0.x64.linux.gnu/bin/julia

Upon new logins, you can use this installation with the following.

module load julia # for setting JULIA_LOAD_PATH to load NERSC-specific perferences
export JULIAUP_DEPOT_PATH=$SCRATCH/julia 
export PATH=$JULIAUP_DEPOT_PATH/juliaup/julia-1.10.2+0.x64.linux.gnu/bin:$PATH

Using Julia with MPI and CUDA¶

As described in the previous section, we do not pre-install packages like MPI.jl and CUDA.jl (many users prefer specific versions, making a central installation unmaintainable). Instead MPI.jl and CUDA.jl will detect any relevant system settings when firstly installed.

However it is always good to check that these settings have been correctly applied. For this reason, many packages that rely on interactions with the system software will provide a versioninfo() function.

For example,

julia> using MPI

julia> MPI.versioninfo()
MPIPreferences:
  binary:  system
  abi:     MPICH
  libmpi:  libmpi_gnu_123.so
  mpiexec: srun

Package versions
  MPI.jl:             0.20.19
  MPIPreferences.jl:  0.1.10

Library information:
  libmpi:  libmpi_gnu_123.so
  libmpi dlpath:  /opt/cray/pe/lib64/libmpi_gnu_123.so
  MPI version:  3.1.0
  Library version:
    MPI VERSION    : CRAY MPICH version 8.1.28.29 (ANL base 3.4a2)
    MPI BUILD INFO : Wed Nov 15 20:57 2023 (git hash 1cde46f)

Indicates that CRAY MPICH is in use by MPI.jl -- which is exactly what we want!

Similarly for CUDA.jl:

julia> using CUDA

julia> CUDA.versioninfo()
CUDA runtime 12.2, local installation
CUDA driver 12.3
NVIDIA driver 525.105.17, originally for CUDA 12.2

CUDA libraries:
- CUBLAS: 12.2.1
- CURAND: 10.3.3
- CUFFT: 11.0.8
- CUSOLVER: 11.5.0
- CUSPARSE: 12.1.1
- CUPTI: 20.0.0
- NVML: 12.0.0+525.105.17

Julia packages:
- CUDA: 5.1.2
- CUDA_Driver_jll: 0.7.0+1
- CUDA_Runtime_jll: 0.10.1+0
- CUDA_Runtime_Discovery: 0.2.3

Toolchain:
- Julia: 1.9.4
- LLVM: 14.0.6

Preferences:
- CUDA_Runtime_jll.version: 12.2
- CUDA_Runtime_jll.local: true

1 device:
  0: NVIDIA A100-PCIE-40GB (sm_80, 17.308 GiB / 40.000 GiB available)

Indicates that a the locally-installed CUDA runtime is in use -- also exactly what we want.

Running MPI.jl Programs¶

You can use Slurm's srun as usual:

srun -n N julia prog.jl

or you can use the mpiexecjl wrapper provided by Julia:

mpiexecjl -n N julia prog.jl

Note that, in order to use Julia's mpiexecjl, you need to install it first:

julia> using MPI

julia> MPI.install_mpiexecjl()
[ Info: Installing `mpiexecjl` to `/global/homes/e/elvis/.julia/bin`...
[ Info: Done!

Then add ~/.julia/bin to your PATH with export PATH=~/.julia/bin:$PATH.

Developing Julia Code for GPUs¶

Please take a look at our Perlmutter transition guide's section on Julia.