Skip to content

Shifter for Beginners Tutorial

Consider podman-hpc before Shifter

While we still support Shifter, NERSC eventually plans to phase it out in favor of podman-hpc. If you are a beginner, you should consider starting with podman-hpc rather than Shifter. Please take a look at the podman-hpc beginner tutorial.

This tutorial is meant to demonstrate all the steps you need to build a Docker container, upload it, and run it as a Shifter image at NERSC.

  1. Install Docker on your local system
  2. Set up your free account on Docker Hub
  3. Build your hello-world Docker image on your local machine
  4. Run your hello-world Docker container on your local machine
  5. Push your Docker image to the Docker Hub registry
  6. Pull your Docker image to NERSC via Shifter
  7. Inspect your Shifter image at NERSC
  8. Submit an interactive job using your Shifter image at NERSC
  9. Submit a batch job using your Shifter image at NERSC
  10. Go forth and be productive with Shifter!

Install Docker on your local system

For now, NERSC does not offer users any place where they can build their own Docker containers. (We are working on this!) This means that users will need to build their images on their own systems before pulling them to NERSC.

Please download and install the Docker client for your appropriate operating system. Once installed, make sure the Docker daemon (or client) is running. Note that this will likely require you to have admin permissions on your system.

Set up your free account on Docker Hub

Since you'll need to push your image to Docker Hub, it's easiest to set up your account and choose your username before you build your image. You may find it easier to use your NERSC username if possible. When you have your Docker Hub username, you can use it in the next few steps. Note that there are many other registries out there, but we're using Docker Hub here since it's currently a very commonly used option.

Build your hello-world Docker image on your local machine

Let's create our first image that will print hello world from inside a container! when we run it. Please make yourself a directory called docker and inside of it, paste the following contents into a file called Dockerfile

FROM ubuntu:22.04

ENTRYPOINT echo "hello from inside a container!"

What do these commands do?

FROM ubutu:22.04 downloads an existing Docker image that already contains the Ubuntu operating system (note that we use a fixed version instead of ubuntu:latest which would be a moving target). In general it's a good idea to use some of the many existing building blocks within the Docker (and wider container) ecosystem.

ENTRYPOINT echo "hello from inside a container!" will execute your echo statement to print hello when the container runs. Note that Docker will always execute the ENTRYPOINT, which we'll see is different than in Shifter.

Now let's build your container. To make things easier later, let's prefix it with your Docker Hub username. We'll assume that your Docker Hub and NERSC username is elvis, so please substitute your own username where it appears. If your Docker Hub username is different than your NERSC username, make sure you name your image with your Docker Hub username. This will make it easy to push to your Docker Hub account when you are ready.

docker build -t elvis/hello-world:1.0 .

What does this do? It simultaneously builds your Docker image based on what you specified in your Dockerfile and also names and tags your image with your Docker Hub username elvis, the name you specified for your image hello-world, and the tag you gave your image 1.0. Now that it's appropriately named and tagged, it will be easier to push to Docker Hub when you're ready.

Other useful Dockerfile commands:

RUN will run a command inside the container environment to modify its contents. This is usually used for installing and configuring software.

COPY will copy files into your Docker image.

WORKDIR will establish or change the current working directory during the Docker build. It will also affect the default directory that used when you run a container with this image.

ENV can be used for setting or appending to existing environment variables.

Building an image on Perlmutter

You can build an image on a Perlmutter login node using Podman.
However, do note that if you intend to use the image with Shifter, you will still have to upload it to a registry to make it available to Shifter.

Run your hello-world Docker container on your local machine

Your Docker image has been built and is now ready to run as a container. An instantiation of an image is a container, so that's why we call it a container in the context of running.

elvis$ docker run --rm elvis/hello-world:1.0
hello from inside a container!

Here you are running your Docker image. The --rm flag instructs the Docker daemon to clean up the Docker container after you are done running it. You don't need this, but if you don't use it, you'll have a lot of unused Docker containers accumulating on your system that you will eventually need to clean up.

Congratulations, you've now built and run your first Docker container.

Push your Docker image to the Docker Hub registry

In order to get your Docker image onto NERSC systems, you will need to push your image to public registry such as Docker Hub. Here we are assuming you have already set up your Docker Hub account. You will need to log in to the Docker public registry via docker login and then you will be prompted for your username and password. You will typically only need to log in once and Docker will store your login credentials for later.

docker login

Once you've done that, you can push your image via

docker push elvis/hello-world:1.0

This will push your image to your Docker Hub account. Please note that if you are using the free account, all Dockerfiles and images are public.

Pull your Docker image onto NERSC via Shifter

Once your image is pushed to Docker Hub, we can pull the image at NERSC using Shifter. Shifter is configured to pull images directly from Docker Hub via the shifterimg pull command, which will automatically convert your Docker image into Shifter format.

shifterimg pull elvis/hello-world:1.0

Inspect your Shifter image at NERSC

You can run your Shifter container interactively on a login node. The Shifter --image flag is used to select your container and is followed by arguments that will be run inside the container. If we specify /bin/bash, we will start a bash shell inside our container.

elvis@login07:~> shifter --image=elvis/hello-world:1.0 /bin/bash
elvis@login07:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
elvis@login07:~$ exit
exit

For comparison, note that NERSC's operating system is SUSE Linux, so this is additional verification that we are in fact executing this command inside our container.

Once inside your image, you can verify the location of your software, the environment variables you have set, etc. You can even run it on our login nodes, although we kindly request that you adhere to our general login node use policies. When you are done inspecting your image, type exit to leave your container.

Shifter users should note that their prompt will not change once inside a Shifter container. If you can't remember whether you have typed exit and are still inside your Shifter container, there are several ways to check:

  • env | grep "SHIFTER_RUNTIME"- this will return 1 if you are inside a Shifter container
  • cat /etc/os-release | grep "SUSE"- this will check to see if the OS is SUSE, which is NERSC's OS. Note that this will not return anything if you are inside a Shifter container.
  • echo $LD_LIBRARY_PATH | grep "udiImage"- this will look to see if there are any Shifter modules in your LD_LIBRARY_PATH. Note this will not return anything if you are outside a Shifter container.

Submit an interactive job using your Shifter image at NERSC

To use your Shifter image inside an interactive job, you can submit as you normally would with the addition of the image you would like to use

salloc -N 1 -t 60 -C cpu -q interactive --image=elvis/hello-world:1.0

This will make your image available to your job. However, to actually use it, you'll need to issue the shifter command. Anything that follows this command will be executed inside your container. One difference from our Docker example is that you must ask Shifter to use your ENTRYPOINT with the --entrypoint flag. Let's try it:

elvis@nid00283:~> srun shifter --entrypoint
hello from inside a container!

Submit a batch job using your Shifter image at NERSC

Submitting a batch job with a Shifter image is very similar. You'll need to request the image you need in the SBATCH directives. Here is an example jobscript we'll call submit-shifter.slurm.

#!/bin/bash

#SBATCH --image=elvis/hello-world:1.0
#SBATCH --qos=debug
#SBATCH --constraint=cpu
#SBATCH -t 00:02:00
#SBATCH -N 1
#SBATCH -o output.o%j

srun shifter --entrypoint

We can submit this job with sbatch submit-shifter.slurm.

If we check the output, we'll see

elvis@login07:> cat output.o50623860
hello from inside a container!

Let's say we don't want to use the ENTRYPOINT we have specified in our image. Instead let's go ahead and print our Ubuntu information like we did above.

#!/bin/bash

#SBATCH --image=elvis/hello-world:1.0
#SBATCH --qos=debug
#SBATCH --constraint=cpu
#SBATCH -t 00:02:00
#SBATCH -N 1
#SBATCH -o output.o%j

srun shifter cat /etc/lsb-release

The output of this job looks like

elvis@login07:> cat output.o50626055
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"

This should look familiar-- it should be same output you saw in our Inspect your Shifter image at NERSC. You can substitute in any command you like after the shifter command. This command will run inside your container. Keep in mind that you may need to COPY or bind-mount a directory if you would like a script to be accessible inside your container.

Go forth and be productive with Shifter

Now that you have finished this tutorial, you might like some more detailed information about the different ways to use Shifter at NERSC. Here are several resources: