Using Singularity for Unity container support
The Unity cluster supports containers with Singularity to enable customers to bring their own portable software stack. Singularity allows the customer to develop and customize their workflow without the need for admin intervention (unlike other technologies such as Docker which require elevated privileges). Singularity allows customers to package a complete environment into a single file known as a container image, which includes system libraries, custom user software, configuration files, and most other dependencies. A Singularity image file can be easily copied and runs on Linux-based computing platform.
Getting Started
Before you can use Singularity you must load the singularity module. This loads the latest version of singularity in you path. (for more on modules see Unity Walk-Through ) Also NOTE: Currently singularity commands ONLY WORK ON COMPUTE NODES.
module load singularity
Quick run
Run your first container:
Singularity using the run command will pull down an image if it is not already cached and run the default script:
$ singularity run library://sylabsed/examples/lolcow
INFO: Downloading library image
79.9MiB / 79.9MiB [================================================================] 100 % 18.1 MiB/s 0s
____________________
< Beware of Bigfoot! >
--------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Eventually if you use many containers your /home/username may fill up as it is limited to 100GB. Check on your image usage with the singularity cache command
$ singularity cache list
or
$ singularity cache list -v
To clear your cache and make space. It will ask for confirmation [Y/n]
$ singularity cache clean
Containers (and where to get them)
Using pre-built containers:
Singularity supports Singularity and Docker OCI images. For more information check the Singularity Documentation page.
This includes support for the OCI container spec from these sites:
As with any software you download from the internet, get your software from reputable sources.
Examples:
Try the latest version of python from DockerHub. (Note the URL without a hostname Singularity defaults to DockerHub. Also the username is missing since it is the official python. The URL contains ONLY the project:tag or project:version) You can do a two step process and pull down the image first. It will then be converted to local singularity image.
$ singularity pull docker://python:latest
$ singularity exec docker://python:latest python
Try Pytorch from NVIDIA NGC. (URL is docker://hostname/username/project:tag)
$ singularity pull docker://nvcr.io/nvidia/pytorch:21.09-py3
$ singularity exec docker://nvcr.io/nvidia/pytorch:21.09-py3 python3
For NVIDIA GPU support please add --nvto your request.
$ singularity pull docker://nvcr.io/nvidia/pytorch:21.09-py3
$ singularity exec --nv docker://nvcr.io/nvidia/pytorch:21.09-py3 python3
Or load from a local image on the filesystem:
$ singularity exec --nv mylocal.sif executable
Drop into a shell within the image. You can see all of the files that make up the image.
$ singularity shell docker://python:latest
INFO: Using cached SIF image
Singularity> cat /etc/issue.net
Debian GNU/Linux 11
Security
Filesystem
The container is limited on what directories it can access. The container and it's applications can access the following directories and their sub directories:
- /home
- /fs/project
- /fs/byo
More information
Sites
Syslabs Singularity User Documentation: https://sylabs.io/guides/latest/user-guide/