# Hands-on #3

## Singularity and MPI

Singularity has a good integration with MPI. With Singularity, the MPI usage model is to call ‘mpirun’ from outside the container, and reference the container from the ‘mpirun’ command. It has to be installed on all of the nodes that are going to use it. A centralized filesystem works best for this. The mpirun command communicates with the binary via the Process Management Interface (PMI) used by OpenMPI. When MPI run gets executed it forks an Orted process that launches Singularity as a container process. The MPI application within the container is linked to the MPI runtime libraries within the container. The MPI runtime libraries then connect and communicate back to the Orted process via a universal PMI.

![Singularity MPI application process](https://3073326825-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-L9PtzjOz4HPAc9i-Kmp%2F-L9Pu2krKYi6YW2a--1m%2F-L9Pu3t1PHBl3ZpEK-rL%2Fsingularity-mpi.png?generation=1523017340769442\&alt=media)

> MPI application process for Singularity

Usage would look like this:

```
mpirun -np 20 singularity exec container.img /path/to/contained_mpi_prog
```

> **Warning** MPI must be newer or equal to the version inside the container

## Building LAMMPS MPI Singularity configuration file

Create/use the lammps template configuration file `lammps-mpi.cfg` in the `/home` directory.

```
BootStrap: yum
OSVersion: 7
MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/os/$basearch/
Include: yum

%setup
  cd $SINGULARITY_ROOTFS/opt
  wget http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz
  wget http://lammps.sandia.gov/tars/lammps-stable.tar.gz

%post
  yum -y groupinstall "Development Tools"

  mkdir -p /opt/mpich
  cd /opt/mpich
  tar xf ../mpich-3.2.tar.gz --strip-components 1

  ./configure --prefix=/usr/local |& tee log.configure
  make -j |& tee log.make
  make install |& tee log.make_install

  mkdir -p /opt/lammps
  cd /opt/lammps
  tar xf ../lammps-stable.tar.gz --strip-components 1

  cd src
  make yes-granular |& tee log.make_yes_granular
  make -j mpi |& tee log.make_mpi

%runscript
  /opt/lammps/src/lmp_mpi "$@"
```

## Creating the container image

* Create a container called `lammps-mpi.img` and modify the size to 2048 MiB.

```
singularity create --size 2048 lammps-mpi.img`
```

* Bootstrap the lammps configuration file created in the previous step.

```
sudo singularity build lammps-mpi.img lammps-mpi.cfg
```

> **Note** Bootstrapping step requires `root` access.

## Running the container image

To execute the container

```
mpirun -np 4 singularity run ~/lammps-mpi.img -i in.granregion.mixer
```

To run LAMMPS on Cambridge HPC

```
srun -Atraining -pbiocloud-normal --reservation=singularity --mpi=pmi2 -N1 -n24 singularity run ~/lammps.img -i ~/in.mixer
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://kks32-courses.gitbook.io/hpc-containers/singularity/building-a-container-configuration-file/hands-on-3.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.