Skip to content

Commit

Permalink
cleanenv and no-mount clarifications
Browse files Browse the repository at this point in the history
  • Loading branch information
mambelli committed Mar 1, 2024
1 parent 65cd103 commit 0067476
Show file tree
Hide file tree
Showing 4 changed files with 51 additions and 35 deletions.
27 changes: 18 additions & 9 deletions _episodes/04-building-containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,26 +57,34 @@ To initialize an interactive session use the `shell` command. And to write files
The `--cleanenv` option is added to make sure that the host environment is not affecting the container.
Finally, the installation of new components will require superuser access inside the container, so use also the `--fakeroot` option, unless you are already root also outside.
```bash
apptainer shell --cleanenv --writable --fakeroot myAlma9
apptainer shell --writable --cleanenv --fakeroot myAlma9
Apptainer> whoami
```
~~~
root
~~~
{: .output}
`--cleanenv` clears the environment. It has been added to make sure that the eventual setting of
Variables like PYTHONPATH or PYTHONHOME can affect the Python execution inside the container.
variables on the host is not affecting the container.
Variables like PYTHONPATH or PYTHONHOME are affecting the Python execution inside the container.
A corrupted Python environment could cause errors like "ModuleNotFoundError: No module named 'encodings'"
> ## Apptainer environment
> [Environment variables in Linux](https://www.geeksforgeeks.org/environment-variables-in-linux-unix/)
> are dynamic values that can affect programs and containers.
> You can use the environment to pass variables to a container.
> Apptainer by default preserves most of the outside environment inside the container
> but has many options to control that.
> You can clear the environment with the `--cleanenv` option, you can set variables with `--env`.
> See the [Apptianer manual](https://apptainer.org/docs/user/main/environment_and_metadata.html))
> You can clear the environment with the `--cleanenv` option and you can set variables with `--env`.
> See the [Apptianer manual](https://apptainer.org/docs/user/main/environment_and_metadata.html)
> for more options and details.
>
> PYTHONPATH and PYTHONHOME affect the Python execution, but other variables could affect other programs so,
> if you don't care about the outside environment, you can add `--cleanenv`
> all the times you start a container (`apptainer shell`, `exec` and `instance start` commands).
{: .callout}
Depending on the Apptainer/Singularity installation (privileged or unprivileged) and the version, you may have some requirements, like the `fakeroot` utility or `newuidmap` and `newgidmap`.

Depending on the Apptainer/Singularity installation (privileged or unprivileged) and the version,
you may have some requirements, like the `fakeroot` utility or `newuidmap` and `newgidmap`.
If you get an error when using `--fakeroot` have a look at the [fakeroot documentation](https://apptainer.org/docs/user/main/fakeroot.html).
> ## `--fakeroot` is not root
> ATTENTION! [`--fakeroot`](https://apptainer.org/docs/user/main/fakeroot.html) allows you to be root inside a container that you own but is not changing who you are outside.
Expand All @@ -88,7 +96,7 @@ First, we need to enable the CRB (Code Ready Builder/PowerTools) and
[EPEL](https://docs.fedoraproject.org/en-US/epel/) (Extra Packages for Enterprise Linux) repositories
and to install the development tools (remember that in this interactive session we are superuser):
```bash
Apptainer> dnf install 'dnf-command(config-manager)'
Apptainer> dnf install 'dnf-command(config-manager)' # this installs dnf-plugins-core
Apptainer> dnf config-manager --set-enabled crb
Apptainer> dnf install epel-release
Apptainer> dnf groupinstall 'Development Tools'
Expand Down Expand Up @@ -118,11 +126,12 @@ Apptainer> make
Apptainer> exit
```

> ## Installing Pythia from the package manager
> Pythia is now distributed as RPM in EPEL so you can use instead (optionally also pythia8-devel):
> ## Installing the Pythia binary package
> Pythia is now distributed as RPM in EPEL, so you can use this easier alternative to install it:
> ```bash
> Apptainer> dnf install pythia8
> Apptainer> dnf install python3-pythia8
> Apptainer> dnf install pythia8-devel # optional, if you need also the development libraries to compile
> ```
{: .callout}
Expand All @@ -132,7 +141,7 @@ few steps. Let's use the Python interface:
```bash
apptainer shell myAlma9
# These first 2 lines are necessary only if Pythia was installed from source
# These first 2 lines are necessary only if Pythia was installed from source, not if you used the binary package
Apptainer> export PYTHONPATH=/opt/pythia/pythia8/lib:$PYTHONPATH
Apptainer> export LD_LIBRARY_PATH=/opt/pythia/pythia8/lib:$LD_LIBRARY_PATH
Apptainer> python3
Expand Down
11 changes: 8 additions & 3 deletions _episodes/07-file-sharing.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ And the same happens with permissions and ownership for files in bind directorie

## Bind paths included by default

For each container executed, Apptainer binds automatically some directories by default, and other defined
by the system admin in the Apptainer configuration. By default, Apptainer binds:
For each container executed,
[Apptainer binds automatically some directories by default](https://apptainer.org/docs/user/main/bind_paths_and_mounts.html#disabling-system-binds),
and other defined by the system admin in the Apptainer configuration. By default, Apptainer binds:
* The user's home directory ($HOME)
* The current directory when the container is executed ($PWD)
* System-defined paths: `/tmp`, `/proc`, `/dev`, etc.
Expand Down Expand Up @@ -68,7 +69,7 @@ mounts automatically your `$HOME` inside the container.
Try this time with
```bash
apptainer shell --no-mount home rootInUbuntu.sif
apptainer shell --no-mount home,cwd rootInUbuntu.sif
```
and you will notice that `$HOME` is not mounted anymore
```bash
Expand All @@ -79,6 +80,10 @@ ls: cannot access '/home/myuser': No such file or directory
~~~
{: .output}

Note how we disabled both `home` and `cwd` (current working directory). This because if you are running the apptainer
command from your home directory, even if you use `--no-mount home` the home directory may still be mounted
because it is also your current directory.

## User-defined bind paths

Apptainer provides mechanisms to specify additional binds when executing a container via command-line
Expand Down
3 changes: 2 additions & 1 deletion _episodes/08-instances.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,8 @@ apptainer instance start --no-mount tmp --cleanenv basicServer.sif myWebService
Reminder from the previous chapter: with `--no-mount tmp` we are asking Apptainer to NOT bind `/tmp` from the host
to the instance (it is mounted by default), we use instead an isolated `/tmp` inside the instance where index.html has
been copied.
And with `--cleanenv` we clear the environment.
And with `--cleanenv` we clear the environment. It is not always necessary but it prevents interferences
from the host environment (see ).


You can confirm in the terminal that the web service is up using `curl` as
Expand Down
45 changes: 23 additions & 22 deletions _extras/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,13 @@ The main course includes only the first 8 episodes
The bonus episode 9 "Building and deploying an Apptainer container to Github Packages"
is optional, also to avoid circular dependencies with Git and CI/CD.

Some more advanced material is interspersed throughout the episodes.
Some more advanced material is interspersed throughout the episodes,
normally as a Tip (callout box with the Pin logo).
This helps understanding how containers and Apptainer work,
but are not strictly necessary for the basic use of Apptainer.
Future releases my separate this out in highlighted sections
to allow both a peruse of the course for a deeper understanding
and a quick run through to learn just the bare minimum.
Future releases may separate this out in highlighted sections with a dedicated callout type
to allow both a peruse of the course for a deeper understanding and a
quick run through to learn just the bare minimum.

## Overall

Expand All @@ -22,30 +23,30 @@ and a quick run through to learn just the bare minimum.
### Links used for explanation

Episode 1 (Introduction)
* [https://hsf-training.github.io/hsf-training-singularity-webpage/01-introduction/index.html](https://hsf-training.github.io/hsf-training-singularity-webpage/01-introduction/index.html)
* [https://www.docker.com/resources/what-container/](https://www.docker.com/resources/what-container/)
* [https://q15928.github.io/2021/01/09/container-101/](https://q15928.github.io/2021/01/09/container-101/)
* [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459)
* [https://docs.sylabs.io/guides/2.6/user-guide/singularity_flow.html](https://docs.sylabs.io/guides/2.6/user-guide/singularity_flow.html)
* [https://docs.sylabs.io/guides/2.6/user-guide/_images/flow.png](https://docs.sylabs.io/guides/2.6/user-guide/_images/flow.png)
* [https://en.wikipedia.org/wiki/Singularity_%28software%29#History](https://en.wikipedia.org/wiki/Singularity_%28software%29#History)
* [https://apptainer.org/documentation/](https://apptainer.org/documentation/)
* [https://apptainer.org/docs/user/latest/](https://apptainer.org/docs/user/latest/)
* [https://github.com/hsf-training/hsf-training-singularity-webpage](https://github.com/hsf-training/hsf-training-singularity-webpage)
* <https://hsf-training.github.io/hsf-training-singularity-webpage/01-introduction/index.html>
* <https://www.docker.com/resources/what-container/>
* <https://q15928.github.io/2021/01/09/container-101/>
* <https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459>
* <https://docs.sylabs.io/guides/2.6/user-guide/singularity_flow.html>
* <https://docs.sylabs.io/guides/2.6/user-guide/_images/flow.png>
* <https://en.wikipedia.org/wiki/Singularity_%28software%29#History>
* <https://apptainer.org/documentation/>
* <https://apptainer.org/docs/user/latest/>
* <https://github.com/hsf-training/hsf-training-singularity-webpage>

Episode 2
* Terminal
* [https://hsf-training.github.io/hsf-training-singularity-webpage/02-running-containers/index.html](https://hsf-training.github.io/hsf-training-singularity-webpage/02-running-containers/index.html)
* [https://circleci.com/blog/docker-image-vs-container/](https://circleci.com/blog/docker-image-vs-container/)
* [https://singularityhub.github.io/library-api/#/?id=library-api](https://singularityhub.github.io/library-api/#/?id=library-api)
* [https://docs.google.com/presentation/d/1GMDyEHw02AqFnD8IV9YxeilQZ0E7zQuovtvaS0hgE0s/edit#slide=id.g2bb13c81de3_0_295](https://docs.google.com/presentation/d/1GMDyEHw02AqFnD8IV9YxeilQZ0E7zQuovtvaS0hgE0s/edit#slide=id.g2bb13c81de3_0_295)
* [https://root.cern/](https://root.cern/)
* [https://apptainer.org/docs/user/latest/](https://apptainer.org/docs/user/latest/)
* <https://hsf-training.github.io/hsf-training-singularity-webpage/02-running-containers/index.html>
* <https://circleci.com/blog/docker-image-vs-container/>
* <https://singularityhub.github.io/library-api/#/?id=library-api>
* <https://docs.google.com/presentation/d/1GMDyEHw02AqFnD8IV9YxeilQZ0E7zQuovtvaS0hgE0s/edit#slide=id.g2bb13c81de3_0_295>
* <https://root.cern/>
* <https://apptainer.org/docs/user/latest/>

Episode 4
* Terminal
* [https://hsf-training.github.io/hsf-training-singularity-webpage/04-building-containers/index.html](https://hsf-training.github.io/hsf-training-singularity-webpage/04-building-containers/index.html)
* [https://circleci.com/blog/docker-image-vs-container/](https://circleci.com/blog/docker-image-vs-container/)
* <https://hsf-training.github.io/hsf-training-singularity-webpage/04-building-containers/index.html>
* <https://circleci.com/blog/docker-image-vs-container/>
* <https://github.com/scikit-hep/uproot5>
* <https://uproot.readthedocs.io/en/latest/>
* <https://apptainer.org/docs/user/latest/cli/apptainer_build.html>
Expand Down

0 comments on commit 0067476

Please sign in to comment.