From 0067476ca1e93194b7bb6f5d5c650754f07ac75c Mon Sep 17 00:00:00 2001 From: Marco Mambelli Date: Fri, 1 Mar 2024 14:25:14 -0600 Subject: [PATCH] cleanenv and no-mount clarifications --- _episodes/04-building-containers.md | 27 +++++++++++------ _episodes/07-file-sharing.md | 11 +++++-- _episodes/08-instances.md | 3 +- _extras/guide.md | 45 +++++++++++++++-------------- 4 files changed, 51 insertions(+), 35 deletions(-) diff --git a/_episodes/04-building-containers.md b/_episodes/04-building-containers.md index 4d8fbe0..e6b9efd 100644 --- a/_episodes/04-building-containers.md +++ b/_episodes/04-building-containers.md @@ -57,7 +57,7 @@ To initialize an interactive session use the `shell` command. And to write files The `--cleanenv` option is added to make sure that the host environment is not affecting the container. Finally, the installation of new components will require superuser access inside the container, so use also the `--fakeroot` option, unless you are already root also outside. ```bash -apptainer shell --cleanenv --writable --fakeroot myAlma9 +apptainer shell --writable --cleanenv --fakeroot myAlma9 Apptainer> whoami ``` ~~~ @@ -65,18 +65,26 @@ root ~~~ {: .output} `--cleanenv` clears the environment. It has been added to make sure that the eventual setting of -Variables like PYTHONPATH or PYTHONHOME can affect the Python execution inside the container. +variables on the host is not affecting the container. +Variables like PYTHONPATH or PYTHONHOME are affecting the Python execution inside the container. A corrupted Python environment could cause errors like "ModuleNotFoundError: No module named 'encodings'" > ## Apptainer environment > [Environment variables in Linux](https://www.geeksforgeeks.org/environment-variables-in-linux-unix/) > are dynamic values that can affect programs and containers. +> You can use the environment to pass variables to a container. > Apptainer by default preserves most of the outside environment inside the container > but has many options to control that. -> You can clear the environment with the `--cleanenv` option, you can set variables with `--env`. -> See the [Apptianer manual](https://apptainer.org/docs/user/main/environment_and_metadata.html)) +> You can clear the environment with the `--cleanenv` option and you can set variables with `--env`. +> See the [Apptianer manual](https://apptainer.org/docs/user/main/environment_and_metadata.html) > for more options and details. +> +> PYTHONPATH and PYTHONHOME affect the Python execution, but other variables could affect other programs so, +> if you don't care about the outside environment, you can add `--cleanenv` +> all the times you start a container (`apptainer shell`, `exec` and `instance start` commands). {: .callout} -Depending on the Apptainer/Singularity installation (privileged or unprivileged) and the version, you may have some requirements, like the `fakeroot` utility or `newuidmap` and `newgidmap`. + +Depending on the Apptainer/Singularity installation (privileged or unprivileged) and the version, +you may have some requirements, like the `fakeroot` utility or `newuidmap` and `newgidmap`. If you get an error when using `--fakeroot` have a look at the [fakeroot documentation](https://apptainer.org/docs/user/main/fakeroot.html). > ## `--fakeroot` is not root > ATTENTION! [`--fakeroot`](https://apptainer.org/docs/user/main/fakeroot.html) allows you to be root inside a container that you own but is not changing who you are outside. @@ -88,7 +96,7 @@ First, we need to enable the CRB (Code Ready Builder/PowerTools) and [EPEL](https://docs.fedoraproject.org/en-US/epel/) (Extra Packages for Enterprise Linux) repositories and to install the development tools (remember that in this interactive session we are superuser): ```bash -Apptainer> dnf install 'dnf-command(config-manager)' +Apptainer> dnf install 'dnf-command(config-manager)' # this installs dnf-plugins-core Apptainer> dnf config-manager --set-enabled crb Apptainer> dnf install epel-release Apptainer> dnf groupinstall 'Development Tools' @@ -118,11 +126,12 @@ Apptainer> make Apptainer> exit ``` -> ## Installing Pythia from the package manager -> Pythia is now distributed as RPM in EPEL so you can use instead (optionally also pythia8-devel): +> ## Installing the Pythia binary package +> Pythia is now distributed as RPM in EPEL, so you can use this easier alternative to install it: > ```bash > Apptainer> dnf install pythia8 > Apptainer> dnf install python3-pythia8 +> Apptainer> dnf install pythia8-devel # optional, if you need also the development libraries to compile > ``` {: .callout} @@ -132,7 +141,7 @@ few steps. Let's use the Python interface: ```bash apptainer shell myAlma9 -# These first 2 lines are necessary only if Pythia was installed from source +# These first 2 lines are necessary only if Pythia was installed from source, not if you used the binary package Apptainer> export PYTHONPATH=/opt/pythia/pythia8/lib:$PYTHONPATH Apptainer> export LD_LIBRARY_PATH=/opt/pythia/pythia8/lib:$LD_LIBRARY_PATH Apptainer> python3 diff --git a/_episodes/07-file-sharing.md b/_episodes/07-file-sharing.md index 337e6c8..fe059b8 100644 --- a/_episodes/07-file-sharing.md +++ b/_episodes/07-file-sharing.md @@ -29,8 +29,9 @@ And the same happens with permissions and ownership for files in bind directorie ## Bind paths included by default -For each container executed, Apptainer binds automatically some directories by default, and other defined -by the system admin in the Apptainer configuration. By default, Apptainer binds: +For each container executed, +[Apptainer binds automatically some directories by default](https://apptainer.org/docs/user/main/bind_paths_and_mounts.html#disabling-system-binds), +and other defined by the system admin in the Apptainer configuration. By default, Apptainer binds: * The user's home directory ($HOME) * The current directory when the container is executed ($PWD) * System-defined paths: `/tmp`, `/proc`, `/dev`, etc. @@ -68,7 +69,7 @@ mounts automatically your `$HOME` inside the container. Try this time with ```bash -apptainer shell --no-mount home rootInUbuntu.sif +apptainer shell --no-mount home,cwd rootInUbuntu.sif ``` and you will notice that `$HOME` is not mounted anymore ```bash @@ -79,6 +80,10 @@ ls: cannot access '/home/myuser': No such file or directory ~~~ {: .output} +Note how we disabled both `home` and `cwd` (current working directory). This because if you are running the apptainer +command from your home directory, even if you use `--no-mount home` the home directory may still be mounted +because it is also your current directory. + ## User-defined bind paths Apptainer provides mechanisms to specify additional binds when executing a container via command-line diff --git a/_episodes/08-instances.md b/_episodes/08-instances.md index 0b7f58b..0ed3465 100644 --- a/_episodes/08-instances.md +++ b/_episodes/08-instances.md @@ -135,7 +135,8 @@ apptainer instance start --no-mount tmp --cleanenv basicServer.sif myWebService Reminder from the previous chapter: with `--no-mount tmp` we are asking Apptainer to NOT bind `/tmp` from the host to the instance (it is mounted by default), we use instead an isolated `/tmp` inside the instance where index.html has been copied. -And with `--cleanenv` we clear the environment. +And with `--cleanenv` we clear the environment. It is not always necessary but it prevents interferences +from the host environment (see ). You can confirm in the terminal that the web service is up using `curl` as diff --git a/_extras/guide.md b/_extras/guide.md index 3a39bbc..558f177 100644 --- a/_extras/guide.md +++ b/_extras/guide.md @@ -7,12 +7,13 @@ The main course includes only the first 8 episodes The bonus episode 9 "Building and deploying an Apptainer container to Github Packages" is optional, also to avoid circular dependencies with Git and CI/CD. -Some more advanced material is interspersed throughout the episodes. +Some more advanced material is interspersed throughout the episodes, +normally as a Tip (callout box with the Pin logo). This helps understanding how containers and Apptainer work, but are not strictly necessary for the basic use of Apptainer. -Future releases my separate this out in highlighted sections -to allow both a peruse of the course for a deeper understanding -and a quick run through to learn just the bare minimum. +Future releases may separate this out in highlighted sections with a dedicated callout type +to allow both a peruse of the course for a deeper understanding and a +quick run through to learn just the bare minimum. ## Overall @@ -22,30 +23,30 @@ and a quick run through to learn just the bare minimum. ### Links used for explanation Episode 1 (Introduction) -* [https://hsf-training.github.io/hsf-training-singularity-webpage/01-introduction/index.html](https://hsf-training.github.io/hsf-training-singularity-webpage/01-introduction/index.html) -* [https://www.docker.com/resources/what-container/](https://www.docker.com/resources/what-container/) -* [https://q15928.github.io/2021/01/09/container-101/](https://q15928.github.io/2021/01/09/container-101/) -* [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459) -* [https://docs.sylabs.io/guides/2.6/user-guide/singularity_flow.html](https://docs.sylabs.io/guides/2.6/user-guide/singularity_flow.html) -* [https://docs.sylabs.io/guides/2.6/user-guide/_images/flow.png](https://docs.sylabs.io/guides/2.6/user-guide/_images/flow.png) -* [https://en.wikipedia.org/wiki/Singularity_%28software%29#History](https://en.wikipedia.org/wiki/Singularity_%28software%29#History) -* [https://apptainer.org/documentation/](https://apptainer.org/documentation/) -* [https://apptainer.org/docs/user/latest/](https://apptainer.org/docs/user/latest/) -* [https://github.com/hsf-training/hsf-training-singularity-webpage](https://github.com/hsf-training/hsf-training-singularity-webpage) +* +* +* +* +* +* +* +* +* +* Episode 2 * Terminal -* [https://hsf-training.github.io/hsf-training-singularity-webpage/02-running-containers/index.html](https://hsf-training.github.io/hsf-training-singularity-webpage/02-running-containers/index.html) -* [https://circleci.com/blog/docker-image-vs-container/](https://circleci.com/blog/docker-image-vs-container/) -* [https://singularityhub.github.io/library-api/#/?id=library-api](https://singularityhub.github.io/library-api/#/?id=library-api) -* [https://docs.google.com/presentation/d/1GMDyEHw02AqFnD8IV9YxeilQZ0E7zQuovtvaS0hgE0s/edit#slide=id.g2bb13c81de3_0_295](https://docs.google.com/presentation/d/1GMDyEHw02AqFnD8IV9YxeilQZ0E7zQuovtvaS0hgE0s/edit#slide=id.g2bb13c81de3_0_295) -* [https://root.cern/](https://root.cern/) -* [https://apptainer.org/docs/user/latest/](https://apptainer.org/docs/user/latest/) +* +* +* +* +* +* Episode 4 * Terminal -* [https://hsf-training.github.io/hsf-training-singularity-webpage/04-building-containers/index.html](https://hsf-training.github.io/hsf-training-singularity-webpage/04-building-containers/index.html) -* [https://circleci.com/blog/docker-image-vs-container/](https://circleci.com/blog/docker-image-vs-container/) +* +* * * *