Can someone please share a Singularity container with Ubuntu 22.04 GNOME GUI desktop and TigerVNC?

[Heng-Zhou]

I am trying to build a Singularity container which has a Ubuntu 22.04 OS, a GNOME GUI desktop, and TigerVNC for connection from MobaXterm. But it is too hard for me and I have devoted too much time to it. Can you please share one with me if you have? A Docker container is also OK if it can be converted to Singularity. Due to the configuration of target server that runs the container, OCI, non-SUID, and cgroups v2 mode are not supported. The version of Singularity in the server is most up-to-date, however.

Thank you.

[dtrudg]

Hi @Heng-Zhou

I just wanted to mention that running recent GNOME desktop environments, with VNC, in a container is extremely complex. This is because of:

• GNOME's tight integrations with components of a full Linux distribution such as systemd and dbus, which are not generally run inside a container, and can be very difficult to pass-through or successfully run inside a container.
• GNOME's requirement for GLX support in the Xserver, which is not supported by all VNC servers.

While some people (myself included in a previous job) have succeeded in setting up containers running a recent GNOME desktop, they often need to be quite specific to the exact OS on the host, and how the host is configured. Hopefully this thread will be seen by someone who has a generic solution, or a solution for an environment that is similar to yours.

It may be worthwhile loooking into running an alternative, lighter weight, desktop environment. I know that people have had more success with e.g. Xfce and MATE running inside containers.

Finally, GNOME is moving away from supporting X for displays in future versions (https://www.phoronix.com/news/GNOME-MR-Drop-X11-Session). This would completely remove the ability to run a full GNOME desktop under existing VNC servers, and may be another reason to look into alternatives if your project is long-term and may wish to move to e.g. Ubuntu 26.04 when that is released in a couple of years. I am not aware of a good solution for running a desktop in a container with wayland (which is what GNOME is moving to from X).

[Heng-Zhou]

While some people (myself included in a previous job) have succeeded in setting up containers running a recent GNOME desktop, they often need to be quite specific to the exact OS on the host, and how the host is configured.

My target host is RedHat 8.8 (Ootpa). I don't need a generic solution. If that is the OS on the host where you successfully set up the GNOME container, can you please share the previous work of yours with me?

GNOME's tight integrations with components of a full Linux distribution such as systemd and dbus, which are not generally run inside a container,

WSL does not have systemd in the initial installation either. So, the Singularity container sounds very much like the situation of WSL. I have successfully set up Ubuntu 22.04 + GNOME + TigerVNC on WSL using systemd, so I hope my existing work could help.

Finally, GNOME is moving away from supporting X for displays in future versions (https://www.phoronix.com/news/GNOME-MR-Drop-X11-Session). This would completely remove the ability to run a full GNOME desktop under existing VNC servers,

This is a bit off-topic, but I'm confused by the paragraph. I remember that the very purpose of Singularity is to build an environment that contains all the dependences that are independent of the host OS, no matter what. For example, if the host OS does not use systemd, I can run systemd in container and enter its name space. If the host OS is using Wayland, I can start X server inside the container so that the machine seems to be running on an OS supporting X. So, if I build the container correctly, I should not have to worry about X being deprecated. Am I right?

[dtrudg]

My target host is RedHat 8.8 (Ootpa). I don't need a generic solution. If that is the OS on the host where you successfully set up the GNOME container, can you please share the previous work of yours with me?

I'm afraid not. I don't have access to the solution because I left that job some time ago, and it was specific to my previous employers RHEL7 system. It also used NICE DCV for graphics acceleration (which is a commercial product).

GNOME's tight integrations with components of a full Linux distribution such as systemd and dbus, which are not generally run inside a container,

WSL does not have systemd in the initial installation either. So, the Singularity container sounds very much like the situation of WSL. I have successfully set up Ubuntu 22.04 + GNOME + and TigerVNC on WSL using systemd, so I hope my existing work could help.

It's quite likely that this will be useful knowledge, yes.

Finally, GNOME is moving away from supporting X for displays in future versions (https://www.phoronix.com/news/GNOME-MR-Drop-X11-Session). This would completely remove the ability to run a full GNOME desktop under existing VNC servers,

This is a bit off-topic, but I'm confused by the paragraph. I remember that the very purpose of Singularity is to build an environment that contains all the dependences that are independent of the host OS, no matter what. For example, if the host OS does not use systemd, I can run systemd in container and enter its name space. If the host OS is using Wayland, I can start X server inside the container so that the machine seems to be running on an OS supporting X. So, if I build the container correctly, I should not have to worry about X being deprecated. Am I right?

It's not an issue of Host <-> Container compatibility as with TigerVNC server inside the container, all of the X stack runs inside of the container. The issue is that GNOME will be dropping support for X in future. This means that inside a container that provides a future version of GNOME, GNOME will not be able to draw to TigerVNC server. This doesn't depend on the host at all, only what's in the container.

TightVNC provides the X display that the GNOME desktop draws itself too. In future, GNOME will only support wayland. It will not draw to an X display, so you won't be able to use it via TigerVNC. This change is currently being discussed and happening in e.g. Fedora, and is likely to come to a future Ubuntu release.

It might not concern you if you anticipate only using Ubuntu 22.04. I just wanted to raise it as a consideration prior to investing a lot of time in an approach that might need to continue working for several years / with newer Linux distributions in the container.

[Heng-Zhou]

If I work it out finally, I will publish it so that everyone can access it. Hope I can receive further helps from you :-)

I got what you mean. That's really an issue. TightVNC does not support Wayland. For example, I'm using Dell's servers remotely in a HPC. Dell's server management system, iDRAC, provides a VNC server for users to view the server screen remotely. But if I log into Ubuntu using Wayland, the remote console will say "NO SIGNAL". Maybe TightVNC can somehow support Wayland in the future (and change its name to WaylandVNC), or maybe there is some other solution rising up ... Anyway, for the moment, I will focus on Ubuntu 22.04 and X in hope that I can work out the container.

[Heng-Zhou]

I made it, finally.

I was mislead by my past (correct and working) experience on creating Ubuntu 22.04.3 Server LTS + GNOME GUI desktop + TigerVNC in WSL. I had thought Singularity is simpler than WSL (e.g, Singularity does not boot while WSL boots), so the guest OS should be more incomplete than that in WSL, so the creation of the container would be harder than that of WSL, as dtrudg commented, "extremely complex". In WSL, since the guest OS is incomplete, the solution is creating a new namespace with systemd being the first process (so init process) in which systemd will do the complete configuration, and then entering this new namespace to start GNOME and VNC server. Because I expected at first that the creation of Singularity container is harder, I was following this solution. This solution needs to run sudo command, so I resorted to fakeroot. The problem of fakeroot settings in the Singularity document is that the root user of the host is not mapped, consequently files and directories owned by real root is changed to nobody. However, many system settings like D-Bus system put configurations in these directories, and they check whether the owner of the directory is "root", perhaps for security consideration. This causes a lot of errors since the owner is now changed to "nobody". Because D-Bus is not working, X Window system is not working, then GNOME, then VNC server, because they are working on top of the previous one.

Realizing these problems, I decided to give up fakeroot mode, and the WSL solution which creates and enters a new namespace in which systemd does the job of booting and initializing. The key here is that although Singularity is simpler and not booting, it can make use of the outside fully fledged and fully initialized and configured host OS, which provides D-Bus system and other system components that are already up and running; we don't have to create our own. In WSL, on the contrary, we have to because WSL is isolated from the host OS, but Singularity is tightly tied to the host OS! It works at last.

I promised that I will publish the definition file. I tested it on both Ubuntu 22.04 Server LTS and Red Hat 8.8. There are no doubt rooms to improve the code, like embedding the xstartup file in the definition and creating a runscript, but the major part of the container is completed. Here it is (named gnome.def):

Bootstrap: library
From: ubuntu

%post
    set -xe
    yes | unminimize
    apt -y update && apt -y upgrade

    # Preconfigure selections
    echo "tzdata tzdata/Areas select America" | debconf-set-selections
    echo "tzdata tzdata/Zones/America select New_York" | debconf-set-selections

    # Update and install without any interactive frontend
    export DEBIAN_FRONTEND=noninteractive
    apt install -y --no-install-recommends tzdata

    #   install GNOME
    apt install -y ubuntu-gnome-desktop
    apt install -y python3 python3-pip procps websockify xterm gcc dbus-x11 xorg wget git vim-tiny vim-gtk3 zsh locales libtinfo-dev libtinfo5 lsb-release libboost-filesystem-dev nano less libgtk-3-bin net-tools iputils-ping strace rsyslog 

    ln -s /usr/lib/x86_64-linux-gnu/libboost_filesystem.so /usr/lib/x86_64-linux-gnu/libboost_filesystem.so.1.66.0
    ln -s /usr/lib/x86_64-linux-gnu/libboost_system.so /usr/lib/x86_64-linux-gnu/libboost_system.so.1.66.0

    apt remove -y light-locker

    # install VNC
    systemctl set-default multi-user.target
    apt install -y tigervnc-standalone-server

    sed -i '/^#.* en_US.* /s/^#//' /etc/locale.gen
    locale-gen

%environment
    export LC_ALL=en_US.UTF-8
    export LANG=en_US.UTF-8
    export LANGUAGE=en_US

To run the container, be sure to bind /run directory, because that's where DBUS_SESSION_BUS_ADDRESS points to (more details here):

singularity shell --writable-tmpfs --bind /run gnome.sif

Then inside the container, you can run vncserver as if in a normal Ubuntu. In case you are not familiar with TigerVNC, following are steps:

1. Run vncserver to set up password, if you haven't already.
2. Run nano ~/.vnc/xstartup and copy the following into it:

#!/bin/sh
# Start Gnome 3 Desktop 
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
vncconfig -iconic &
dbus-launch --exit-with-session gnome-session

3. Run chmod +x ~/.vnc/xstartup to make it executable.
4. Run vncserver -kill :1 to kill the VNC session created in 1). The display could be :2, so please pay attention to it in step 1).
5. Run vncserver -localhost no -geometry 3840x2160 :5 to start the GNOME and VNC server that transfers the GUI desktop back to your local machine. Adjust 3840x2160 according to your actual screen resolution.
6. Use MobaXterm to connect the VNC server. Be sure that the port is 5905 because we use display :5 in the last step. The settings varies according to your networking configuration, so I am not going to details. For me, I used two jump hosts to connect to the target HPC.

Below is the screenshot. Enjoy! :-)

[Heng-Zhou]

In order to support systemd, TigerVNC changed the implementation a lot, so vncserver is unusable in Ubuntu 24.04. Following is a revision of the Singularity def file in the answer so that it works with Ubuntu 24.04 without sudo.

Bootstrap: docker
From: ubuntu

%post
    apt -y update && apt -y upgrade
    apt-get -y install unminimize
    # Preconfigure selections
    echo "tzdata tzdata/Areas select America" | debconf-set-selections
    echo "tzdata tzdata/Zones/America select New_York" | debconf-set-selections
    yes | unminimize

    # Update and install without any interactive frontend
    export DEBIAN_FRONTEND=noninteractive
    apt install -y --no-install-recommends tzdata

    # install GNOME
    apt install -y ubuntu-gnome-desktop
    apt install -y python3 python3-pip procps websockify gcc dbus-x11 xorg wget git vim-tiny vim-gtk3 zsh locales libncurses-dev libtinfo6 lsb-release libboost-filesystem-dev nano less libgtk-3-bin net-tools iputils-ping strace rsyslog

    # install GNOME GUI related packages
    apt install -y --no-install-recommends gnome-tweaks
    apt install -y gnome-shell-extensions
    gsettings set org.gnome.desktop.interface text-scaling-factor 1.2

    sed -i '/^#.* en_US.* /s/^#//' /etc/locale.gen
    locale-gen

    # install VNC
    apt install -y tigervnc-standalone-server
    systemctl set-default multi-user.target

%environment
    export LC_ALL=en_US.UTF-8
    export LANG=en_US.UTF-8
    export LANGUAGE=en_US

To use it after generating the .sif file:

1. In the host command line, use vncpasswd command to generate password for VNC connection. The password filename and position is the same as previous version.
2. Run nano ~/.vnc/Xtigervnc-session and copy the following contents into it:

#!/bin/sh
# Start Gnome 3 Desktop
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
XDG_CURRENT_DESKTOP=ubuntu:GNOME dbus-launch --exit-with-session gnome-session --session=ubuntu

Set the file to be executable: chmod +x ~/.vnc/Xtigervnc-session.

3. Shell into the .sif file (assume rdp.sif):
   singularity shell --writable-tmpfs --bind /run rdp.sif
4. In the container, run
   tigervncserver -SecurityTypes VncAuth,TLSVnc -geometry 3840x2160 -localhost no :5
   to start the VNC service
5. Use MobaXterm to connect the VNC server.
6. To stop the VNC service, run in the container:
   tigervncserver -kill :5

Some notes:

• Singularity Container Library has not released Ubuntu 24.04 LTS base OS so far, so here I pull Ubuntu 24.04 from docker.
• Terminal would not run if simply set LC_ALL=C in %environment section.
• The version of TigerVNC is 1.13.1. No sudo is needed when starting it.
• The desktop I use is ubuntu. You can use "Extensions" app to turn the Dash on (turn on "Ubuntu Dock" extension) if it is not already. You can change the last line of Xtigervnc-session to dbus-launch --exit-with-session gnome-session if you'd like to use GNOME desktop.