From af7298d5d94ece60dd38ec5a1f974f9cbdf76545 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Tue, 22 Oct 2024 13:59:38 +0100
Subject: [PATCH 01/19] Support binding a OAK cache to
 /home/odkuser/.data/oaklib.

Users may want to bind their local OAK cache into the
/home/odkuser/.data/oaklib directory within the container, so that any
OAK process started from within the container can benefit from the
user's cache.

When running in user mode, we must then make sure that the
/home/odkuser/.data/oaklib directory (which would be automatically
created by Docker when setting up the binding) belongs to the odkuser,
instead of the root user.
---
 scripts/entrypoint.sh | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/entrypoint.sh b/scripts/entrypoint.sh
index e34fd1e0..cfdeeaa8 100644
--- a/scripts/entrypoint.sh
+++ b/scripts/entrypoint.sh
@@ -8,7 +8,11 @@ REAL_USER_ID=${ODK_USER_ID:-1000}
 REAL_GROUP_ID=${ODK_GROUP_ID:-1000}
 
 groupadd -o -g $REAL_GROUP_ID odkuser
-useradd -o -s /bin/bash -u $REAL_USER_ID -g $REAL_GROUP_ID -c "ODK User" -m odkuser
+useradd -o -s /bin/bash -u $REAL_USER_ID -g $REAL_GROUP_ID -c "ODK User" -M odkuser
+mkdir -p /home/odkuser
+chown odkuser:odkuser /home/odkuser
+[ -d /home/odkuser/.data ] && chown odkuser:odkuser /home/odkuser/.data
+[ -d /home/odkuser/.data/oaklib ] && chown odkuser:odkuser /home/odkuser/.data/oaklib
 PATH=$PATH:/home/odkuser/.local/bin
 
 exec sudo -H -E -u odkuser "$@"

From 23543a3978fbad109044f40e65efd53a52c90f94 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Fri, 18 Oct 2024 11:52:24 +0100
Subject: [PATCH 02/19] Fix permissions issue on macOS when seeding. (#1106)

Setting Git's configuration option safe.directory to /work is enough to allow
working on a pre-seeded repository (when the repository root is bound to
/work in the container), but does not work when seeding a new
repository, because then the root of the newly created repository is not
the /work directory itself, but is in fact located two directories below
that (under /work/target/<name>). The safe.directory directive has thus
no effect when seeding.

We must set safe.directory to "*" instead, to completely disable the
permission check regardless of where the repository is. Security-wise,
this does not change anything -- that check does not bring any security
benefit in the context of the ODK, and we were already disabling it when
working on pre-seeded repositories.

closes #1105
---
 docker/odklite/Dockerfile | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/docker/odklite/Dockerfile b/docker/odklite/Dockerfile
index 42a7c092..436fcedf 100644
--- a/docker/odklite/Dockerfile
+++ b/docker/odklite/Dockerfile
@@ -76,16 +76,19 @@ RUN wget -nv https://github.com/lihaoyi/Ammonite/releases/download/$AMMONITE_VER
 # Install RDF/XML validation script
 COPY scripts/check-rdfxml.sh /tools/check-rdfxml
 
-# Force Git to accept working on a repository owned by someone else
-RUN echo "[safe]\n    directory = /work" >> /etc/gitconfig
-
 # Make sure we run under an existing user account with UID/GID
 # matching the UID/GID of the calling user
 COPY scripts/entrypoint.sh /usr/local/sbin/odkuser-entrypoint.sh
 RUN chmod 755 /usr/local/sbin/odkuser-entrypoint.sh
 ENTRYPOINT ["/usr/local/sbin/odkuser-entrypoint.sh"]
-# secure_path is enabled by default in Debian/Ubuntu, disable it
-RUN sed -i '/secure_path/d' /etc/sudoers
+
+# Misc tweaks:
+# - disable secure_path (we want to be able to use sudo with our custom PATH)
+# - remove default user account (we create our own)
+# - force Git to accept working on a repository owned by someone else
+RUN sed -i '/secure_path/d' /etc/sudoers && \
+    userdel -r ubuntu && \
+    echo "[safe]\n    directory = *" >> /etc/gitconfig
 
 # Install a script that provides information about the ODK and its tools
 COPY scripts/odk-info.sh /tools/odk-info

From b1a9d85c6a8113aa87e481acf9181510c8a3ce38 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Fri, 23 Aug 2024 13:43:34 +0100
Subject: [PATCH 03/19] Do not error out when seed is calling without a
 positional argument.

Calling `odk.py seed` without any argument results in a crash because
the `repo` variable (supposed to hold the name of the repository) is set
to None, which is not expected by the Jinja template (it expects a
string).

When no positional arguments are specified, we set the `repo` variable
to `noname` instead, thereby allowing the creation of a full repository
with a dummy name instead of causing an error.
---
 odk/odk.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/odk/odk.py b/odk/odk.py
index 014c7080..da304fdf 100755
--- a/odk/odk.py
+++ b/odk/odk.py
@@ -996,7 +996,7 @@ def seed(config, clean, outdir, templatedir, dependencies, title, user, source,
             raise Exception('max one repo; current={}'.format(repo))
         repo = repo[0]
     else:
-        repo = None
+        repo = "noname"
     mg.load_config(config,
                    imports=dependencies,
                    title=title,

From be5a5515679b5c26097d71c763abb8eafd592999 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Fri, 23 Aug 2024 13:46:22 +0100
Subject: [PATCH 04/19] Fail early if we don't have a Git username and email.

When seeding, and unless the --skipgit option has been used, we need a
username and email address to pass to Git. If we don't have those, the
seeding process will crash at the very end, after all files have been
generated.

Since the Git username and email are necessary, we should detect whether
we have them at the very beginning, and exit cleanly (with a message
telling the user what they need to to) as soon as possible, without even
initiating a process that is bound to fail.
---
 odk/odk.py | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/odk/odk.py b/odk/odk.py
index da304fdf..07e0f1b7 100755
--- a/odk/odk.py
+++ b/odk/odk.py
@@ -1007,6 +1007,11 @@ def seed(config, clean, outdir, templatedir, dependencies, title, user, source,
         project.id = repo
     if outdir is None:
         outdir = "target/{}".format(project.id)
+    if not skipgit:
+        if not "GIT_AUTHOR_NAME" in os.environ and not gitname:
+            raise click.ClickException("missing Git username; set GIT_AUTHOR_NAME or use --gitname")
+        if not "GIT_AUTHOR_EMAIL" in os.environ and not gitemail:
+            raise click.ClickException("missing Git email; set GIT_AUTHOR_EMAIL or use --gitemail")
     if clean:
         if os.path.exists(outdir):
             shutil.rmtree(outdir)

From 0fe7d8bf78332bb35b24ceefb6b87bb222c2d7b3 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Fri, 23 Aug 2024 13:49:04 +0100
Subject: [PATCH 05/19] Do not show a stacktrace if we fail because we got too
 many args.

The seed command expects only one positional arguments. If we got more
than one, we should exit with a clean error message, instead of crashing
with an uncaught exception and the associated stacktrace.
---
 odk/odk.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/odk/odk.py b/odk/odk.py
index 07e0f1b7..a3d39f54 100755
--- a/odk/odk.py
+++ b/odk/odk.py
@@ -993,7 +993,7 @@ def seed(config, clean, outdir, templatedir, dependencies, title, user, source,
     mg = Generator()
     if len(repo) > 0:
         if len(repo) > 1:
-            raise Exception('max one repo; current={}'.format(repo))
+            raise click.ClickException('max one repo; current={}'.format(repo))
         repo = repo[0]
     else:
         repo = "noname"

From 079472ba3f223043d41e01661b50f71bf2724876 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Fri, 23 Aug 2024 19:31:43 +0100
Subject: [PATCH 06/19] Make sure the SSH socket belongs to odkuser. (#1096)

When we run under a non-privileged user account (which is the default
situation), we need to make the SSH authentication socket, if it exists,
belongs to the odkuser. Under some conditions, Docker may create that
socket as belonging to root by default.
---
 scripts/entrypoint.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/entrypoint.sh b/scripts/entrypoint.sh
index cfdeeaa8..0ed80915 100644
--- a/scripts/entrypoint.sh
+++ b/scripts/entrypoint.sh
@@ -14,5 +14,6 @@ chown odkuser:odkuser /home/odkuser
 [ -d /home/odkuser/.data ] && chown odkuser:odkuser /home/odkuser/.data
 [ -d /home/odkuser/.data/oaklib ] && chown odkuser:odkuser /home/odkuser/.data/oaklib
 PATH=$PATH:/home/odkuser/.local/bin
+[ -S /run/host-services/ssh-auth.sock ] && chown odkuser /run/host-services/ssh-auth.sock
 
 exec sudo -H -E -u odkuser "$@"

From 04f437cb6fd70a66fe1c5d0db36aa8de244edf03 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Fri, 9 Aug 2024 21:54:25 +0100
Subject: [PATCH 07/19] Update SSSOM-Java.

Bump the SSSOM-Java (ROBOT plugin and command line tool) to the latest
0.9.0 version.
---
 Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Dockerfile b/Dockerfile
index ef02b13f..d94c23e2 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -12,7 +12,7 @@ ENV ODK_VERSION $ODK_VERSION
 # Software versions
 ENV JENA_VERSION=4.9.0
 ENV KGCL_JAVA_VERSION=0.4.0
-ENV SSSOM_JAVA_VERSION=0.7.7
+ENV SSSOM_JAVA_VERSION=0.9.0
 
 # Avoid repeated downloads of script dependencies by mounting the local coursier cache:
 # docker run -v $HOME/.coursier/cache/v1:/tools/.coursier-cache ...

From d0a6423bc98d4b752453b391497e0c9e50c90e35 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 28 Oct 2024 15:30:10 +0000
Subject: [PATCH 08/19] Update KGCL plugin to 0.5.0.

We update the KGCL plugin for ROBOT to the latest released version
0.5.0. That version brings only one meaningful (user-visible) change
compared to the version currently in the ODK, but it is a rather
important one: the ability to refer to nodes by their label rather than
their ID.

That is, it makes it possible to write

  obsolete 'my class'

rather than

  obsolete EX:1234
---
 Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Dockerfile b/Dockerfile
index d94c23e2..12b0ea60 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -11,7 +11,7 @@ ENV ODK_VERSION $ODK_VERSION
 
 # Software versions
 ENV JENA_VERSION=4.9.0
-ENV KGCL_JAVA_VERSION=0.4.0
+ENV KGCL_JAVA_VERSION=0.5.0
 ENV SSSOM_JAVA_VERSION=0.9.0
 
 # Avoid repeated downloads of script dependencies by mounting the local coursier cache:

From 68c0816c52f99c11f95673cfbcf8a5a6fe86cc99 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Thu, 30 May 2024 07:56:52 +0100
Subject: [PATCH 09/19] Update base system to Ubuntu 22.04.

---
 docker/builder/Dockerfile | 2 +-
 docker/odklite/Dockerfile | 2 +-
 docker/robot/Dockerfile   | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/docker/builder/Dockerfile b/docker/builder/Dockerfile
index f94e3290..ac2333f0 100644
--- a/docker/builder/Dockerfile
+++ b/docker/builder/Dockerfile
@@ -3,7 +3,7 @@
 # (typically because pre-compiled binaries don't exist for arm64) and
 # we don't want to build directly on the final image (to avoid
 # cluttering the image with build-time dependencies).
-FROM ubuntu:22.04
+FROM ubuntu:24.04
 WORKDIR /build
 
 # Software versions
diff --git a/docker/odklite/Dockerfile b/docker/odklite/Dockerfile
index 436fcedf..4021946b 100644
--- a/docker/odklite/Dockerfile
+++ b/docker/odklite/Dockerfile
@@ -1,4 +1,4 @@
-FROM ubuntu:22.04
+FROM ubuntu:24.04
 LABEL maintainer="obo-tools@googlegroups.com"
 
 ENV ROBOT_VERSION=1.9.6
diff --git a/docker/robot/Dockerfile b/docker/robot/Dockerfile
index b1229ade..5e62347c 100644
--- a/docker/robot/Dockerfile
+++ b/docker/robot/Dockerfile
@@ -1,4 +1,4 @@
-FROM ubuntu:22.04
+FROM ubuntu:24.04
 LABEL maintainer="obo-tools@googlegroups.com" 
 
 WORKDIR /tools

From 898699b43191bb1eca49096c2ebab045f5aebc7a Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Thu, 30 May 2024 07:57:57 +0100
Subject: [PATCH 10/19] Explicitly install the NPM package manager.

The NodeJs package manager (NPM) is no longer automatically installed
when we install NodeJS, so we ask for it explicitly.

We need it to install Obographviz.

Ideally it should be possible to install both NPM and Obographviz in
the builder image and then to transfer only Obographviz to the final
ODKFull image, thereby reducing the clutter in ODKFull, but this will
need further investigation.
---
 Dockerfile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Dockerfile b/Dockerfile
index 12b0ea60..c35892e6 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -40,6 +40,7 @@ RUN apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-i
     xlsx2csv \
     gh \
     nodejs \
+    npm \
     graphviz \
     python3-psycopg2 \
     swi-prolog

From 91592fbd89d5465cd89f128a7003d7c703c40c74 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Thu, 30 May 2024 11:03:08 +0100
Subject: [PATCH 11/19] Install libpcre3 in ODKFull.

This is a runtime dependency of Konclude (even the statically compiled
one that we use on x86_64). It was probably automatically pulled by
another package on Ubuntu 22.04, but it is not on 22.04, so we need to
ask for it explicitly.
---
 Dockerfile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Dockerfile b/Dockerfile
index c35892e6..881d046d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -43,7 +43,8 @@ RUN apt-get update && DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-i
     npm \
     graphviz \
     python3-psycopg2 \
-    swi-prolog
+    swi-prolog \
+    libpcre3
 
 # Install run-time dependencies for Soufflé.
 RUN DEBIAN_FRONTEND="noninteractive" apt-get install -y --no-install-recommends \

From a5cc5b56dd7162e15487f68daa1fb0edb0da3247 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 24 Jun 2024 15:46:38 +0100
Subject: [PATCH 12/19] Force system-wide installation of OBO Dashboard.

Pip is now by default refusing to install anything in the system-wide
Python library path, which is considered the "private garden" of the
underlying operating system (e.g., on Ubuntu, only APT tools should add
Python packages to the system-wide path).

This may be fine in general, in a user-facing scenario, but not for the
ODK which is in effect a "read-only" system overall. The OBO Dashboard
is part of the tools/libraries we provide with the ODK and there is no
reason for us to package it separately inside a virtual environment (the
recommended way of installing non-system Python packages) -- the entire
ODK is already a "virtual environment".

Ultimately the right thing to do here would be for the OBO Dashboard
upstream to make proper releases, which we could then install at the
same time as any other Python packages.
---
 Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Dockerfile b/Dockerfile
index 881d046d..dda62dc1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -97,7 +97,7 @@ COPY scripts/obodash /tools
 RUN chmod +x /tools/obodash && \
     git clone --depth 1 https://github.com/OBOFoundry/OBO-Dashboard.git && \
     cd OBO-Dashboard && \
-    python3 -m pip install -r requirements.txt && \
+    python3 -m pip install -r requirements.txt --break-system-packages && \
     echo " " >> Makefile && \
     echo "build/robot.jar:" >> Makefile && \
     echo "	echo 'skipped ROBOT jar download.....' && touch \$@" >> Makefile && \

From f2cf8ee97eb7fe3c583535b632e5999c5e689a50 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Thu, 30 May 2024 08:03:07 +0100
Subject: [PATCH 13/19] Update the main ODK script for newer Python.

The odk.py script has some issues when we run it under Python 3.12, we
fix them here.
---
 odk/odk.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/odk/odk.py b/odk/odk.py
index a3d39f54..406a2ba2 100755
--- a/odk/odk.py
+++ b/odk/odk.py
@@ -778,7 +778,7 @@ class ExecutionContext(JsonSchemaMixin):
     project : Optional[OntologyProject] = None
     meta : str = ""
 
-    
+
 @dataclass
 class Generator(object):
     """
@@ -787,7 +787,7 @@ class Generator(object):
     """
 
     ## TODO: consider merging Generator and ExecutionContext?
-    context : ExecutionContext = ExecutionContext()
+    context : ExecutionContext = field(default_factory=ExecutionContext)
 
     def generate(self, input : str) -> str:
         """
@@ -1088,7 +1088,7 @@ def seed(config, clean, outdir, templatedir, dependencies, title, user, source,
         print(" 5. See the section under '…or push an existing repository from the command line'")
         print("    E.g.:")
         print("cd {}".format(outdir))
-        print("git remote add origin git\@github.com:{org}/{repo}.git".format(org=project.github_org, repo=project.repo))
+        print("git remote add origin git@github.com:{org}/{repo}.git".format(org=project.github_org, repo=project.repo))
         print("git branch -M {branch}\n".format(branch=project.git_main_branch))
         print("git push -u origin {branch}\n".format(branch=project.git_main_branch))
         print("BE BOLD: you can always delete your repo and start again\n")

From 12be08c0424636100a65cec5835b6ce83f668046 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 24 Jun 2024 15:54:21 +0100
Subject: [PATCH 14/19] Silence Python warning about staged installation.

PIP emits a warning when we attempt to install executable scripts in a
location that is not in the system PATH.

In this instance the warning is not warranted. The scripts are installed
in a staging location ON PURPOSE -- they are later copied over to the
final ODK images, where the scripts will end up in the system PATH.

So we shut that warning down.
---
 docker/builder/Dockerfile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docker/builder/Dockerfile b/docker/builder/Dockerfile
index ac2333f0..75de3fc9 100644
--- a/docker/builder/Dockerfile
+++ b/docker/builder/Dockerfile
@@ -45,6 +45,7 @@ COPY constraints.txt /build/constraints.txt
 RUN python3 -m pip install \
         -r /build/requirements.txt.lite \
         -c /build/constraints.txt \
+        --no-warn-script-location \
         --root /staging/lite
 # Then those needed by the odkfull image.
 # After installing those packages, we forcibly remove from the odkfull
@@ -54,6 +55,7 @@ RUN python3 -m pip install \
 RUN python3 -m pip install \
         -c /build/constraints.txt \
         -r /build/requirements.txt \
+        --no-warn-script-location \
         --root /staging/full && \
         cd /staging/lite && \
         find . -type f | while read f ; do rm -f /staging/full/$f ; done && \

From 52d0139c5f243863e76291d2fc789a6fdc4fd4c4 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 29 Jul 2024 11:26:35 +0100
Subject: [PATCH 15/19] Force PIP to use setuptools < 72.

Where to begin? It's just another Python screw-up.

Developers of setuptools have announced, back in 2017 (!) their
intention to remove the `test` command (found in
setuptools.command.test) [1].

Today, they thought that 7 years of advance deprecation warnings were
enough, and they published setuptools v72.0.0 which no longer provides
that command. [2]

But the Python ecosystem being what it is (a joke that has lasted long
enough), it turns out that 7 years was not actually enough, and there
are still many Python packages out there that didn't get the memo and
that are still dependent on setuptools.command.test (including some
packages that are used in the ODK).

There are several ways of fixing this:

a) Ditch Python altogether. This would be by far the best solution, but
one that is, alas, unlikely to happen any time soon.

b) Fix _all_ the packages used by the ODK (and their dependencies) so
that they do what they should have done in the past 7 years. This is the
correct solution, but one that will take time (7 years were already not
enough, so who knows how long it is going to take?).

c) Torture the setuptools developers until they agree to revert the
removal of the `test` command. I believe this may be illegal in many
jurisdictions.

d) Force the ODK build process to use a version of `setuptools` older
than 72.0.0. This is what we are doing here. Yes, this is a local
workaround and we should favour "upstream fixing" instead (solution b),
but if we want to be be able to build the ODK before 2031 (optimistic
estimation of the time it will take to get all the Python packages
fixed), we don't have much choice.

[1] https://github.com/pypa/setuptools/issues/931
[2] https://setuptools.pypa.io/en/stable/history.html
---
 docker/builder/Dockerfile | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/docker/builder/Dockerfile b/docker/builder/Dockerfile
index 75de3fc9..8783b6c4 100644
--- a/docker/builder/Dockerfile
+++ b/docker/builder/Dockerfile
@@ -41,8 +41,9 @@ RUN apt-get update && \
 COPY requirements.txt.full /build/requirements.txt
 COPY requirements.txt.lite /build/requirements.txt.lite
 COPY constraints.txt /build/constraints.txt
+RUN echo "setuptools<72" > /build/pip-constraints.txt
 # First the packages needed by the odklite image.
-RUN python3 -m pip install \
+RUN PIP_CONSTRAINT=/build/pip-constraints.txt python3 -m pip install \
         -r /build/requirements.txt.lite \
         -c /build/constraints.txt \
         --no-warn-script-location \
@@ -52,7 +53,7 @@ RUN python3 -m pip install \
 # staging tree any file already present in the odklite staging tree
 # (caused by packages installed in both trees) to avoid needlessly
 # increasing the size of the odkfull image.
-RUN python3 -m pip install \
+RUN PIP_CONSTRAINT=/build/pip-constraints.txt python3 -m pip install \
         -c /build/constraints.txt \
         -r /build/requirements.txt \
         --no-warn-script-location \

From bab81c2512cf38ba353ddc1a0e5dd75221de39f2 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 29 Jul 2024 18:03:08 +0100
Subject: [PATCH 16/19] Replace j2cli with jinjanator.

The j2cli project (which provides the j2 tool) is no longer maintained
and is broken under Python 3.12, so we replace it with Jinjanator
(providing the similar command jinjanate).
---
 Makefile              | 1 +
 requirements.txt.full | 3 +--
 requirements.txt.lite | 3 +--
 3 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/Makefile b/Makefile
index 47ec3b8e..8ba488df 100644
--- a/Makefile
+++ b/Makefile
@@ -35,6 +35,7 @@ test_odklite_programs:
 	@./tests/test-program.sh DOSDP-TOOLS dosdp-tools -v
 	@./tests/test-program.sh OWLTOOLS owltools --version
 	@./tests/test-program.sh AMMONITE sh amm --help
+	@./tests/test-program.sh JINJANATOR jinjanate --version
 	@./tests/test-program.sh ODK odk.py
 
 test_odkfull_programs: test_odklite_programs
diff --git a/requirements.txt.full b/requirements.txt.full
index a792dbe0..1ec0ac41 100644
--- a/requirements.txt.full
+++ b/requirements.txt.full
@@ -40,8 +40,7 @@ mkdocs-table-reader-plugin
 funowl
 kgx
 tsvalid
-j2cli[yaml]
-j2cli
+jinjanator
 pyspellchecker
 jsonschema2md
 curies
diff --git a/requirements.txt.lite b/requirements.txt.lite
index 5268d309..d9ebc1f6 100644
--- a/requirements.txt.lite
+++ b/requirements.txt.lite
@@ -11,8 +11,7 @@ pyyaml
 ruamel.yaml
 dosdp
 tsvalid
-j2cli[yaml]
-j2cli
+jinjanator
 lightrdf
 sssom
 babelon

From 27ad3ded36cae617fadd4ace2be82ee62a4c754a Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 28 Oct 2024 15:54:31 +0000
Subject: [PATCH 17/19] Update Python constraints.

---
 constraints.txt | 278 ++++++++++++++++++++++++------------------------
 1 file changed, 139 insertions(+), 139 deletions(-)

diff --git a/constraints.txt b/constraints.txt
index 2f2d4e24..96b05efb 100644
--- a/constraints.txt
+++ b/constraints.txt
@@ -1,103 +1,105 @@
 airium==0.2.6
-alabaster==0.7.16
+alabaster==1.0.0
 annotated-types==0.7.0
 antlr4-python3-runtime==4.9.3
-anyio==4.4.0
+anyio==4.6.2.post1
 appdirs==1.4.4
 argon2-cffi==23.1.0
 argon2-cffi-bindings==21.2.0
 arrow==1.3.0
 asttokens==2.4.1
 async-lru==2.0.4
-attrs==23.2.0
-Babel==2.15.0
+attrs==24.2.0
+babel==2.16.0
 babelon==0.2.9
 backports.tarfile==1.2.0
 bcp47==0.1.0
 beautifulsoup4==4.12.3
 bidict==0.23.1
 bleach==6.1.0
-bmt==1.4.3
-cachetools==5.3.3
-cattrs==23.2.3
-certifi==2024.7.4
-cffi==1.16.0
+bmt==1.4.4
+cachetools==5.5.0
+cattrs==24.1.2
+certifi==2024.8.30
+cffi==1.17.1
 CFGraph==0.2.1
 chardet==5.2.0
-charset-normalizer==3.3.2
-class_resolver==0.4.3
+charset-normalizer==3.4.0
+class_resolver==0.5.2
 click==8.1.7
 click-default-group==1.2.4
 colorama==0.4.6
 comm==0.2.2
 commonmark==0.9.1
-contourpy==1.2.1
-cryptography==42.0.8
-curies==0.7.9
+contourpy==1.3.0
+cryptography==43.0.3
+curies==0.7.10
 cycler==0.12.1
 dacite==1.8.1
 daff==1.3.46
 dataclasses==0.6
 dataclasses-json==0.6.7
 dataclasses-jsonschema==2.16.0
-debugpy==1.8.2
+debugpy==1.8.7
 decorator==5.1.1
 defusedxml==0.7.1
 Deprecated==1.2.14
 deprecation==2.1.0
 diskcache==5.6.3
-distlib==0.3.8
+distlib==0.3.9
 distro==1.9.0
 docker==6.1.3
-docutils==0.18.1
+docutils==0.21.2
 dosdp==0.1.10.dev1
 EditorConfig==0.12.4
-et-xmlfile==1.1.0
+et_xmlfile==2.0.0
 eutils==0.6.0
-exceptiongroup==1.2.1
-executing==2.0.1
+exceptiongroup==1.2.2
+executing==2.1.0
 fastjsonschema==2.20.0
 fastobo==0.12.3
-filelock==3.15.4
-fonttools==4.53.0
+filelock==3.16.1
+fonttools==4.54.1
 fqdn==1.5.1
 funowl==0.2.3
 ghp-import==2.1.0
 google==3.0.0
-google-api-core==2.19.1
-google-api-python-client==2.136.0
-google-auth==2.31.0
+google-api-core==2.21.0
+google-api-python-client==2.149.0
+google-auth==2.35.0
 google-auth-httplib2==0.2.0
-google-auth-oauthlib==1.2.0
-googleapis-common-protos==1.63.2
+google-auth-oauthlib==1.2.1
+googleapis-common-protos==1.65.0
 graphviz==0.20.3
-greenlet==3.0.3
-gspread==6.1.2
+greenlet==3.1.1
+gspread==6.1.4
 gspread-formatting==1.2.0
 h11==0.14.0
 hbreader==0.9.1
-httpcore==1.0.5
+httpcore==1.0.6
 httplib2==0.22.0
-httpx==0.27.0
-idna==3.7
+httpx==0.27.2
+idna==3.10
 ijson==3.3.0
 imagesize==1.4.1
-importlib_metadata==8.0.0
-importlib_resources==6.4.0
+importlib_metadata==8.5.0
+importlib_resources==6.4.5
 inflection==0.5.1
 iniconfig==2.0.0
 ipykernel==6.29.5
-ipython==8.26.0
-ipywidgets==8.1.3
-isodate==0.6.1
+ipython==8.29.0
+ipywidgets==8.1.5
+isodate==0.7.2
 isoduration==20.11.0
-j2cli==0.3.10
 jaraco.classes==3.4.0
-jaraco.context==5.3.0
-jaraco.functools==4.0.1
+jaraco.context==6.0.1
+jaraco.functools==4.1.0
 jedi==0.19.1
 jeepney==0.8.0
 Jinja2==3.1.4
+jinjanator==24.4.0
+jinjanator-plugins==24.2.0
+jiter==0.6.1
 jsbeautifier==1.15.1
 jsobject==0.10.2
 json-flattener==0.1.9
@@ -106,148 +108,146 @@ jsonasobj==1.3.1
 jsonasobj2==1.0.4
 jsonlines==4.0.0
 jsonpatch==1.33
-jsonpath-ng==1.6.1
+jsonpath-ng==1.7.0
 jsonpath-rw==1.4.0
-jsonpickle==3.2.2
+jsonpickle==3.3.0
 jsonpointer==3.0.0
-jsonschema==4.22.0
-jsonschema-specifications==2023.12.1
-jsonschema2md==1.1.0
+jsonschema==4.23.0
+jsonschema-specifications==2024.10.1
+jsonschema2md==1.3.0
 jsonstreams==0.6.0
-jupyter==1.0.0
+jupyter==1.1.1
 jupyter-console==6.6.3
 jupyter-events==0.10.0
 jupyter-lsp==2.2.5
-jupyter_client==8.6.2
+jupyter_client==8.6.3
 jupyter_core==5.7.2
-jupyter_server==2.14.1
+jupyter_server==2.14.2
 jupyter_server_terminals==0.5.3
-jupyterlab==4.2.3
+jupyterlab==4.2.5
 jupyterlab_pygments==0.3.0
-jupyterlab_server==2.27.2
-jupyterlab_widgets==3.0.11
-keyring==25.2.1
+jupyterlab_server==2.27.3
+jupyterlab_widgets==3.0.13
+keyring==25.5.0
 kgcl-rdflib==0.5.0
-kgcl_schema==0.6.8
-kgx==2.4.0
-kiwisolver==1.4.5
-lark==1.1.9
+kgcl_schema==0.6.9
+kgx==2.4.2
+kiwisolver==1.4.7
+lark==1.2.2
 lightrdf==0.4.0
-linkml==1.8.1
+linkml==1.8.5
 linkml-dataops==0.1.0
 linkml-renderer==0.3.1
-linkml-runtime==1.8.0
+linkml-runtime==1.8.3
 llm==0.13.1
-lxml==5.2.2
-Markdown==3.6
+lxml==5.3.0
+Markdown==3.7
 markdown-it-py==3.0.0
-MarkupSafe==2.1.5
-marshmallow==3.21.3
-matplotlib==3.9.1
+MarkupSafe==3.0.2
+marshmallow==3.23.0
+matplotlib==3.9.2
 matplotlib-inline==0.1.7
-matplotlib-venn==0.11.10
+matplotlib-venn==1.1.1
 mdurl==0.1.2
 mergedeep==1.3.4
 mistune==3.0.2
-mkdocs==1.6.0
+mkdocs==1.6.1
 mkdocs-get-deps==0.2.0
-mkdocs-material==9.5.28
+mkdocs-material==9.5.42
 mkdocs-material-extensions==1.3.1
 mkdocs-mermaid2-plugin==1.1.1
-mkdocs-table-reader-plugin==2.2.2
+mkdocs-table-reader-plugin==3.1.0
 more-click==0.1.2
-more-itertools==10.3.0
-mypy==1.10.1
+more-itertools==10.5.0
+mypy==1.13.0
 mypy-extensions==1.0.0
 nbclient==0.10.0
 nbconvert==7.16.4
 nbformat==5.10.4
 ndex2==3.9.0
-neo4j==4.4.12
+neo4j==5.25.0
 nest-asyncio==1.6.0
-networkx==3.3
-nh3==0.2.17
-notebook==7.2.1
+networkx==3.4.2
+nh3==0.2.18
+notebook==7.2.2
 notebook_shim==0.2.4
 numpy==1.26.4
-oaklib==0.6.11
+oaklib==0.6.18
 oauthlib==3.2.2
 ols-client==0.1.4
-ontobio==2.9.1
+ontobio==2.9.10
 ontodev-cogs==0.3.3
 ontodev-gizmos==0.3.2
 ontoportal-client==0.0.4
-openai==1.35.10
+openai==1.52.2
 openpyxl==3.1.5
 ordered-set==4.1.0
 overrides==7.7.0
 packaging==24.1
-paginate==0.5.6
-pandas==2.2.2
+paginate==0.5.7
+pandas==2.2.3
 pandocfilters==1.5.1
 pansql==0.0.1
 parse==1.20.2
 parso==0.8.4
 pathspec==0.12.1
 pexpect==4.9.0
-pillow==10.4.0
+pillow==11.0.0
 pkginfo==1.10.0
-platformdirs==4.2.2
-plotly==5.22.0
+platformdirs==4.3.6
+plotly==5.24.1
 pluggy==1.5.0
 ply==3.11
 prefixcommons==0.1.12
-prefixmaps==0.2.4
+prefixmaps==0.2.6
 prologterms==0.0.6
-prometheus_client==0.20.0
-prompt_toolkit==3.0.47
-pronto==2.5.7
-proto-plus==1.24.0
-protobuf==5.27.2
-psutil==6.0.0
+prometheus_client==0.21.0
+prompt_toolkit==3.0.48
+pronto==2.5.8
+proto-plus==1.25.0
+protobuf==5.28.3
+psutil==6.1.0
 ptyprocess==0.7.0
-pure-eval==0.2.2
+pure_eval==0.2.3
 pyarrow==15.0.2
-pyasn1==0.6.0
-pyasn1_modules==0.4.0
+pyasn1==0.6.1
+pyasn1_modules==0.4.1
 pycparser==2.22
-pydantic==2.8.2
-pydantic_core==2.20.1
+pydantic==2.9.2
+pydantic_core==2.23.4
 pydotplus==2.0.2
-PyGithub==2.3.0
+PyGithub==2.4.0
 Pygments==2.18.0
 PyJSG==0.11.10
-PyJWT==2.8.0
-pymdown-extensions==10.8.1
+PyJWT==2.9.0
+pymdown-extensions==10.11.2
 PyNaCl==1.5.0
 pyparsing==2.4.7
-pyproject-api==1.7.1
+pyproject-api==1.8.0
 PyShEx==0.8.1
 PyShExC==0.9.1
-pysolr==3.9.0
+pysolr==3.10.0
 pyspellchecker==0.8.1
-pystow==0.5.4
-pytest==8.2.2
+pystow==0.5.6
+pytest==8.3.3
 pytest-logging==2015.11.4
 python-dateutil==2.9.0.post0
 python-dotenv==1.0.1
 python-json-logger==2.0.7
-python-ulid==2.7.0
+python-ulid==3.0.0
 PyTrie==0.4.0
-pytz==2024.1
-PyYAML==6.0.1
+pytz==2024.2
+PyYAML==6.0.2
 pyyaml_env_tag==0.1
-pyzmq==26.0.3
-qtconsole==5.5.2
-QtPy==2.4.1
+pyzmq==26.2.0
 ratelimit==2.2.1
-rdflib==7.0.0
+rdflib==7.1.1
 rdflib-jsonld==0.6.1
 rdflib-shim==1.0.3
-readme_renderer==43.0
+readme_renderer==44.0
 recommonmark==0.7.1
 referencing==0.35.1
-regex==2024.5.15
+regex==2024.9.11
 requests==2.32.3
 requests-cache==1.2.1
 requests-oauthlib==2.0.0
@@ -256,15 +256,15 @@ rfc3339-validator==0.1.4
 rfc3986==2.0.0
 rfc3986-validator==0.1.1
 rfc3987==1.3.8
-rich==13.7.1
-rpds-py==0.18.1
+rich==13.9.3
+rpds-py==0.20.0
 rsa==4.9
 ruamel.yaml==0.18.6
-ruamel.yaml.clib==0.2.8
-scipy==1.14.0
+ruamel.yaml.clib==0.2.12
+scipy==1.14.1
 seaborn==0.13.2
 SecretStorage==3.3.3
-semsimian==0.2.16
+semsimian==0.2.19
 semsql==0.3.3
 Send2Trash==1.8.3
 ShExJSG==0.8.2
@@ -272,56 +272,56 @@ six==1.16.0
 sniffio==1.3.1
 snowballstemmer==2.2.0
 sortedcontainers==2.4.0
-soupsieve==2.5
+soupsieve==2.6
 sparqlslurper==0.5.1
 SPARQLWrapper==2.0.0
-Sphinx==7.3.7
-sphinxcontrib-applehelp==1.0.8
-sphinxcontrib-devhelp==1.0.6
-sphinxcontrib-htmlhelp==2.0.5
+Sphinx==8.1.3
+sphinxcontrib-applehelp==2.0.0
+sphinxcontrib-devhelp==2.0.0
+sphinxcontrib-htmlhelp==2.1.0
 sphinxcontrib-jsmath==1.0.1
-sphinxcontrib-qthelp==1.0.7
-sphinxcontrib-serializinghtml==1.1.10
-SQLAlchemy==2.0.31
+sphinxcontrib-qthelp==2.0.0
+sphinxcontrib-serializinghtml==2.0.0
+SQLAlchemy==2.0.36
 SQLAlchemy-Utils==0.38.3
 sqlite-fts4==1.0.3
 sqlite-migrate==0.1b0
-sqlite-utils==3.36
-sssom==0.4.11
-sssom-schema==0.15.2
+sqlite-utils==3.37
+sssom==0.4.12
+sssom-schema==1.0.0
 stack-data==0.6.3
 stringcase==1.2.0
 tabulate==0.9.0
-tenacity==8.4.2
-termcolor==2.4.0
+tenacity==9.0.0
+termcolor==2.5.0
 terminado==0.18.1
 terminaltables==3.1.10
-tinycss2==1.3.0
-tomli==2.0.1
+tinycss2==1.4.0
+tomli==2.0.2
 tornado==6.4.1
-tox==4.16.0
-tqdm==4.66.4
+tox==4.23.2
+tqdm==4.66.6
 traitlets==5.14.3
 tsvalid==0.0.3
 twine==5.1.1
-types-python-dateutil==2.9.0.20240316
+types-python-dateutil==2.9.0.20241003
 typing-inspect==0.9.0
 typing_extensions==4.12.2
-tzdata==2024.1
+tzdata==2024.2
 UpSetPlot==0.9.0
 uri-template==1.3.0
 uritemplate==4.1.1
 url-normalize==1.4.3
-urllib3==2.2.2
+urllib3==2.2.3
 validators==0.20.0
-virtualenv==20.26.3
-watchdog==4.0.1
+virtualenv==20.27.0
+watchdog==5.0.3
 wcwidth==0.2.13
-webcolors==24.6.0
+webcolors==24.8.0
 webencodings==0.5.1
 websocket-client==1.8.0
-widgetsnbextension==4.0.11
+widgetsnbextension==4.0.13
 wrapt==1.16.0
 xmltodict==0.13.0
 yamldown==0.1.8
-zipp==3.19.2
+zipp==3.20.2

From 29c27b9f33c75a237ca82ac6c189524b4dad344b Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Thu, 30 May 2024 10:54:32 +0100
Subject: [PATCH 18/19] Do not install virtualenv in the builder image.

Installing virtualenv in the builder image has nafarious consequences
when we try to later install Python packages.

That's because the Ubuntu package for virtualenv automatically installs
platformdirs version 2.5.1, which then prevents us from installing the
platformdirs 4.x that we need as a dependency for some of our packages.

The only reason we had virtualenv in the builder image was that we use
it to run the update-constraints workflow (in which we try to install
all our Python packages in a virtualenv). So here, we

1) remove virtualenv from the builder image;

2) amend the update-constraints.sh script to make it install virtualenv
   itself.
---
 Makefile                  | 2 +-
 docker/builder/Dockerfile | 3 +--
 update-constraints.sh     | 5 +++++
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 8ba488df..1b914e03 100644
--- a/Makefile
+++ b/Makefile
@@ -169,7 +169,7 @@ publish-multiarch-dev:
 		.
 
 constraints.txt: requirements.txt.full
-	docker run -v $$PWD:/work -w /work --rm -ti obolibrary/odkbuild:latest /work/update-constraints.sh
+	docker run -v $$PWD:/work -w /work --rm -ti obolibrary/odkbuild:latest /work/update-constraints.sh --install-virtualenv
 
 clean-tests:
 	rm -rf target/*
diff --git a/docker/builder/Dockerfile b/docker/builder/Dockerfile
index 8783b6c4..31f58786 100644
--- a/docker/builder/Dockerfile
+++ b/docker/builder/Dockerfile
@@ -27,8 +27,7 @@ RUN apt-get update && \
         rustc \
         cargo \
         python3-dev \
-        python3-pip \
-        python3-virtualenv
+        python3-pip
 
 # Build the Python packages.
 # On x86_64, most if not all of these packages should be available as
diff --git a/update-constraints.sh b/update-constraints.sh
index 89a2cd78..2882d9e6 100755
--- a/update-constraints.sh
+++ b/update-constraints.sh
@@ -2,6 +2,11 @@
 
 set -e
 
+if [ "x$1" = x--install-virtualenv ]; then
+    apt-get update
+    DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-virtualenv
+fi
+
 virtualenv tmpdir
 . tmpdir/bin/activate
 python3 -m pip install -U pip

From b6e32020814b9ab961993ab9ac9b29436a5a9fe7 Mon Sep 17 00:00:00 2001
From: Damien Goutte-Gattat <dpg44@cam.ac.uk>
Date: Mon, 28 Oct 2024 16:57:09 +0000
Subject: [PATCH 19/19] Prepare 1.5.3 release.

Bump version number and update the changelog.
---
 CHANGELOG.md            | 20 ++++++++++++++++++++
 Makefile                |  2 +-
 docker/odklite/Makefile |  2 +-
 3 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0f25cf67..fbe530eb 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,22 @@
+# v1.5.3
+
+- Updates:
+    - Base image updated to Ubuntu 24.04
+    - Python updated to version 3.12
+    - Several Python packages updated, including:
+        - `oaklib` (OAK) 0.6.18
+        - `linkml` 1.8.5 and `linkml-runtime` 1.8.3
+        - `sssom-schema` 1.0.0 and `sssom` 0.4.12
+    - KGCL plugin for ROBOT updated to version 0.5.0
+    - SSSOM command-line tool and plugin
+- Bugfixes:
+    - Fix permission issue on macOS when seeding ([#1105](https://github.com/INCATools/ontology-development-kit/issues/1105))
+    - Fix permission on the SSH socket ([#1096](https://github.com/INCATools/ontology-development-kit/issues/1096))
+    - Fix lack of robustness of the odk.py script ([#1097](https://github.com/INCATools/ontology-development-kit/issues/1097))
+
+BREAKING CHANGE: [J2cli](https://github.com/kolypto/j2cli), which is no longer maintained, has been removed and replaced with [Jinjanator](https://github.com/kpfleming/jinjanator). Standard ODK-generated workflows never used that tool, but if you have custom workflows that are using the `j2` command, you will need to update them to make them use `jinjanate` instead.
+
+
 # v1.5.2
 
 This version updates the Ontology Access Kit (`oaklib`) to improve
@@ -16,6 +35,7 @@ compatibility with the OWLAPI version 4.5.29 used in ROBOT 1.9.6.
     - Added back `class-count-by-prefix.sparql` ([https://github.com/INCATools/ontology-development-kit/issues/1030](#1030))
     - Disabled `table-reader` mkdocs plygin ([https://github.com/INCATools/ontology-development-kit/issues/1028](#1028))
 
+
 # v1.5
 
 For more detailed changes see:
diff --git a/Makefile b/Makefile
index 1b914e03..93d06f7b 100644
--- a/Makefile
+++ b/Makefile
@@ -62,7 +62,7 @@ docs:
 	@ODK_IMAGE=odklite ./odk.sh python ./odk/schema_documentation.py
 
 # Building docker image
-VERSION = "v1.5.2"
+VERSION = "v1.5.3"
 IM=obolibrary/odkfull
 IMLITE=obolibrary/odklite
 ROB=obolibrary/robot
diff --git a/docker/odklite/Makefile b/docker/odklite/Makefile
index b6807cdf..4c6c4306 100644
--- a/docker/odklite/Makefile
+++ b/docker/odklite/Makefile
@@ -1,6 +1,6 @@
 
 # Building docker image
-VERSION = "v1.5.2"
+VERSION = "v1.5.3"
 IM=obolibrary/odklite
 TAGS_OPTION=-t $(IM):$(VERSION) -t $(IM):latest