Fix ipa-server-configure-first script to work with ipv4 and ipv6 addrs

[SJay3]

Now script work with different ip addresses include ipv6

[adelton]

Thank you for the pull request.

I sadly was not able to run the CI directly here on this PR (entitlements for RHEL content would be needed), so I now started it as https://github.com/adelton/freeipa-container/actions/runs/10267872034.

[adelton]

We seem to have rootless podman 4 setups failing with this change. So this will need to be investigated.

[SJay3]

@adelton Thanks! I don't have the opportunity to test on Podman and didn't see anything useful in the logs. Could you help me to test in podman?

[adelton]

I will look at it but it will take me some time before I will get to it.

[SJay3]

@adelton Could you rerun podman rootless tests with environment DEBUG_TRACE=1 and give me ipareplica-install.log, ipa-server-run.log and ipa-server-configure-first.log please? I'll trying to explore logs and fix my scripts

[adelton]

I'm currently trying to figure out how to make it easier for folks to run the tests in their forks / PRs, even if they don't have Red Hat subscription entitlements to run the RHEL builds.

[adelton]

However, the specific problem with the failure in https://github.com/adelton/freeipa-container/actions/runs/10267872034 with this PR change is that the new code forces all found interfaces to be set in DNS as IP addresses for the FreeIPA installation. That breaks setups where --ip-address is used to set one specific IP address, which happens to be the case on rootless podman setups: https://github.com/freeipa/freeipa-container/blob/master/tests/run-master-and-replica.sh#L173.

The logic which sets one IP address as the IP address is there since 2014 (c371bd4). It might not be the best logic since it relies on the order of network interfaces, but so far it seemed to work fine, primarily because you typically have just one global IPv4 address in the container.

So before even thinking about IPv6, if you are suggesting a change along the lines, we might really need to flesh out all the aspects of the sub-problem of having multiple interfaces and multiple global addresses in the container and how they should work and interact.

Which gets me to another question: what is the use-case that you try to address exactly? Wouldn't using explicit --ip-address option with the IPv6 address work just fine for your needs? That way we could avoid changing the automagic guessing of the right IP address altogether ...

[SJay3]

Ok. I'll try to explain the case I encountered.
We have a network that works entirely on ipv6, and a separate network segment that works in dual stack. Services in dual stack must respond on ipv4 and ipv6 addresses.
The freeipa master or replica installation scripts themselves support multiple --ip-address parameter specifications. And it works great. We have already tested this mechanism on our infrastructure both in dualstack and ipv6-only network.
https://manpages.ubuntu.com/manpages/xenial/en/man1/ipa-replica-install.1.html

The second aspect is that if you enable ipv6 in Docker, then 2 addresses will still arrive in the container.

But let's assume that we will use only the ipv6 address. We will still need to change the launch script in the container so that it can check which of the addresses freeipa is launched on. So why not support dual stack?

And the 3rd aspect. What if we can have 2 ipv4 addresses for the service? The host can have many interfaces and we want freeipa running in the container to be able to respond to any of them, or to some specific ones.

All tests except rootles podman passed. So without a detailed log it is not at all clear whether the problem is really in multiple addresses. But if so, then the environment for rootles podman may not be configured correctly.

[SJay3]

Could you tell me why podman with sudo run command with dns
sudo podman run -d --name freeipa-replica -h replica.example.test -v /tmp/freeipa-test-3304/data-replica:/data:Z --dns=10.88.0.5 -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp
https://github.com/adelton/freeipa-container/actions/runs/10267872034/job/28415219033#step:7:625
and rootless podman run without
podman run -d --name freeipa-replica --pod=ipa1-replica -v /tmp/freeipa-test-3458/data-replica:/data:Z --dns= -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp
https://github.com/adelton/freeipa-container/actions/runs/10267872034/job/28415222703#step:7:648

[adelton]

The freeipa master or replica installation scripts themselves support multiple --ip-address parameter specifications. And it works great.

So what prevents you using that approach exactly?

The second aspect is that if you enable ipv6 in Docker, then 2 addresses will still arrive in the container.

So why not just specify them both with --ip-address, if that it what you desire? Or just one of them, depending on what the goal is?

We will still need to change the launch script in the container so that it can check which of the addresses freeipa is launched on. So why not support dual stack?

You can use --ip6 to specify an explicit static IPv6 address for the container, and use the same value as --ip-address parameter to ipa-server-install, can't you?

The host can have many interfaces and we want freeipa running in the container to be able to respond to any of them, or to some specific ones.

Again, just specify them explicitly using --ip-address.

But if so, then the environment for rootles podman may not be configured correctly.

Well, the veth setup works to allow testing intra-container communication under rootless container. So it seems to be correct and working for that purpose.

[abbra]

From the successful run:

+ sudo podman run -d --name freeipa-replica -h replica.example.test -v /tmp/freeipa-test-3304/data-replica:/data:Z --dns=10.88.0.5 -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp

From the failed run:

podman run -d --name freeipa-replica --pod=ipa1-replica -v /tmp/freeipa-test-3458/data-replica:/data:Z --dns= -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp
...
The host name ipa.example.test does not match the value replica.example.test obtained by reverse lookup on IP address 10.0.2.100

We don't pass the right hostname to this instance. I think it is this part:

       if [ "$(id -u)" != 0 -a "$docker" == podman -a "$replica" != none ] ; then
                if [ "$N" == "freeipa-master" ] ; then
                        OPTS="$OPTS --pod=$BASE-master"
                else
                        OPTS="$OPTS --pod=$BASE-replica"
                fi
        else
                OPTS="$OPTS -h $HOSTNAME"
        fi

E.g. we only add explicit hostname option when running as root. @adelton I don't see why you are not passing the -h $HOSTNAME in rootless case. You do set the right hostname to the pod but does it really work?

        # create and start pods to get their host pids; not running containers in them yet
        podman pod create --name $BASE-master --hostname ipa.example.test --add-host ipa.example.test:172.29.0.1 --dns=127.0.0.1
        podman pod create --name $BASE-replica --hostname replica.example.test --dns=172.29.0.1

However, in podman, the pod's hostname is only propagated to the instance if UTS namespace of the host is private. Granted, the namespace must be private by default, but may be this is what fails here?

[adelton]

Could you tell me why podman with sudo run command with dns
sudo podman run -d --name freeipa-replica -h replica.example.test -v /tmp/freeipa-test-3304/data-replica:/data:Z --dns=10.88.0.5 -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp
https://github.com/adelton/freeipa-container/actions/runs/10267872034/job/28415219033#step:7:625
and rootless podman run without
podman run -d --name freeipa-replica --pod=ipa1-replica -v /tmp/freeipa-test-3458/data-replica:/data:Z --dns= -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp
https://github.com/adelton/freeipa-container/actions/runs/10267872034/job/28415222703#step:7:648

Technical answer is because a rootless container does not have IPAddress set:

          "NetworkSettings": {
               "EndpointID": "",
               "Gateway": "",
               "IPAddress": "",
               "IPPrefixLen": 0,
               "IPv6Gateway": "",
               "GlobalIPv6Address": "",
               "GlobalIPv6PrefixLen": 0,
               "MacAddress": "",
               "Bridge": "",
               "SandboxID": "",
               "HairpinMode": false,
               "LinkLocalIPv6Address": "",
               "LinkLocalIPv6PrefixLen": 0,
               "Ports": {},
               "SandboxKey": "/run/user/1000/netns/netns-10298900-848d-3b71-dfc9-6538f6f4a47b"
          },

so https://github.com/freeipa/freeipa-container/blob/master/tests/run-master-and-replica.sh#L241 does not find any value.

The reason why it does not matter and does not break things is because we've already set the --dns in the pods:
https://github.com/freeipa/freeipa-container/blob/master/tests/run-master-and-replica.sh#L134-L135.

[adelton]

podman run -d --name freeipa-replica --pod=ipa1-replica -v /tmp/freeipa-test-3458/data-replica:/data:Z --dns= -e *** localhost/freeipa-server:fedora-40 no-exit ipa-replica-install -U --principal admin --no-ntp
...
The host name ipa.example.test does not match the value replica.example.test obtained by reverse lookup on IP address 10.0.2.100

Which is however caused by the fact that both the master and replica container have that same IP address 10.0.2.100:

ipa.example.test has address 10.0.2.100
ipa.example.test has address 172.29.0.1
ipa.example.test has IPv6 address fd00::2024:ccff:fe39:3809

And as mentioned above, the proposed patch forces all IP addresses it finds to DNS, in spite of the fact that we very carefully only set the 172.29.0.1 with --ip-address.

E.g. we only add explicit hostname option when running as root. @adelton I don't see why you are not passing the -h $HOSTNAME in rootless case. You do set the right hostname to the pod but does it really work?

It does. You cannot use -h when you already have it set on the pod.

If I try to do

--- a/tests/run-master-and-replica.sh
+++ b/tests/run-master-and-replica.sh
@@ -100,7 +100,7 @@ function run_ipa_container() {
        fi
        if [ "$(id -u)" != 0 -a "$docker" == podman -a "$replica" != none ] ; then
                if [ "$N" == "freeipa-master" ] ; then
-                       OPTS="$OPTS --pod=$BASE-master"
+                       OPTS="$OPTS --pod=$BASE-master -h $HOSTNAME"
                else
                        OPTS="$OPTS --pod=$BASE-replica"
                fi

I get

+ podman run -d --name freeipa-master --pod=ipa1-master -h ipa.example.test -v /tmp/freeipa-test-334039/data:/data:Z -e PASSWORD=Secret123 localhost/freeipa-server:fedora-40 exit-on-finished -U -r EXAMPLE.TEST --setup-dns --no-forwarders --auto-reverse --allow-zone-overlap --ip-address=172.29.0.1 --no-ntp
Error: invalid config provided: cannot set hostname when joining the pod UTS namespace: invalid configuration

However, in podman, the pod's hostname is only propagated to the instance if UTS namespace of the host is private. Granted, the namespace must be private by default, but may be this is what fails here?

Could you please elaborate? I don't think that setting the hostname has anything to do with this problem. It's caused by the patch setting the DNS records to IP addresses we really don't want set, and replica then sees them.

[SJay3]

So what prevents you using that approach exactly?

So why not just specify them both with --ip-address, if that it what you desire? Or just one of them, depending on what the goal is?

This is exactly what I do. But if I specify the parameter --ip-address multiple times, the script in the container should update both addresses, right?
This is exactly the reason why I wrote this patch.

[adelton]

If you specify the --ip-address parameter, that is processed by the ipa-server-install/ipa-replica-install and the script in the container should not do anything.

[SJay3]

Why? What happens when freeipa container ip address changes after successeful replica installation ? Why we need ipa-server-configure-first script when start container?
Could you give me please a debug logs in rootless podman?

[adelton]

You asi why the --ip-address gets processed by ipa-server-install / ipa-replica-install? That's the way it is, we try not to touch anything or do anything differently in the container that we don't have to.

When the container IP changes, hopefully the static IP address of the host is still valid. That's the main point of using the --ip-address approach -- tell the FreeIPA setup that it is available on the host interfaces (typically you'd want the relevant ports published to the host, of course). Or you can manually update the DNS, just like you would if the IP address of your non-containerized FreeIPA machine changed.

The primary reason for the ipa-server-configure-first is to run ipa-server-install / ipa-replica-install during the first start of the container, and handling of upgrades, when the container image changes.

As I mentioned above, I'm trying to come up with an easier way for folks to be able to run their GitHub Actions on PRs. You can of course do it today by editing the .github/workflows/build-test.yaml, removing the RHEL parts, and adding whatever debugging you desire.