DNS Failure

[francisw]

Please can someone put me out of my misery?

# docker run -it tiredofit/nginx-php-fpm:alpine-8.0 bash
...
2022-03-28.06:56:49 [STARTING] ** [monitoring] [1] Starting Zabbix Agent (modern) 6.0.2
2022-03-28.06:56:49 [STARTING] ** [scheduling] [1] Starting cron
[tiredofit/nginx-php-fpm:6.4.6 06:56:57] # ping www.google.com
ping: www.google.com: Name does not resolve

Even adding nameserver 8.8.8.8 to resolve.conf does not fix - but the following works...
# docker run -it ubuntu bash
...
# apt-get update
# apt-get install iputils-ping
# ping www.google.com
PING www.google.com (142.250.180.4) 56(84) bytes of data.
64 bytes from lhr25s32-in-f4.1e100.net (142.250.180.4): icmp_seq=1 ttl=37 time=49.9 ms

[tiredofit]

Can you get me the output of ls -l / while inside the container?

[francisw]

Hi Dave, and thank you for taking the time to assist.

docker run -it tiredofit/nginx-php-fpm:alpine-8.0 ls -l
...
drwxr-xr-x 1 root root 4096 Mar 29 07:09 assets
drwxr-xr-x 1 root root 4096 Mar 19 09:16 bin
drwxr-xr-x 2 root root 12288 Mar 18 23:38 command
drwxr-xr-x 5 root root 360 Mar 29 07:09 dev
drwxr-xr-x 1 root root 4096 Mar 29 07:09 etc
drwxr-xr-x 2 root root 4096 Mar 16 20:15 home
-rwxr-xr-x 1 root root 907 Mar 8 01:55 init
drwxr-xr-x 1 root root 4096 Mar 19 09:16 lib
drwxr-xr-x 5 root root 4096 Mar 16 20:15 media
drwxr-xr-x 2 root root 4096 Mar 16 20:15 mnt
drwxr-xr-x 2 root root 4096 Mar 16 20:15 opt
drwxr-xr-x 6 root root 4096 Mar 18 23:38 package
dr-xr-xr-x 144 root root 0 Mar 29 07:09 proc
drwx------ 1 root root 4096 Mar 29 07:09 root
drwxr-xr-x 1 root root 4096 Mar 29 07:09 run
drwxr-xr-x 1 root root 4096 Mar 19 09:16 sbin
drwxr-xr-x 2 root root 4096 Mar 16 20:15 srv
dr-xr-xr-x 13 root root 0 Mar 29 07:09 sys
drwxrwxrwt 1 root root 4096 Mar 29 07:09 tmp
drwxr-xr-x 1 root root 4096 Mar 19 09:16 usr
drwxr-xr-x 1 root root 4096 Mar 19 09:16 var
drwxr-xr-x 1 root root 4096 Mar 29 07:09 www

[francisw]

I am available most times (CET) for interactive debugging if it will help you/us - really appreciate you publishing your work, I want to replace our home-cooked deployment containers.

[francisw]

Hi Dave,

The strangest series of events.

I reported this because we had two people in different countries observing the same issue (using docker-compose), with over a week of probing.

When I run the above test today it resolves fine. When I run in docker-compose it also resolves fine.

I will see if it has miraculously vanished for my friend and come back.

[tiredofit]

I believe you! Strange things happen sometimes, especially with some of the auto builds and Github's instability lately.

I only asked for the root file system layout as I encountered a similar thing which had me baffled up until last Friday.
Because I copy files into /etc/ from a folder called /install/etc - Somehow the folder permissions of that folder ended up looking instead of yours:

drwxr-xr-x 1 root root 4096 Mar 29 07:09 etc

to something like this:

drwx------ 1 root root 4096 Mar 29 07:09 etc

What that did is make anything that needed to look up files inside that folder unable to access. Take for example - ping will look for /etc/resolv.conf. There are a tonne of other utilities that rely on config inside of that folder as well. I couldn't update package lists, install new things, monitor my system, nothing. As soon as I changed even the folder via chmod 0755 /etc everything started working again.

So I don't know if they are the exact same thing but since i experienced it a few weeks back finally solved last Friday I'm sensing some coincidence. With the latest S6 Overlay (init system) that I use in the image permissions have changed drastically and I'm beginning to think I'll need to write a helper script to ensure some core directories don't have their permissions clobbered by either a host build that doesn't support permissions (windows?), or a github runner that is experiencing issues (the past few weeks).

Let's keep this open though as I am up for debugging this further if it happens again.

[francisw]

Thank you for your help Dave, I found the cause and solution, documented here for anyone that comes across it.

It was my version of Docker Desktop, I was behind on updates. docker/for-mac#5020

It was fixed in Docker Desktop for Mac V3.1.0

[tiredofit]

Perfect! I remember when the musl upgrade was causing all sorts of chaos a few years back well. Glad you are back on track!