Support --headless=new

[sfcgeorge]

Describe the bug

Chrome has a new headless mode! https://developer.chrome.com/articles/new-headless/

The old mode is --headless, the new one is --headless=chrome in v108 and below (current Zenika version) or --headless=new in subsequent versions.

From within the container it seems to work fine docker compose exec chrome wget -O- http://localhost:9222/json/list

But from another container I get "Connection refused" docker compose exec web wget -O- http://chrome:9222/json/list

To Reproduce

Based on this issue #158 I think it's --remote-debugging-address=0.0.0.0 not working, which would be a Chrome bug, so I've reported it there https://bugs.chromium.org/p/chromium/issues/detail?id=1427419

I'm opening the issue here for others to find or in case I'm wrong and there's some other magic flag that would fix it.

The new mode will become default one day and eventually the old mode will be removed.

Possible solution

Tried various flag incantations with no change.

  chrome:
    image: zenika/alpine-chrome:latest
    entrypoint: ''
    command:
      [
        chromium-browser,
        # Start Chrome with no GUI
        "--headless=chrome", # change to --headless=new in Chrome 109
        # Enable the remote web inspector for debugging specs
        "--remote-debugging-address=0.0.0.0",
        "--remote-debugging-port=9222",
        # "--whitelisted-ips",
        # "--allowed-origins=*",
        # Various GPU related flags to make Chrome faster more stable
        # "--disable-gpu",
        "--disable-software-rasterizer",
        # "--disable-features=VizDisplayCompositor",
        # /dev/shm partition is too small in some VMs, causing Chrome to crash
        "--disable-dev-shm-usage",
        # Stuff I don't understand but seems to be needed to run
        "--no-sandbox",
        "--disable-setuid-sandbox",
        "--disable-web-security",
        # "--user-data-dir=/tmp/chromeuser",
        # "--cap-add SYS_ADMIN",
        # "--enable-automation",
        "--allow-insecure-localhost",
        "--enable-features=NetworkService",
      ]
    ports:
      - "9222:9222"

Logs

Chrome logs (setting --user-data-dir doesn't fix this):

eola-chrome-1  | [0324/112046.572111:ERROR:chrome_main_delegate.cc(951)] Web security may only be disabled if '--user-data-dir' is also specified with a non-default value.
eola-chrome-1  | [1:25:0324/112046.670204:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
eola-chrome-1  | [1:25:0324/112046.670262:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
eola-chrome-1  | [1:36:0324/112046.697345:ERROR:bus.cc(399)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
eola-chrome-1  | [1:36:0324/112046.697563:ERROR:bus.cc(399)] Failed to connect to the bus: Could not parse server address: Unknown address type (examples of valid types are "tcp" and on UNIX "unix")
eola-chrome-1  | 
eola-chrome-1  | DevTools listening on ws://127.0.0.1:9222/devtools/browser/d5ac929c-8d7e-44ed-87ee-ce6eff9662cd
eola-chrome-1  | [1:81:0324/112046.751720:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
eola-chrome-1  | [1:81:0324/112046.751750:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
eola-chrome-1  | [1:81:0324/112046.751824:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
eola-chrome-1  | [1:81:0324/112046.751899:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory
eola-chrome-1  | [1:81:0324/112046.751934:ERROR:bus.cc(399)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory

Versions

Software	Version(s)
Chromium	108.0.5359.125

Additional context

This whitelisted-ips setting also sounded promising but I don't think that's it #198

[a-teisseire]

Until the Chromium team fixes this issue upstream, and for those who need a hacky workaround, you can always use socat (or anything similar) to forward to the debugging port which is only exposed on localhost inside the container:

wrap.sh:

#!/bin/sh

chromium-browser --headless=new --remote-debugging-port=9221 "$@" &
socat TCP-LISTEN:9222,fork,reuseaddr TCP:127.0.0.1:9221

Dockerfile:

FROM zenika/alpine-chrome:112

USER root
RUN apk add --no-cache socat
USER chrome

COPY wrap.sh .

ENTRYPOINT ["sh", "wrap.sh"]

When running the container, I can reach http://localhost:9222/json/version on my machine.

{
   "Browser": "Chrome/112.0.5615.165",
   "Protocol-Version": "1.3",
   "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
   "V8-Version": "11.2.214.14",
   "WebKit-Version": "537.36 (@c262f36e6b1d711ee42d4fbe1343b49960593f18)",
   "webSocketDebuggerUrl": "ws://127.0.0.1:9222/devtools/browser/bb3e6520-fa06-48a3-a8cd-26ac02fcf2c4"
}

The good thing is that the field webSocketDebuggerUrl seems to automatically get the right port from the request so that it doesn't break the integrations using this discovery mechanism.

[ghost]

It's an old issue, but I have found workaround with host header

const getBrowserWSEndpoint = async () => {
    const cdpVersionUrl = `http://${process.env.CHROME_HEADLESS_HOST}:${process.env.CHROME_HEADLESS_PORT}/json/version`

    const response = await axios.get(cdpVersionUrl, {
        headers: { Host: "127.0.0.1:9222" }
    });

    const browserWSEndpoint = response.data.webSocketDebuggerUrl.replace(
        /127.0.0.1/g,
        process.env.CHROME_HEADLESS_HOST
    );

    return browserWSEndpoint
}

then connect with WebSocket

    import { connect } from 'puppeteer';
    const browserWSEndpoint = await getBrowserWSEndpoint();
    const browser = await connect({
        browserWSEndpoint
    });

docker-compose service

  alpine-chrome:
      image: zenika/alpine-chrome:119
      container_name: alpine-chrome
      command:
        - --remote-debugging-address=0.0.0.0
        - --remote-debugging-port=9222
        - --no-sandbox
        - --disable-gpu

if you are using kubernetes

    spec:
      containers:
        - name: headless-chrome-backend
          image: zenika/alpine-chrome:119
          command:
            - /usr/bin/chromium-browser
            - --headless
            - --no-sandbox
            - --remote-debugging-address=0.0.0.0
            - --remote-debugging-port=9222
            - --disable-gpu
          ports:
            - containerPort: 9222
          livenessProbe:
            httpGet:
              path: /
              port: 9222
              httpHeaders:
                - name: host
                  value: 127.0.0.1

[epicwhale]

@mrm-ot I am not quite able to follow what your solution does if the connection is failing from a remote host? Spoofing the host header doesn't seem to be something that's working. Are you running your script from within the same container? would love to understand your fix a bit more!

@sfcgeorge were you able to find an elegant solution in the end?