Use latest Firefox version with Selenium 2 on GitHub Actions #391
Comments
|
@aik099 @mvorisek @stof unfortunately, the request to update the firefox-se2 image was understandably not so successful. I think I could be able make such a docker image with the pointers they provided, but that leaves a few open questions:
|
|
Probably at least 244mb - that's the current size of the image. That's well below GitHub's container registry limits.
Understood. To answer your question from the other issue, my first thought is to create a new repo with just a workflow that generates and pushes such an image to the GitHub registry, which we can use later. |
|
That's good news. So GitHub has a place to store Docker images and we have image-building instructions from @SeleniumHQ. This way we can have mini-version of Selenium Docker images without our own tagging system. Interesting sub-project. Some docs I've found: Not sure if we should clone https://github.com/SeleniumHQ/docker-selenium/tree/trunk or do something else. |
|
I've playing around with Docker to see if I can manage to build an image using instructions from SeleniumHQ/docker-selenium#2454 (comment) . The build succeeded with below patching instructions: Then you can create a container using just built image and don't forget to map 4444:4444 port in that container. The selenium server in container does start, but when I attempt to operate on Firefox from it I'll get timeout error: The error suggests, that I haven't properly matched Selenium Server to the Geckodriver version and Firefox version (as said in https://www.edureka.co/community/5548/unable-to-connect-to-host-127-0-0-1-on-port-7055-after-45000-ms ). The same thing happens if I try to use A super easy reproduction example would be to create this And then run This will get you the most recent Firefox version (88.0), that is compatible with this os. Probably I need to install something to make it work, but I have no idea what exactly. |
|
Sorry to hear it didn't work out, but good to see some progress nevertheless. What confuses me in your (final) instructions is that it seems to upgrade firefox after running |
It's work in progress. At least it's not a compilation error, that I saw at beginning.
What is wrong wrong with this? You mean we're not controlling what Firefox version is used and getting the latest compatible to the OS? Did another try to build an image with same Firefox/geckodriver versions as it's supposed to be built. It doesn't work either, but the error is different: Now I've tried looking at the build warnings I've received and turned out some of |
(Selenium 2) Firefox 88.0.1 and GeckoDriver 0.29.1 on 2.53.1-berylliumResults with Firefox 88.0.1 and GeckoDriver 0.29.1 (actually used by Selenium 4 image: https://hub.docker.com/layers/selenium/standalone-firefox/88.0.1-geckodriver-0.29.1-20210513/images/sha256-84603a539c99ceb2ece196f0e6c4c7665b9b11bf6d48d4bd9e0015f694b50d58 ): From Docker container logs: (Selenium 2) Firefox 132.0.1 and GeckoDriver 0.35.0 on 3.141.59-zirconiumI've also tried using This gives a bit different error, but problem is the same ( (Selenium 3) Firefox 132.0.1 and GeckoDriver 0.35.0 on 3.141.59-zirconiumBoth errors exist in the official Since Selenium Server 2 can talk with the latest Firefox and the latest geckodriver I don't understand why it doesn't when used in a Docker container. (Selenium 2) Firefox 132.0 and Geckodriver 0.35.0 on 2.53.1-berylliumDocker logs using debug image: While connected via VNC I've seen, that Selenium does start Firefox and then does nothing for 45 seconds and then closes firefox (or firefox crashes).
|
|
another option is to stop using a docker image for that in our CI process, installing Selenium and Firefox separately in the GHA runner (and Firefox is even pre-installed in the runner image: https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md#browsers-and-drivers) |
We did that in the past (in Travis CI times). I don't remember why we've decided to abandon that approach in favor of a selenium Docker image. |
|
I think we did that in GHA because it was simpler to setup (but this is simpler only as long as we have maintained images) |
You know that the runner is basically docker? You can just copy&paste those commands into a new Dockerfile and we're ready to go. The selenium repo setup might have been nice if it was quick and fast, but it turns out it isn't. That doesn't mean we throw away all (selenium) images and docker even, just because of that.
Uhm, let's see...
|
|
If Selenium 2/3 won't work on the latest GitHub Actions machine, then we're forced to stay with Docker. Here is the summary of my reanimation experiments on Selenium 2 Docker images:
I even was able to run Selenium 3 inside a @uuf6429 , any idea how to overcome this problem? |
|
@aik099 I'm sorry I didn't have time yet to deep dive into this. Yesterday I tried a simple dockerfile (based on your short example above) and was trying to analyse the last layer ( I'm well aware of Docker's advantages - it's a far cry from vagrant and pre-built VMs, but it's also not the most polished tool out there. I deeply resent that, especially the way they treat their users, but I'm not aware of a good alternative. |
|
@uuf6429 , were you able to get Docker image build with latest Firefox and Selenium 2 and see these errors? Here is latest patch to be applied over diff --git a/Base/Dockerfile b/Base/Dockerfile
index 7d326cf6..05a3d5ff 100644
--- a/Base/Dockerfile
+++ b/Base/Dockerfile
@@ -20,6 +20,10 @@ RUN apt-get update -qqy \
sudo \
unzip \
wget \
+ libasound2 \
+ tzdata \
+ pciutils \
+ libglu1-mesa-dev \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/* \
&& sed -i 's/securerandom\.source=file:\/dev\/random/securerandom\.source=file:\/dev\/urandom/' ./usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/java.security
diff --git a/Hub/Dockerfile.txt b/Hub/Dockerfile.txt
index 99295231..828004d0 100644
--- a/Hub/Dockerfile.txt
+++ b/Hub/Dockerfile.txt
@@ -6,14 +6,16 @@ MAINTAINER Selenium <[email protected]>
EXPOSE 4444
-ENV GRID_NEW_SESSION_WAIT_TIMEOUT -1
-ENV GRID_JETTY_MAX_THREADS -1
-ENV GRID_NODE_POLLING 5000
-ENV GRID_CLEAN_UP_CYCLE 5000
-ENV GRID_TIMEOUT 30000
-ENV GRID_BROWSER_TIMEOUT 0
-ENV GRID_MAX_SESSION 5
-ENV GRID_UNREGISTER_IF_STILL_DOWN_AFTER 30000
+EXPOSE 4445
+
+ENV GRID_NEW_SESSION_WAIT_TIMEOUT=-1
+ENV GRID_JETTY_MAX_THREADS=-1
+ENV GRID_NODE_POLLING=5000
+ENV GRID_CLEAN_UP_CYCLE=5000
+ENV GRID_TIMEOUT=30000
+ENV GRID_BROWSER_TIMEOUT=0
+ENV GRID_MAX_SESSION=5
+ENV GRID_UNREGISTER_IF_STILL_DOWN_AFTER=30000
COPY generate_config /opt/selenium/generate_config
COPY entry_point.sh /opt/bin/entry_point.sh
diff --git a/NodeBase/Dockerfile.txt b/NodeBase/Dockerfile.txt
index 47efc8fc..d442e177 100644
--- a/NodeBase/Dockerfile.txt
+++ b/NodeBase/Dockerfile.txt
@@ -1,13 +1,14 @@
MAINTAINER Selenium <[email protected]>
-ENV DEBIAN_FRONTEND noninteractive
-ENV DEBCONF_NONINTERACTIVE_SEEN true
+ENV DEBIAN_FRONTEND=noninteractive
+ENV DEBCONF_NONINTERACTIVE_SEEN=true
#===================
# Timezone settings
# Possible alternative: https://github.com/docker/docker/issues/3359#issuecomment-32150214
#===================
-ENV TZ "US/Pacific"
+ENV TZ="US/Pacific"
+
RUN echo "${TZ}" > /etc/timezone \
&& dpkg-reconfigure --frontend noninteractive tzdata
@@ -31,10 +32,10 @@ RUN chmod +x /opt/bin/entry_point.sh
#============================
# Some configuration options
#============================
-ENV SCREEN_WIDTH 1360
-ENV SCREEN_HEIGHT 1020
-ENV SCREEN_DEPTH 24
-ENV DISPLAY :99.0
+ENV SCREEN_WIDTH=1360
+ENV SCREEN_HEIGHT=1020
+ENV SCREEN_DEPTH=24
+ENV DISPLAY=:99.0
USER seluser
diff --git a/NodeChrome/Dockerfile.txt b/NodeChrome/Dockerfile.txt
index ba518e9f..dedf1edb 100644
--- a/NodeChrome/Dockerfile.txt
+++ b/NodeChrome/Dockerfile.txt
@@ -36,9 +36,9 @@ RUN wget --no-verbose -O /tmp/chromedriver_linux64.zip https://chromedriver.stor
#========================
# Selenium Configuration
#========================
-ENV NODE_MAX_INSTANCES 1
-ENV NODE_MAX_SESSION 1
-ENV NODE_REGISTER_CYCLE 5000
+ENV NODE_MAX_INSTANCES=1
+ENV NODE_MAX_SESSION=1
+ENV NODE_REGISTER_CYCLE=5000
COPY generate_config /opt/selenium/generate_config
RUN chmod +x /opt/selenium/generate_config
diff --git a/NodeChromeDebug/Dockerfile b/NodeChromeDebug/Dockerfile
index 3280c229..b614950b 100644
--- a/NodeChromeDebug/Dockerfile
+++ b/NodeChromeDebug/Dockerfile
@@ -20,8 +20,8 @@ RUN apt-get update -qqy \
#=================
# Locale settings
#=================
-ENV LANGUAGE en_US.UTF-8
-ENV LANG en_US.UTF-8
+ENV LANGUAGE=en_US.UTF-8
+ENV LANG=en_US.UTF-8
RUN locale-gen en_US.UTF-8 \
&& dpkg-reconfigure --frontend noninteractive locales \
&& apt-get update -qqy \
diff --git a/NodeDebug/Dockerfile.txt b/NodeDebug/Dockerfile.txt
index 045d2f8a..3a435ae7 100644
--- a/NodeDebug/Dockerfile.txt
+++ b/NodeDebug/Dockerfile.txt
@@ -15,8 +15,14 @@ RUN apt-get update -qqy \
#=================
# Locale settings
#=================
-ENV LANGUAGE en_US.UTF-8
-ENV LANG en_US.UTF-8
+ENV LANGUAGE=en_US.UTF-8
+ENV LANG=en_US.UTF-8
+
+RUN apt-get update -qqy \
+ && apt-get -qqy --no-install-recommends install \
+ locales \
+ && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
+
RUN locale-gen en_US.UTF-8 \
&& dpkg-reconfigure --frontend noninteractive locales \
&& apt-get update -qqy \
diff --git a/NodeFirefox/Dockerfile.txt b/NodeFirefox/Dockerfile.txt
index 95ed318c..c279059f 100644
--- a/NodeFirefox/Dockerfile.txt
+++ b/NodeFirefox/Dockerfile.txt
@@ -5,7 +5,10 @@ USER root
#=========
# Firefox
#=========
-ARG FIREFOX_VERSION=47.0.1
+#ARG FIREFOX_VERSION=47.0.2
+#ARG FIREFOX_VERSION=50.0.2
+#ARG FIREFOX_VERSION=88.0.1
+ARG FIREFOX_VERSION=130.0
RUN apt-get update -qqy \
&& apt-get -qqy --no-install-recommends install firefox \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/* \
@@ -20,7 +23,9 @@ RUN apt-get update -qqy \
#============
# GeckoDriver
#============
-ARG GECKODRIVER_VERSION=0.10.0
+#ARG GECKODRIVER_VERSION=0.10.0
+#ARG GECKODRIVER_VERSION=0.29.1
+ARG GECKODRIVER_VERSION=0.35.0
RUN wget --no-verbose -O /tmp/geckodriver.tar.gz https://github.com/mozilla/geckodriver/releases/download/v$GECKODRIVER_VERSION/geckodriver-v$GECKODRIVER_VERSION-linux64.tar.gz \
&& rm -rf /opt/geckodriver \
&& tar -C /opt -zxf /tmp/geckodriver.tar.gz \
@@ -33,9 +38,9 @@ RUN wget --no-verbose -O /tmp/geckodriver.tar.gz https://github.com/mozilla/geck
#========================
# Selenium Configuration
#========================
-ENV NODE_MAX_INSTANCES 1
-ENV NODE_MAX_SESSION 1
-ENV NODE_REGISTER_CYCLE 5000
+ENV NODE_MAX_INSTANCES=1
+ENV NODE_MAX_SESSION=1
+ENV NODE_REGISTER_CYCLE=5000
COPY generate_config /opt/selenium/generate_config
RUN chmod +x /opt/selenium/generate_config \
&& chown -R seluser:seluser /opt/selenium
diff --git a/NodeFirefoxDebug/Dockerfile.txt b/NodeFirefoxDebug/Dockerfile.txt
index 815156f3..4b5d72e6 100644
--- a/NodeFirefoxDebug/Dockerfile.txt
+++ b/NodeFirefoxDebug/Dockerfile.txt
@@ -16,8 +16,9 @@ RUN apt-get update -qqy \
#=================
# Locale settings
#=================
-ENV LANGUAGE en_US.UTF-8
-ENV LANG en_US.UTF-8
+ENV LANGUAGE=en_US.UTF-8
+ENV LANG=en_US.UTF-8
+
RUN locale-gen en_US.UTF-8 \
&& dpkg-reconfigure --frontend noninteractive locales \
&& apt-get update -qqy \
diff --git a/Standalone/Dockerfile.txt b/Standalone/Dockerfile.txt
index efbbdbf6..23dfab20 100644
--- a/Standalone/Dockerfile.txt
+++ b/Standalone/Dockerfile.txt
@@ -11,3 +11,5 @@ RUN chmod +x /opt/bin/entry_point.sh
USER seluser
EXPOSE 4444
+
+EXPOSE 4445
diff --git a/StandaloneChromeDebug/Dockerfile b/StandaloneChromeDebug/Dockerfile
index e5780cd1..ea0fdc23 100644
--- a/StandaloneChromeDebug/Dockerfile
+++ b/StandaloneChromeDebug/Dockerfile
@@ -20,8 +20,8 @@ RUN apt-get update -qqy \
#=================
# Locale settings
#=================
-ENV LANGUAGE en_US.UTF-8
-ENV LANG en_US.UTF-8
+ENV LANGUAGE=en_US.UTF-8
+ENV LANG=en_US.UTF-8
RUN locale-gen en_US.UTF-8 \
&& dpkg-reconfigure --frontend noninteractive locales \
&& apt-get update -qqy \
diff --git a/Test/Dockerfile b/Test/Dockerfile
index bbb98751..3a24e5c6 100644
--- a/Test/Dockerfile
+++ b/Test/Dockerfile
@@ -1,5 +1,5 @@
FROM node:4.5.0-onbuild
MAINTAINER Selenium <[email protected]>
-ENV TEST_CMD ""
+ENV TEST_CMD=""
# The remainder of this build will be completed by the upstream image's ONBUILD commands
Here are the latest instructions: |
I've used Docker Desktop and "SSHed" into the container using Bash to see what's happening inside any trying to do different stuff. |
|
The idea behind dive/analysis was to see exactly what changed in the Mostly because I was confused from what you said, that running Anyway, I'll give this a look later on today. |
Upgrading existing packages doesn't break anything OS-wide (the new Firefox version after the upgrade works by itself). It's something that Selenium Server 2 does to communicate with Firefox, which causes a problem on Ubuntu specifically. We can almost be certain about this because Selenium Server 3 is working just fine with the same Firefox version on the same container, whereas Selenium Server 2 doesn't. I know for sure, that Selenium Server 2 on macOS does work with Firefox 130.0.2. This proves once again, that it's a Ubuntu-specific issue, that needs to be uncovered.
Maybe it was a typo (usage of |
No problem at all. On that point, that's also an aspect of Docker that I like - I tend to avoid manual changes/adjustments and keep iterating on the Dockerfile since in theory it should guarantee consistency (and I know that I can simply share that file with other people). |
|
I've set up a mostly working (builds & stuff connects) Dockerfile, however something seems to break (seems consistent with the earlier comment). Here are the test results:
This seems consistent with claims like: https://groups.google.com/g/selenium-developers/c/d8KpXpmDp2g?pli=1 as well as the releases notes for geckodriver v0.15.0 and v0.16.0. That said, there were reports in the past that the port 7055 problem could be related to unsigned extensions or so - I haven't played with that yet. Here's my FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update -y && \
apt-get install -y \
wget \
curl \
unzip \
openjdk-8-jdk \
ca-certificates \
libx11-6 \
libx11-dev \
libxext6 \
libxrender1 \
libgtk-3-0 \
libdbus-1-3 \
libasound2 \
libnss3 \
libgdk-pixbuf2.0-0 \
libdbus-glib-1-2 \
x11vnc \
xvfb \
xterm \
git \
python3 \
python3-pip \
libpci-dev \
libegl1-mesa \
&& rm -rf /var/lib/apt/lists/*
# Install noVNC
RUN git clone https://github.com/novnc/noVNC.git /opt/noVNC && \
ln -s /opt/noVNC/vnc.html /opt/noVNC/index.html
# Install Firefox
ENV FIREFOX_VERSION=95.0.1
RUN wget -q "https://ftp.mozilla.org/pub/firefox/releases/${FIREFOX_VERSION}/linux-x86_64/en-US/firefox-${FIREFOX_VERSION}.tar.bz2" -O /tmp/firefox.tar.bz2 && \
tar -xjf /tmp/firefox.tar.bz2 -C /opt && \
mv /opt/firefox /opt/firefox-${FIREFOX_VERSION} && \
ln -s /opt/firefox-${FIREFOX_VERSION}/firefox /usr/local/bin/firefox
# Install GeckoDriver
ENV GECKODRIVER_VERSION=v0.35.0
RUN wget -q "https://github.com/mozilla/geckodriver/releases/download/${GECKODRIVER_VERSION}/geckodriver-${GECKODRIVER_VERSION}-linux64.tar.gz" -O /tmp/geckodriver.tar.gz && \
tar -xzf /tmp/geckodriver.tar.gz -C /usr/local/bin && \
chmod +x /usr/local/bin/geckodriver
# Install Selenium
ENV SELENIUM_VERSION=2.53.1
RUN wget -q "https://github.com/SeleniumHQ/selenium/releases/download/selenium-${SELENIUM_VERSION}/selenium-server-standalone-${SELENIUM_VERSION}.jar" -O /opt/selenium-server.jar
ENV SELENIUM_PORT=4444
ENV VNC_PORT=5900
ENV NOVNC_PORT=7900
# Expose ports for VNC and noVNC
EXPOSE ${SELENIUM_PORT} ${VNC_PORT} ${NOVNC_PORT}
ENV DISPLAY=:1
#ENV MOZ_HEADLESS=1
#ENV MOZ_WEBRENDER=0
#ENV MOZ_DISABLE_GPU=1
#ENV MOZ_ENABLE_WAYLAND=1
# Start VNC server and Selenium server, and run noVNC
CMD [ "sh", "-c", "\
/usr/bin/Xvfb ${DISPLAY} -screen 0 1280x1024x24 & \
x11vnc -display ${DISPLAY} -forever -shared -nopw -ncache 10 -rfbport ${VNC_PORT} & \
/opt/noVNC/utils/novnc_proxy --vnc localhost:${VNC_PORT} --listen ${NOVNC_PORT} & \
java -jar /opt/selenium-server.jar -port ${SELENIUM_PORT} --debug"]
PS: Installing Footnotes |
|
Hmm, I've found some additional details:
It's unclear to me in which case geckodriver <0.15.0 would have made sense. I think at this point I'd consider simply disabling (or ignoring-failures of) the firefox-on-selenium2 build (or at least, the failing D&D test). The chrome build is good enough and it should be clear to end users that selenium 2 is greatly outdated and therefore there will be some lack of support. |
|
Maybe all these restrictions apply on Ubuntu (Linux)? I've justed executed test I'm wondering why this works on both Mac and Windows, but not on Linux. |
|
I don't fully understand your setup. So you mean that behat/mink + selenium is on your local machine and Firefox is on Windows in browserstack? Would it be possible to find me the exact capabilities being sent? |
return new Selenium2Driver(
'firefox',
array(
'build' => 'MinkSelenium2Driver Tests - Selenium2 Setting Detection',
'browserstack.local' => 'true',
'browserstack.localIdentifier' => 'nativeapp:...',
'acceptInsecureCerts' => true,
'version' => 131,
),
'http://username:[email protected]/wd/hub'
);
The desired capabilities, that BrowserStack receives are as follows: |
|
I tried a few combinations and ideas without success. Finally, I tried a Docker-less setup on my Windows machine with Selenium 2, Firefox 132 and Geckodriver 0.35.0 and I pretty much got the same error (Firefox did indeed start): |
|
Double-checking on Windows 11 (in VM) locally + Selenium 2 + newer Firefox + Geckodriver 0.35.0 resulted in the same 45000 ms timeout. So obviously BrowserStack is doing something to make Selenium 2 work with newer Firefox versions, that regular developers can't do. This means we shouldn't even support Selenium 2 with newer Firefox browser versions, because it can't work. @uuf6429 , Let's instead focus on attempting to exclude that We can likely skip that test based on the |
* Support MinkSelenium2Driver page timeout * Trigger deprecation of old-style page timeout * Replace duplicate code with comment * Test deprecations * Rework custom tests to not depend on magic setup * Set up standard defaults to avoid contaminated sessions * [debug] Only run tests that fail * [debug] Adjusted PHPUnit 10 test filtering expression to work on PHPUnit 9 * [debug] remove irrelevant (for debugging) build config sections to speed it up * [debug] Reduce matrix size even more * Don't let driver define Selenium Server default timeouts * [debug] confirm, that timeout tests executed at the beginning would break window tests down the road * Revert "[debug] confirm, that timeout tests executed at the beginning would break window tests down the road" This reverts commit b33289b. * [debug] see if Selenium actually sets timeouts as needed * Don't reset timeout at the test end, because new driver instance is created in setup * [debug] PhpStan fix * [debug] see if session with incorrect timeout is shared between tests * Fixed formatting error * [debug] see if deprecation catching causes timeout issues * [debug] see of sleepy page causes a timeout issue * [debug] change data provider declaration of the timeouts test * [debug] removed debug output in the driver code, because not timeouts are causing the issue with window test * [debug] Look inside GitHub Actions with an SSH session * [debug] start web server outside of PHPUnit bootstrap file * [debug] remove unrelated build config parts * [debug] corrected host of locally started mink server * [debug] corrected hostname for started web server checking * [debug] another try to get local web server running * [debug] reduce test count to figure out why web server is not working * [debug] Experiment with WEB_FIXTURES_HOST env var detection * [debug] another try to debug what URL is actually used by test suite * [debug] another try * Allow overriding env vars from "phpunit.xml.dist" via env vars from GitHub Actions * [debug] removed some of the debug code * [debug] change local web server detection code to avoid it's crashing in the checking process * [debug] try to make web server available for Selenium Server * [debug] undo experimental port mapping * [debug] attempt to start web sever on all ips * [debug] rolled back CI code that reduced build count * [debug] rolled back the debug code, that started web server outside of PHPUnit code * rolled back changes to "phpunit.xml.dist" (to be discussed in other issue) * Partially skip short "pageLoad" timeout test in Google Chrome * Fully skip short "pageLoad" timeout test in Google Chrome * Only skip test added in this PR for Google Chrome * Apparently "chromium" and "edge" browsers also freeze, when setting "pageLoad" timeout several times. So skip test in there as well. * Exclude timeout deprecation test only on GitHub Actions * Minor test fixes and improvements * Disable test on old firefox * Fix minkphp/MinkSelenium2Driver#391 --------- Co-authored-by: Alex <[email protected]>
The https://hub.docker.com/r/selenium/standalone-firefox image provided by Selenium only supports Firefox 47 when used with Selenium server 2.x versions. This causes problems in a test suite because a working code (e.g. dragging a div into itself in #359) fails due:
Need to create an image with a modern Firefox/geckodriver version and use it on GitHub Actions.
The text was updated successfully, but these errors were encountered: