feat: auto browser download #1029

KuznetsovRoman · 2024-11-13T01:22:26Z

What is done

Added ability to automatically download browsers and/or webdrivers for these browsers.

Supported browsers to download:

chrome > 73
firefox

Supported drivers to download:

chromedriver (for chrome)
geckodriver (for firefox)
edgedriver (for edge)

With "--local" (or gridUrl: "local") and webdriver protocol, webdiver instances are automatically run for these browsers:

chrome (>73)
firefox
edge (if the browser itself is installed. Driver will be installed automatically)
safari (mac only)

When launching tests with "--local" (and optionally webdriver protocol) all supported browsers/drivers will be installed automatically, if necessary. With "--local" we at all cost are trying to avoid network requests to google/mozilla/... servers.

Example

Set up a testplane config, using webdriver protocol and describe some chrome/firefox browsers with defined versions
Launch eigher using npx testplane --local, either by setting "gridUrl": "local"

In order to just download browser, npx testplane install-browsers could be used. This way we also check if our browsers are up-to-date, so if we have 114.0.5696.0 downloaded and user wants [email protected], we check for newest [email protected] version and see it is "114.0.5735.133". Then this new version is downloaded

If used without arguments, all of the config browsers will be installed
If argument is browserId from config, the corresponding browser will be installed
If argument is @, it will be installed.
These can be combined: npx testplane install-browsers my-chrome chrome@115

Directory to save testplane browsers and drivers can be set with TESTPLANE_BROWSERS_PATH env variable. By default, ~/.testplane is used.

KuznetsovRoman · 2024-11-13T01:23:20Z

package.json

@@ -54,6 +54,7 @@
    "@babel/code-frame": "7.24.2",
    "@gemini-testing/commander": "2.15.4",
    "@jspm/core": "2.0.1",
+    "@puppeteer/browsers": "2.4.0",


download chrome

download chromium (replaces chrome for versions >73, <115)

download firefox

KuznetsovRoman · 2024-11-13T01:25:50Z

package.json

@@ -64,12 +65,16 @@
    "bluebird": "3.5.1",
    "chalk": "2.4.2",
    "clear-require": "1.0.1",
+    "cli-progress": "3.12.0",


multiple progress bar

KuznetsovRoman · 2024-11-13T01:26:22Z

package.json

    "debug": "2.6.9",
    "devtools": "8.39.0",
+    "edgedriver": "5.6.1",


download edge driver

KuznetsovRoman · 2024-11-13T01:27:02Z

package.json

    "error-stack-parser": "2.1.4",
    "expect-webdriverio": "3.6.0",
+    "extract-zip": "2.0.1",


extract zip archive with chromedriver, which is installed manually (>73, <115)

KuznetsovRoman · 2024-11-13T01:27:21Z

package.json

    "fastq": "1.13.0",
    "fs-extra": "5.0.0",
+    "geckodriver": "4.5.0",


download and run geckodriver

KuznetsovRoman · 2024-11-13T01:46:23Z

src/browser-installer/utils.ts

+    if (platform === BrowserPlatform.MAC_ARM && Number(milestone) < MIN_CHROMIUM_MAC_ARM_VERSION) {
+        return BrowserPlatform.MAC;
+    }


for chrome, use mac64 platform, if chromium version is old and mac_arm version does not exist

KuznetsovRoman · 2024-11-13T01:47:04Z

src/browser-installer/utils.ts

+export const getFirefoxBrowserDir = (): string => getBrowsersDir(); // path is set by @puppeteer/browsers.install
+export const getChromeBrowserDir = (): string => getBrowsersDir(); // path is set by @puppeteer/browsers.install
+
+export const retryFetch = async (


As we fetch some data from github, retries wont hurt

KuznetsovRoman · 2024-11-13T01:47:45Z

src/browser-pool/basic-pool.ts

@@ -26,10 +28,11 @@ export class BasicPool implements Pool {

        this._activeSessions = {};
        this._cancelled = false;
+        this._wdPool = new WebdriverPool();


all browsers of the same browser-pool share WebdriverPool,
which is passed by reference to NewBrowser instance

KuznetsovRoman · 2024-11-13T01:48:59Z

src/browser-pool/webdriver-pool.ts

+type ChildProcessWithStatus = { process: ChildProcess; gridUrl: string; isBusy: boolean };
+export type WdProcess = { gridUrl: string; free: () => void; kill: () => void };
+
+export class WebdriverPool {


We have N browsers.
This class handles webdriver processes:

we can reuse existing webdriver process, which is free because browser, which requested webdriver process, no longer needs it (browser.quit is called)

we need to launch new webdriver process (if all existing webdriver processes are busy and we need to launch NewBrowser)

WebdriverPool can reuse webdriver process between browsers with different browserId, if their browser and browserVersion matches

port in this class acts like process id as it is unique

KuznetsovRoman · 2024-11-13T01:49:54Z

src/browser/new-browser.ts

            warn(`WARNING: Can not close session: ${(e as Error).message}`);
+            this._wdProcess?.kill();


If session can't be closed, we kill webdriver process, so later we could launch the new one

KuznetsovRoman · 2024-11-13T10:45:10Z

src/browser/new-browser.ts

+        if (this._isLocalGridUrl() && config.automationProtocol === "webdriver") {
+            gridUrl = await this._getLocalWebdriverGridUrl();
+        } else {
+            // if automationProtocol is not "webdriver", fallback to default grid url from "local"
+            // because in "devtools" protocol we dont need gridUrl, but it still has to be valid URL
+            gridUrl = config.gridUrl === LOCAL_GRID_URL ? DEFAULT_GRID_URL : config.gridUrl;
+        }


"isLocalGridUrl" === either "gridUrl" is "local", either testplane is launched with --local

If isLocalGridUrl == true and automationProtocol === "webdriver", we get webdriver process and use its gridUrl

if isLocalGridUrl == true and automationProtocol is "devtools", gridUrl won't be used either way, but we need to pass any valid url

KuznetsovRoman · 2024-11-13T10:45:57Z

src/browser/new-browser.ts

+        return this._isLocalGridUrl()
+            ? this._addExecutablePath(config, capabilitiesWithAddedHeadless)


Here we don't check for automationProtocol, so we could use our binaries for "devtools" automation protocol

KuznetsovRoman · 2024-11-13T10:47:36Z

src/browser/new-browser.ts

+        const executablePath = await installBrowser(
+            this._config.desiredCapabilities?.browserName as SupportedBrowser,
+            this._config.desiredCapabilities?.browserVersion as string,
+        );


Installs browser (if necessary) and returns its executable path
For safari and edge null is returned (we won't set binary path in capabilities, but wdio will be able to find them, if they are installed)

KuznetsovRoman · 2024-11-13T10:56:55Z

src/browser-installer/chrome/browser.ts

+            unpack: true,
+        }).then(result => result.executablePath);
+
+    return installBinary(Browser.CHROME, platform, buildId, installFn);


registry.installBinary does not mean we will call "installFn" function.
If multiple "installBinary" are called with same browser platform and buildId, "installFn" would only be called at most once. And if we already have it installed, "installFn" wouldn't be called

src/cli/commands/install-browsers/index.ts

sipayRT · 2024-11-14T13:26:50Z

src/browser-installer/install.ts

+            await installEdgeDriver(browserVersion, { force });
+        }
+
+        return null;


why do we return null here but not binary (as for chrome/firefox)

Because we can't download edge or safari. We return null so NewBrowser wouldn't set desiredCapabilities.
With safari and edge currently, as we are using globally installed browser version, we don't need to specify binaryPath. This will change after edge autodownload support though.

src/browser-installer/install.ts

sipayRT · 2024-11-18T08:29:06Z

src/browser-installer/chrome/browser.ts

@@ -0,0 +1,61 @@
+import { resolveBuildId, canDownload, install as puppeteerInstall } from "@puppeteer/browsers";
+import { MIN_CHROME_FOR_TESTING_VERSION } from "../constants";


min_supported_chrome_version?

No.
We have these chrome versions:

MIN_CHROMIUM_VERSION (73)

MIN_CHROMIUM_MAC_ARM_VERSION (93)

MIN_CHROME_FOR_TESTING_VERSION (113)

So there are 3 ranges:

[73, 92]. Minimal chrome to use. Chromium is used. (mac x64 version for mac arm, because there is no mac arm for these versions)

[93, 112]. Chromium is still used, but now Mac_arm version for mac.

113+. Chrome for testing is used.

sipayRT · 2024-11-18T08:32:18Z

src/browser-installer/chrome/browser.ts

+    const milestone = getMilestone(version);
+
+    if (Number(milestone) < MIN_CHROME_FOR_TESTING_VERSION) {
+        browserInstallerDebug(`couldn't install chrome@${version}, installing chromium instead`);


we need to inform user about min supported version here. it's not clear why we are going to install chromium instead of chrome

It is not user warning log. It won't be logged out to the user, and user don't need to distinguish between "chromium" and "chrome for testing" It is debug log.

sipayRT · 2024-11-18T15:01:38Z

test/src/browser-installer/chrome/index.ts

+        getPortStub = sandbox.stub().resolves(12345);
+        waitPortStub = sandbox.stub().resolves();
+
+        runChromeDriver = proxyquire("../../../../src/browser-installer/chrome", {


it might be better to test this functionality with integration tests. There are too many stubs right now

Yes, everything is stubbed, because it is not important at chrome/index.ts level
We only need to know installChromeDriver is called/not called here, as well as spawn is called/not called.

installChromeDriver is tested in another test file, and none of those stubs contains any important logic

I still think those tests do test chrome/index.ts functionality (and nothing more)

test/src/browser-installer/registry.ts

test/src/browser-installer/run.ts

sipayRT · 2024-11-18T15:13:37Z

test/src/browser-installer/run.ts

+
+    afterEach(() => sandbox.restore());
+
+    [true, false, undefined].forEach(debug => {


sipayRT · 2024-11-18T15:17:20Z

test/src/cli/commands/install-browsers/index.ts

+        installBrowsersWithDriversStub = sandbox.stub();
+
+        cli = proxyquire("../../../../../src/cli", {
+            [path.resolve(process.cwd(), "src/cli/commands/install-browsers")]: proxyquire(


no, you can't do that. Double proxyquire nesting is too much :)

Just copied it from here: https://github.com/gemini-testing/testplane/blob/master/test/src/cli/commands/list-tests/index.ts#L39-L50

Is there any better way?

yes - integration test ;)

KuznetsovRoman commented Nov 13, 2024

View reviewed changes

KuznetsovRoman force-pushed the TESTPLANE-269.auto_browsers branch 2 times, most recently from a3693cd to 58c27d2 Compare November 13, 2024 11:08

sipayRT requested changes Nov 18, 2024

View reviewed changes

sipayRT approved these changes Nov 25, 2024

View reviewed changes

KuznetsovRoman added 4 commits November 26, 2024 03:05

feat: auto browser download

2d1bb74

fix: apply edits

9b0b642

fix: convert browser name to lower case when applying headless

d6fa184

fix: apply edits

61f8403

KuznetsovRoman force-pushed the TESTPLANE-269.auto_browsers branch from 7e5fe69 to 61f8403 Compare November 26, 2024 00:05

KuznetsovRoman merged commit 8229698 into master Nov 26, 2024
2 checks passed

KuznetsovRoman deleted the TESTPLANE-269.auto_browsers branch November 26, 2024 00:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: auto browser download #1029

feat: auto browser download #1029

KuznetsovRoman commented Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

KuznetsovRoman Nov 13, 2024

sipayRT Nov 14, 2024

KuznetsovRoman Nov 18, 2024

sipayRT Nov 18, 2024

KuznetsovRoman Nov 18, 2024 •

edited

Loading

sipayRT Nov 18, 2024

KuznetsovRoman Nov 18, 2024 •

edited

Loading

sipayRT Nov 18, 2024

KuznetsovRoman Nov 18, 2024

sipayRT Nov 18, 2024

sipayRT Nov 18, 2024

KuznetsovRoman Nov 18, 2024

sipayRT Nov 19, 2024

		warn(`WARNING: Can not close session: ${(e as Error).message}`);
		this._wdProcess?.kill();

		return this._isLocalGridUrl()
		? this._addExecutablePath(config, capabilitiesWithAddedHeadless)

		@@ -0,0 +1,61 @@
		import { resolveBuildId, canDownload, install as puppeteerInstall } from "@puppeteer/browsers";
		import { MIN_CHROME_FOR_TESTING_VERSION } from "../constants";


		afterEach(() => sandbox.restore());

		[true, false, undefined].forEach(debug => {

feat: auto browser download #1029

feat: auto browser download #1029

Conversation

KuznetsovRoman commented Nov 13, 2024 • edited Loading

What is done

Example

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

KuznetsovRoman Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KuznetsovRoman commented Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 13, 2024 •

edited

Loading

KuznetsovRoman Nov 18, 2024 •

edited

Loading

KuznetsovRoman Nov 18, 2024 •

edited

Loading