[Enhancement] Properly lock docker ci images chromium version #4459

peterzhuamazon · 2024-02-16T21:37:06Z

Hi,

Starting from here:

[RELEASE] Release version 2.12.0 #4115 (comment)

Looking for ways to lock chromium to version 114 on all LINUX images to replace bundled electron. For example, if there is any chromium cli/binary installation available on npm or other package manager.
https://www.npmjs.com/package/download-chromium

Thanks.

PRs:

The text was updated successfully, but these errors were encountered:

peterzhuamazon · 2024-02-16T21:40:16Z

Have confirmed that this is an outdated package and the version is not chromium version but rather the package version itself and is only having an older version of chromium:
https://www.npmjs.com/package/download-chromium?activeTab=readme

peterzhuamazon · 2024-02-17T00:02:09Z

Try to directly feed an external electron to cypress, and cypress does not like it:


We could not identify a known browser at the path you provided: C:\Users\ContainerAdministrator\scoop\apps\volta\1.0.8\appdata\bin\electron.exe

The output from the command we ran was:

Unable to find browser with path C:\Users\ContainerAdministrator\scoop\apps\volta\1.0.8\appdata\bin\electron.exe
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

peterzhuamazon · 2024-02-19T23:07:42Z

Will fork the download-chromium repo and make some changes so it will run on other platforms.
As of now able to get the windows one running.

peterzhuamazon · 2024-02-19T23:21:38Z

Seems like the official chromium snapshots repo is not maintained since 2018:
https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=win_rel/

peterzhuamazon · 2024-02-20T00:14:12Z

Thank of using thorium to replace chromium:
https://thorium.rocks/

peterzhuamazon · 2024-02-20T00:14:50Z

More snapshots:
https://chromium.woolyss.com/

peterzhuamazon · 2024-02-20T00:16:02Z

This seems better:
https://github.com/macchrome

peterzhuamazon · 2024-02-20T00:31:43Z

Seems like the best way for now:


[ci-runner@272b01270bd0 ~]$ npx @puppeteer/browsers install chrome@114
Need to install the following packages:
  @puppeteer/[email protected]
Ok to proceed? (y) y
Downloading chrome r114.0.5735.133 - 327.3 MB [====================] 100% 0.0s
[email protected] /home/ci-runner/chrome/linux-114.0.5735.133/chrome-linux64/chrome
npm notice
npm notice New major version of npm available! 9.5.1 -> 10.4.0
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.4.0
npm notice Run npm install -g [email protected] to update!
npm notice
[ci-runner@272b01270bd0 ~]$ ./chrome/linux-114.0.5735.133/chrome-linux64/chrome --version
Google Chrome for Testing 114.0.5735.133

https://www.chromium.org/getting-involved/download-chromium/

peterzhuamazon · 2024-02-20T17:42:10Z

Runs well:


[ci-runner@04add00fbd93 opensearch-dashboards-functional-test]$ ./integtest.sh -s true -t customImportMapDashboards -v 2.12.0 -o default -r false
npm WARN exec The following package was not found and will be installed: @puppeteer/[email protected]
Downloading chrome r114.0.5735.133 - 327.3 MB [====================] 100% 0.0s

> [email protected] postinstall
> husky install

husky - Git hooks installed

up to date, audited 370 packages in 864ms

2 moderate severity vulnerabilities

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.
Test Files List:
cypress/integration/plugins/custom-import-map-dashboards/1_import_vector_map_tab.spec.js cypress/integration/plugins/custom-import-map-dashboards/2_opensearchMapLayer.spec.js cypress/integration/plugins/custom-import-map-dashboards/3_add_saved_object.spec.js cypress/integration/plugins/custom-import-map-dashboards/4_documentsLayer.spec.js
BROWSER_PATH: /home/ci-runner/opensearch-dashboards-functional-test/chrome/linux-114.0.5735.133/chrome-linux64/chrome
run security enabled tests
yarn run v1.22.21
$ env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=3000 --browser /home/ci-runner/opensearch-dashboards-functional-test/chrome/linux-114.0.5735.133/chrome-linux64/chrome --spec 'cypress/integration/plugins/custom-import-map-dashboards/*'
[1911:0220/094140.247997:ERROR:gpu_init.cc(453)] Passthrough is not supported, GL is swiftshader, ANGLE is

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        9.5.4                                                                          │
  │ Browser:        Custom Chrome for (headless)                                                   │
  │ Node Version:   v18.19.0 (/home/ci-runner/.nvm/versions/node/v18.19.0/bin/node)                │
  │ Specs:          4 found (plugins/custom-import-map-dashboards/1_import_vector_map_tab.spec.js, │
  │                  plugins/custom-import-map-dashboards/2_opensearchMapLayer.spec.js, plugins/cu │
  │                 stom-import-map-dashboards/3_add_saved_object.spec.js, plugins/custom-import-m │
  │                 ap-dashboards/4_docume...)                                                     │
  │ Searched:       cypress/integration/plugins/custom-import-map-dashboards/*                     │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  plugins/custom-import-map-dashboards/1_import_vector_map_tab.spec.js            (1 of 4)

peterzhuamazon · 2024-02-20T18:00:59Z

There are follow up issues with this approach, seems like there are libs missing which caused:

Linux:


  Running:  plugins/custom-import-map-dashboards/1_import_vector_map_tab.spec.js            (1 of 4)

Timed out waiting for the browser to connect. Retrying...

Timed out waiting for the browser to connect. Retrying again...

The browser never connected. Something is wrong. The tests cannot run. Aborting...

The browser never connected. Something is wrong. The tests cannot run. Aborting...

Windows:


  Verify the presence of import custom map tab in region map plugin
    1) "before all" hook for "checks import custom map tab is present"
    2) "after all" hook for "checks import custom map tab is present"


  0 passing (1m)
  2 failing

  1) Verify the presence of import custom map tab in region map plugin
       "before all" hook for "checks import custom map tab is present":
     AssertionError: Timed out retrying after 60000ms: Expected to find element: `button[data-test-subj="addSampleDataSetecommerce"]`, but never found it.

Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Verify the presence of impo...`
      at MiscUtils.addSampleData (http://localhost:5601/__cypress/tests?p=cypress\integration\plugins\custom-import-map-dashboards\1_import_vector_map_tab.spec.j
s:2574:79)
      at Context.eval (http://localhost:5601/__cypress/tests?p=cypress\integration\plugins\custom-import-map-dashboards\1_import_vector_map_tab.spec.js:171:15)

  2) Verify the presence of import custom map tab in region map plugin
       "after all" hook for "checks import custom map tab is present":
     CypressError: You attempted to make a chai-jQuery assertion on an object that is neither a DOM object or a jQuery object.

The chai-jQuery assertion you used was:

  > visible

The invalid subject you asserted on was:

  > undefined

To use chai-jQuery assertions your subject must be valid.

This can sometimes happen if a previous assertion changed the subject.

peterzhuamazon · 2024-02-20T19:58:44Z

https://chromiumdash.appspot.com/

peterzhuamazon · 2024-02-20T20:10:57Z

Apparently chromium@1262835 which is Chromium 124 passes on the stuck issues here: #4241

npx @puppeteer/[email protected] install chromium@1262835.

peterzhuamazon · 2024-02-20T20:29:32Z

Chromium 124 stuck on the 4/4 steps of customImportMapDashboards, try out Chromium 112 (r1108766).

peterzhuamazon · 2024-02-20T20:39:48Z

Chromium 112:

Linux:


       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  plugins/custom-import-map-dashboard      00:24        1        1        -        -        - │
  │    s/1_import_vector_map_tab.spec.js                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/custom-import-map-dashboard      01:55        1        1        -        -        - │
  │    s/2_opensearchMapLayer.spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/custom-import-map-dashboard      01:31        1        1        -        -        - │
  │    s/3_add_saved_object.spec.js                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/custom-import-map-dashboard      02:44        2        2        -        -        - │
  │    s/4_documentsLayer.spec.js                                                                  │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        06:36        5        5        -        -        -

Windows:



       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  plugins/custom-import-map-dashboard      00:24        1        1        -        -        - │
  │    s/1_import_vector_map_tab.spec.js                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/custom-import-map-dashboard      01:54        1        1        -        -        - │
  │    s/2_opensearchMapLayer.spec.js                                                              │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/custom-import-map-dashboard      01:31        1        1        -        -        - │
  │    s/3_add_saved_object.spec.js                                                                │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  plugins/custom-import-map-dashboard      02:45        2        2        -        -        - │
  │    s/4_documentsLayer.spec.js                                                                  │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        06:36        5        5        -        -        -

Done in 413.45s.

peterzhuamazon · 2024-02-20T22:38:10Z

Seems like puppeteer/browser is still downloading x64 version for arm64:


[ci-runner@112c5b708c1f opensearch-build]$ file /home/ci-runner/opensearch-build/chromium/linux-1108766/chrome-linux/chrome
/home/ci-runner/opensearch-build/chromium/linux-1108766/chrome-linux/chrome: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[xxHash]=f7755b2c384ff2a1, not stripped

peterzhuamazon · 2024-02-20T22:40:20Z

Seems like it has been an issue since 2021:

The chromium binary is not available for arm64 puppeteer/puppeteer#7740

My issues:

The chromium binary is not available for arm64 puppeteer/puppeteer#7740 (comment)

peterzhuamazon · 2024-02-20T23:38:26Z

Seems like a proper solution right here:
puppeteer/puppeteer#7740 (comment)

peterzhuamazon · 2024-02-20T23:40:40Z

https://github.com/puppeteer/puppeteer/releases/tag/puppeteer-core-v19.8.0
chromium: roll to Chromium 112.0.5614.0 (r1108766)

feat(chromium): roll to r1057 (microsoft/playwright#22093)
microsoft/playwright@da3f2c0

Use Playwright version:
https://playwright.azureedge.net/builds/chromium/1057/chromium-linux-arm64.zip

peterzhuamazon · 2024-02-21T00:55:24Z

npx [email protected] install chromium will lock on Chromium 112.0.5615.29

% npx [email protected] install chromium
Need to install the following packages:
  [email protected]
Ok to proceed? (y) y
╔═══════════════════════════════════════════════════════════════════════════════╗
║ WARNING: It looks like you are running 'npx playwright install' without first ║
║ installing your project's dependencies.                                       ║
║                                                                               ║
║ To avoid unexpected behavior, please install your dependencies first, and     ║
║ then run Playwright's install command:                                        ║
║                                                                               ║
║     npm install                                                               ║
║     npx playwright install                                                    ║
║                                                                               ║
║ If your project does not yet depend on Playwright, first install the          ║
║ applicable npm package (most commonly @playwright/test), and                  ║
║ then run Playwright's install command to download the browsers:               ║
║                                                                               ║
║     npm install @playwright/test                                              ║
║     npx playwright install                                                    ║
║                                                                               ║
╚═══════════════════════════════════════════════════════════════════════════════╝
BEWARE: your OS is not officially supported by Playwright; downloading fallback build.
Downloading Chromium 112.0.5615.29 (playwright build v1055) from https://playwright.azureedge.net/builds/chromium/1055/chromium-linux.zip
144.3 Mb [====================] 100% 0.0s
Chromium 112.0.5615.29 (playwright build v1055) downloaded to /home/<>/.cache/ms-playwright/chromium-1055
BEWARE: your OS is not officially supported by Playwright; downloading fallback build.
Downloading FFMPEG playwright build v1008 from https://playwright.azureedge.net/builds/ffmpeg/1008/ffmpeg-linux.zip
2.6 Mb [====================] 100% 0.0s
FFMPEG playwright build v1008 downloaded to /home/<>/.cache/ms-playwright/ffmpeg-1008
(base)
(24-02-21 0:54:26) <0> [~/test]
dev-dsk-zhujiaxi-2a-c8d595b4 % /home/<>/.cache/ms-playwright/chromium-1055/chrome-linux/chrome --version
Chromium 112.0.5615.29

peterzhuamazon · 2024-02-21T01:01:07Z

** Update:
https://github.com/microsoft/playwright/releases/tag/v1.32.3

https://playwright.azureedge.net/builds/chromium/1055/chromium-linux-arm64.zip
https://playwright.azureedge.net/builds/chromium/1055/chromium-linux.zip
https://playwright.azureedge.net/builds/chromium/1055/chromium-win64.zip
https://playwright.azureedge.net/builds/chromium/1055/chromium-mac.zip
https://playwright.azureedge.net/builds/chromium/1055/chromium-mac-arm64.zip

[Feature] installer on Windows On ARM microsoft/playwright#28785 (comment)

/home/<>/.cache/ms-playwright/chromium-1055/chrome-linux-arm64/chrome
/home/<>/.cache/ms-playwright/chromium-1055/chrome-linux/chrome
C:\Users\ContainerAdministrator\AppData\Local\ms-playwright\chromium-1055\chrome.exe

peterzhuamazon · 2024-02-22T00:35:09Z

More changes to the scripts of installing chromium versions on different OSes.

peterzhuamazon · 2024-02-22T19:31:35Z

New setup:


C:/Users/Administrator/opensearch-dashboards-functional-test/chromium/chrome-win/chrome.exe chromium-112 os-win arch-x64

ProductVersion   FileVersion      FileName
--------------   -----------      --------
112.0.5615.29    112.0.5615.29    C:\Users\Administrator\opensearch-dashboards-functional-test\chromium\chrome-win\chrome.exe



/Users/<>/amazonworks/sourcecodes/peterzhuamazon/opensearch-dashboards-functional-test/chromium/chrome-mac/Chromium.app/Contents/MacOS/Chromium chromium-112 os-mac arch-x64
Chromium 112.0.5615.29


/local/home/<>/opensearch-dashboards-functional-test-peterzhuamazon/chromium/chrome-linux/chrome chromium-112 os-linux arch-x64
Chromium 112.0.5615.29


/home/ec2-user/opensearch-dashboards-functional-test/chromium/chrome-linux/chrome chromium-112 os-linux arch-arm64
Chromium 112.0.5615.49

peterzhuamazon added bug Something isn't working integtest test docker ci-test-automation jenkins Jenkins related issue labels Feb 16, 2024

peterzhuamazon self-assigned this Feb 16, 2024

peterzhuamazon added this to OpenSearch Engineering Effectiveness Feb 16, 2024

github-project-automation bot moved this to Backlog in OpenSearch Engineering Effectiveness Feb 16, 2024

github-actions bot added the untriaged Issues that have not yet been triaged label Feb 16, 2024

peterzhuamazon mentioned this issue Feb 16, 2024

[RELEASE] Release version 2.12.0 #4115

Closed

88 tasks

peterzhuamazon removed the untriaged Issues that have not yet been triaged label Feb 16, 2024

peterzhuamazon mentioned this issue Feb 20, 2024

Use @puppeteer/browsers to lock chromium version to 112 opensearch-project/opensearch-dashboards-functional-test#1116

Closed

1 task

peterzhuamazon moved this from Backlog to In Progress in OpenSearch Engineering Effectiveness Feb 21, 2024

peterzhuamazon mentioned this issue Feb 23, 2024

Add download_chromium to integtest.sh for standalone chromium installation opensearch-project/opensearch-dashboards-functional-test#1123

Merged

1 task

peterzhuamazon closed this as completed in opensearch-project/opensearch-dashboards-functional-test#1123 Feb 26, 2024

github-project-automation bot moved this from In Progress to Done in OpenSearch Engineering Effectiveness Feb 26, 2024

peterzhuamazon mentioned this issue Feb 27, 2024

Remove chromium installation as it is installed in opensearch-project/opensearch-dashboards-functional-test#1123 #4480

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Enhancement] Properly lock docker ci images chromium version #4459

[Enhancement] Properly lock docker ci images chromium version #4459

peterzhuamazon commented Feb 16, 2024 •

edited

Loading

peterzhuamazon commented Feb 16, 2024

peterzhuamazon commented Feb 17, 2024

peterzhuamazon commented Feb 19, 2024

peterzhuamazon commented Feb 19, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024 •

edited

Loading

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024 •

edited

Loading

peterzhuamazon commented Feb 21, 2024 •

edited

Loading

peterzhuamazon commented Feb 21, 2024 •

edited

Loading

peterzhuamazon commented Feb 22, 2024

peterzhuamazon commented Feb 22, 2024 •

edited

Loading

[Enhancement] Properly lock docker ci images chromium version #4459

[Enhancement] Properly lock docker ci images chromium version #4459

Comments

peterzhuamazon commented Feb 16, 2024 • edited Loading

peterzhuamazon commented Feb 16, 2024

peterzhuamazon commented Feb 17, 2024

peterzhuamazon commented Feb 19, 2024

peterzhuamazon commented Feb 19, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024 • edited Loading

peterzhuamazon commented Feb 20, 2024

peterzhuamazon commented Feb 20, 2024 • edited Loading

peterzhuamazon commented Feb 21, 2024 • edited Loading

peterzhuamazon commented Feb 21, 2024 • edited Loading

peterzhuamazon commented Feb 22, 2024

peterzhuamazon commented Feb 22, 2024 • edited Loading

peterzhuamazon commented Feb 16, 2024 •

edited

Loading

peterzhuamazon commented Feb 20, 2024 •

edited

Loading

peterzhuamazon commented Feb 20, 2024 •

edited

Loading

peterzhuamazon commented Feb 21, 2024 •

edited

Loading

peterzhuamazon commented Feb 21, 2024 •

edited

Loading

peterzhuamazon commented Feb 22, 2024 •

edited

Loading