feat: auto ubuntu packages download for local browsers #1036
Conversation
| @@ -8,7 +8,6 @@ export default { | |||
| 79: 707231, | |||
| 80: 722374, | |||
| 81: 737198, | |||
| 82: 750023, | |||
There was a problem hiding this comment.
Removed as API is not responding to request "which version of chromedriver should i use with this version of chrome": https://chromedriver.storage.googleapis.com/LATEST_RELEASE_82
| name: Collect ubuntu browser dependencies | ||
| on: | ||
| schedule: | ||
| - cron: 0 0 1 * * |
| @@ -9,10 +9,11 @@ | |||
| ], | |||
| "scripts": { | |||
| "build": "tsc --build && npm run copy-static && npm run build-bundles", | |||
| "copy-static": "copyfiles 'src/browser/client-scripts/*' build", | |||
| "copy-static": "copyfiles 'src/browser/client-scripts/*' 'src/**/*.json' build", | |||
There was a problem hiding this comment.
Also copy these autogenerated json
| const [chromeDriverPath] = await Promise.all([ | ||
| installChromeDriver(chromeVersion), | ||
| installChrome(chromeVersion), | ||
| shouldInstallUbuntuPackageDependencies ? installUbuntuPackageDependencies() : null, |
There was a problem hiding this comment.
Ensure ubuntu packages are ready before spawning child process
| return await Promise.all([ | ||
| installChrome(browserVersion, { force }), | ||
| shouldInstallWebDriver && installChromeDriver(browserVersion, { force }), | ||
| needToInstallUbuntuPackages && installUbuntuPackageDependencies(), |
There was a problem hiding this comment.
Ensure ubuntu package are ready (e.g. for devtools)
| await fs.outputJSON(this._sharedObjectsMapPath, sortObject(sharedObjectsMap), { spaces: 4 }); | ||
| await fs.outputJSON(this._processedBrowsersCachePath, sortObject(processedBrowsers), { spaces: 4 }); |
There was a problem hiding this comment.
Using 4 spaces because the linter wants it
| // Those are couldn't be seen with readelf -d | ||
| export const EXTRA_FIREFOX_SHARED_OBJECTS = ["libdbus-glib-1.so.2", "libXt.so.6"]; |
There was a problem hiding this comment.
Usually there is a note in dynamic section of an elf file "which libraries does it load", but for a long time, starting from the very first versions, firefox needs these 2 shared objects, and they are not recorded in dynamic section of an elf file
|
|
||
| logger.log(`Fetched ${browserVersions.length} browser milestones`); | ||
|
|
||
| const browsersToDownload = cache.filterProcessedBrowsers(browserVersions); |
There was a problem hiding this comment.
Browsers are only downloaded once
If we would like to add new ubuntu release, we would not need to download any browser
| return "libc6"; | ||
| } | ||
|
|
||
| const relevantPackageName = _.minBy(packages, packageName => calcLevenshtein(sharedObject, packageName)) as string; |
There was a problem hiding this comment.
Using levenshtein in order to resolve shared objects to their main packages, and not just to some package, which is more popular (in example, if libnss3.so is used in firefox, it would return "libnss3" even if firefox is more popular)
There was a problem hiding this comment.
Directory with ubuntu binaries call wrappers
| @@ -0,0 +1,43 @@ | |||
| name: Collect ubuntu browser dependencies | |||
There was a problem hiding this comment.
Scheduled action:
- Collects ubuntu dependencies for new browser versions
- Creates a PR to add dependency if there is any new library
- If there are no extra dependencies, do nothing.
- If there is extra browser, but no new dependencies, creates PR to update cache, so it would not need to download the browser later
KuznetsovRoman
force-pushed
the
TESTPLANE-352.ubuntu_deps
branch
3 times, most recently
from
64a1db9
to
dbacdc9
Compare
KuznetsovRoman
force-pushed
the
TESTPLANE-352.ubuntu_deps
branch
from
dbacdc9
to
6f9aeaf
Compare
CI side (collect-ubuntu-browser-dependencies, is not being built):
Cache is stored in VCS and is splitted to 2 files:
Client side (src/browser-installer/ubuntu-packages, being packed):
LD_LIBRARY_PATHwhen launching webdriver so it could populate to browser, so browser would use downloaded packages