Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for JetBrains Runtime #637

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open

Conversation

gmitch215
Copy link

@gmitch215 gmitch215 commented Jun 5, 2024

Current Workaround

- name: Setup JBR 21
  uses: gmitch215/setup-java@4e56c31b28053b7081e102121153633227d6dd8e
  with:
    distribution: 'jetbrains'
    java-version: 21
    cache: 'gradle'
  # For GitHub API (use if you believe you'd run into rate limits)
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Description:
This PR adds support for the JetBrains Runtime, which is required in dependencies like jewel and useful for Compose Multiplatform.

The PR uses the binaries listed in Binaries for Developers , using the JBRSDK flavor as shown in the releases page for the default flavor jdk. Alternative flavors include jdk+jcef and jdk+ft with jre counterparts.

It uses the GitHub API to find and parse versions, then calculates the URL using cache-redirector.jetbrains.com with the parsed SemVer, OS, Arch, and build number, which all output a .tar.gz file. Left a few comments in there for maintainability. Some tags have inconsistent naming conventions, or have been removed from public view for various reasons, so it will filter out those versions.

Since this uses the GitHub API, you can optionally specify a GITHUB_TOKEN environmental variable to increase your rate limit.

Related issue:

Check list:

  • Mark if documentation changes are required.
  • Mark if tests were added or updated to cover the changes.

Notes
Tested using jetbrains-installer.test.ts. Most of the tests were copied from other installers, but some I couldn't figure out how to bring over, so additional work may be required. I also copied downloadTool from the Temurin installer.

I also executed npm run build for dist/setup/index.js. I think something else was also thrown because I was on TypeScript 5.3, so I'm not sure how that will affect ncc.

- Add Installer, Models
- Includes Tests & Test Manifest Data
- Add to `e2e-versions.yml`
- Run `npm run build`
- Update README.md
@gmitch215 gmitch215 requested a review from a team as a code owner June 5, 2024 20:55
@Goooler
Copy link

Goooler commented Jun 19, 2024

I'm using your branch as a workaround

  # TODO: replace this once https://github.com/actions/setup-java/pull/637 gets merged.
  - uses: gmitch215/setup-java@07ea605e4b1f0f8b0621155fb1722d6094c3e183
    with:
      distribution: 'jetbrains'
      java-version: 17

with an error:

Installed distributions
  Error: No supported distribution was found for input jetbrains

is there anything I can do?

See:

@gmitch215
Copy link
Author

Hmm, I'll debug and get back here later.

@gmitch215
Copy link
Author

Seems I just forgot to add it to the map of recognized runtimes to their IDs. Update your commit hash and it should succeed.

My project passed with the following configuration:

      - name: Setup JBR 17
        uses: gmitch215/setup-java@7f3ab2648f3d20faa20c4fe6e52a455757e0f38a
        with:
          distribution: 'jetbrains'
          java-version: 17
          cache: 'gradle'

Output:
image

@Goooler
Copy link

Goooler commented Jun 19, 2024

It could be picked correctly now!

But seems some libs are missing in this JDK installed.

Execution failed for task ':checkRuntime'.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
> Failed to check JDK distribution: 'jlink', 'jpackage' are missing

  JDK distribution path: /Users/runner/hostedtoolcache/Java_JetBrains_jdk/17.0.11-1312.2/arm64/Contents/Home

See https://github.com/Goooler/kotlin-explorer/actions/runs/9575738587/job/26401058613.

@Goooler
Copy link

Goooler commented Jun 19, 2024

The JDK installed by Gradle toolchain worked fine.

Invalid Java installation found at '/Users/runner/.gradle/jdks/jetbrains_s_r_o_-17-aarch64-os_x/jbrsdk_jcef-17.0.11-osx-aarch64-b1207.24/Contents/Home' (Auto-provisioned by Gradle). It will be re-checked in the next build. This might have performance impact if it keeps failing. Run the 'javaToolchains' task for more details.

Seems we need Binaries for developers instead of Binaries for launching IntelliJ IDEA, which is installed by your action. Check https://github.com/JetBrains/JetBrainsRuntime/releases/tag/jbr-release-17.0.11b1312.2.

@gmitch215
Copy link
Author

gmitch215 commented Jun 19, 2024

I switched the URL to the ones listed in the Binaries for Developers section, still getting the same errors. Going to do some more research.

@gmitch215
Copy link
Author

gmitch215 commented Jun 19, 2024

I guessed correctly! :checkRuntime seems to pass.

      - name: Setup JBR 17
        uses: gmitch215/setup-java@99fc2135f7f7b08068e180cb5f29340f9de70720
        with:
          distribution: 'jetbrains'
          java-version: 17
          cache: 'gradle'

(This action failed due to below)

image

Note: Running on windows-latest seems to fail, so I'll fix that too.

@Goooler
Copy link

Goooler commented Jun 19, 2024

99fc213 works for me now, someone who needs this can ref romainguy/kotlin-explorer#62.

Thanks for your quick response!

@gmitch215
Copy link
Author

Got windows-latest to build now. Everything else works fine too.

      - name: Setup JBR 17
        uses: gmitch215/setup-java@e1f4c15facc0cf68c0e764340e374be034e4a859
        with:
          distribution: 'jetbrains'
          java-version: 17
          cache: 'gradle'

@eskatos
Copy link

eskatos commented Jul 6, 2024

Thank you @gmitch215, I'm using your branch as a workaround too and can confirm it works fine!
It would be great if this contribution could be accepted.

@eskatos
Copy link

eskatos commented Jul 8, 2024

On a relatively complex workflow with a fan-out setup and composite actions that parallelize builds I get rate limit failures from time to time:

Run gmitch215/setup-java@36531813787dd0ff3269feeb1144b35b3d68f927
  with:
    java-version: 21
    java-package: jdk
    distribution: jetbrains
    check-latest: false
    server-id: github
    server-username: GITHUB_ACTOR
    server-password: GITHUB_TOKEN
    overwrite-settings: true
    job-status: success
    token: ***
Installed distributions
  Trying to resolve the latest version from remote
  Error: API rate limit exceeded for 52.159.14[7](https://github.com/eskatos/private-repository/actions/runs/1234567890/job/1234567890#step:3:8).63. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

Is there something that can be done in this action's implementation or on my side?

@gmitch215
Copy link
Author

gmitch215 commented Jul 9, 2024

@eskatos The JetBrains cache-redirector server does rate limit by IP. To my knowledge, there isn't a way to change the rate limit. The root URL's info page is internal as well, and only accessible by employees.

Your best bet is to do something that changes the IP (not sure if re-running it manually does this, but worth a shot). Unless someone knows how the runner could provide authentication.

Edit: This error page is about the GitHub API, which is something I need to add. However, the above is still true.

Edit 2: Done.

- name: Setup JBR 21
  uses: gmitch215/setup-java@b4d76a354bef7187244f74eea4e80e674a7b4b55
  with:
    distribution: 'jetbrains'
    java-version: 21
    cache: 'gradle'

@eskatos
Copy link

eskatos commented Jul 9, 2024

Re-running did make the job pass indeed.

This error page is about the GitHub API, which is something I need to add.

Thank you for fixing this!

@eskatos
Copy link

eskatos commented Jul 9, 2024

With b4d76a3 I'm getting this:

  Trying to resolve the latest version from remote
  Error: Bad credentials

@gmitch215
Copy link
Author

gmitch215 commented Jul 9, 2024

@eskatos I forgot GitHub doesn't automatically inject the GITHUB_TOKEN environment variable on action workflows.

The action should now check if it is provided. Update to this:

- name: Setup JBR 21
  uses: gmitch215/setup-java@6d2c5e1f82f180ae79f799f0ed6e3e5efb4e664d
  with:
    distribution: 'jetbrains'
    java-version: 21
    cache: 'gradle'
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

@Jonatha1983
Copy link

Hi, @gmitch215 - thanks for your work!!!
Small question: for now, we should keep using

uses: gmitch215/setup-java@6d2c5e1f82f180ae79f799f0ed6e3e5efb4e664d

Till this is merged?

Thanks again

@gmitch215
Copy link
Author

Hi, @gmitch215 - thanks for your work!!!

Small question: for now, we should keep using


uses: gmitch215/setup-java@6d2c5e1f82f180ae79f799f0ed6e3e5efb4e664d

Till this is merged?

Thanks again

Sure. The most recent commit for the workaround is available in the top message of this PR.

@gmitch215
Copy link
Author

@HarithaVattikuti been 2 weeks since we last spoke, any updates?

@mahabaleshwars
Copy link
Contributor

Hi @gmitch215, a few checks are failing in the PR:

Build the project using Prettier: Ensure that you run Prettier to format the code properly.

JDK 11 Naming Convention Issue: When using JDK 11, the system expects the version to be in the format 11_0_16b2043.64 instead of 11.0.16-b2043.64. This discrepancy is causing the current failure.

Please update the PR once these issues are resolved.

- Fix JDK 11 URL Bug
- Add JDK URL Testing to ensure versions can be downloaded
- Run Prettier
@gmitch215
Copy link
Author

@mahabaleshwars Issues fixed. I also added a test suite that checks if the URLs return 403 (access denied, meaning not found) instead of 200/302.

@mahabaleshwars
Copy link
Contributor

Hello @gmitch215,

Many of the checks are still failing, including those for JDK 11. Additionally, some links requested for JDK 11 are not available. For example, when tried on a Windows runner, the following URL was requested from the workflow: https://cache-redirector.jetbrains.com/intellij-jbr/jbrsdk_jcef-11.0.13-windows-x64-b1504.49.tar.gz, but it is not available at the source. Could you please rectify these issues and update the PR?

Before committing, please run the following commands in your project to ensure proper formatting and build success:

npm run format
npm run build
npm run test
These commands will handle Prettier formatting and verify the distribution (Dist).

Thank you!

@gmitch215
Copy link
Author

Switched to plain JBRSDK and fixed the URL.

@aparnajyothi-y
Copy link
Contributor

Hello @gmitch215, Many of the checks are still failing, including those for JDK 11. Please review the changes on your end and address these issues. Once updated, kindly update the PR.

@mahabaleshwars
Copy link
Contributor

Hello @gmitch215,

Could you please confirm if the issues have been resolved and the PR has been updated? We would like to proceed with testing and approval once it's ready.

@gmitch215
Copy link
Author

The issues haven't been resolved yet - I'm still working on trying to figure out what the issues are with JDK 11 specifically. I'll get back to you once I've completed it.

- Add `GITHUB_TOKEN` environment variable for JetBrains requests
- Add `jetbrains` to other E2E tests
@gmitch215
Copy link
Author

Try this build. I also added some more tests to e2e-versions.yml along with including the GITHUB_TOKEN to prevent rate limits.

@gmitch215
Copy link
Author

gmitch215 commented Nov 7, 2024

It appears that sometimes JetBrains will make certain URLs inaccessible for some reason, so I check with HEAD requests to make sure they return a 200. Running npm run test with the default method didn't show any significant speed differences.

@gmitch215
Copy link
Author

This one should work. There is an additional _nomod suffix for some versions, and it now properly filters out invalid versions.

I also added some new package types:

  • jdk for jbrsdk
  • jdk+jcef for jbrsdk with JCEF
  • jdk+ft for jbrsdk (FreeType)

Along with its JRE counterparts.

There are also some GitHub API-related issues in e2e-versions.yml. This should include GITHUB_TOKEN as an environmental variable.

@gmitch215
Copy link
Author

gmitch215 commented Nov 11, 2024

Final issues were rate limits and issues e2e-versions.yml. Added GITHUB_TOKEN environmental variable to those issues.

The fail on the general validation tests on macos-latest is a GraalVM issue and is not relevant to this PR.

Edit: All JetBrains checks have passed. This PR should be considered complete.

@gmitch215
Copy link
Author

@aparnajyothi-y @mahabaleshwars Any updates on getting this approved now that all the checks have passed?

@mahabaleshwars
Copy link
Contributor

Hi @gmitch215,

The updated PR resolves most issues, but there are some version-specific discrepancies as per this documentation :

Tests fail for certain versions (e.g., 11_0_15) but work for others (e.g., 11.0.15).
Users may input versions with underscores (e.g., 11_0_15) instead of dots (e.g., 11.0.15), which should be addressed.
Some versions (e.g., 11.0.10) do not work, while others (e.g., 11.0.11) do.
It would be beneficial to update the setup-java documentation to reflect these scenarios, especially the version formatting and compatibility issues.

@gmitch215
Copy link
Author

Hi @gmitch215,

The updated PR resolves most issues, but there are some version-specific discrepancies as per this documentation :

Tests fail for certain versions (e.g., 11_0_15) but work for others (e.g., 11.0.15). Users may input versions with underscores (e.g., 11_0_15) instead of dots (e.g., 11.0.15), which should be addressed.

I am a little confused. Are you saying 11_0_15 doesn't work but 11.0.15 does? They're all based on the GitHub tags the bot publishes. I'll validate whether underscores are inputted, since that behavior isn't intended, but that behavior should be expected.

Some versions (e.g., 11.0.10) do not work, while others (e.g., 11.0.11) do. It would be beneficial to update the setup-java documentation to reflect these scenarios, especially the version formatting and compatibility issues.

JetBrains seems to have removed 11.0.10 from their releases page, despite being available via URL. I'll have to add it manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JDK support distribution JBR (jetbrains runtime version) Support JetBrainsRuntime distribution
7 participants