Add ghc version to CI matrix for bindists #1923

avdv · 2023-07-31T13:36:58Z

This PR adds a ghc dimension to the CI build matrix for the "Build & Test - bindist" job and runs jobs for GHC 9.2.5 as well as GHC 9.4.5.

Also, we have to bump the minimal supported Bazel version to 5.3.0 in this PR, since we use an API that was introduced in that version and gating it with a version check was tripping up the Stardoc documentation generation (e.g. see here).

Fixes #1933 and fixes #1900

aherrmann

Thank you for figuring this out, from what I saw that was not easy!
I left a few small comments, and questions for clarification. My main concern is any potential impact on downstream users, or rather to what extent the impact is confined to our CI.

aherrmann · 2023-08-24T14:30:16Z

haskell/private/ghc_ci.bzl

+        print("Using GHC version {} from env variable `GHC_VERSION`".format(ghc_version))
+
+    repository_ctx.file("BUILD")
+    repository_ctx.file("ghc_version.bzl", content = "GHC_VERSION = {}".format(repr(ghc_version)))


what happens if $GHC_VERSION is unset?

In this case the file will contain GHC_VERSION = None. This will cause the default GHC version to be used.

aherrmann · 2023-08-24T14:38:48Z

extensions/ghc_version.bzl

@@ -0,0 +1,15 @@
+""" Module extension which configures the GHC version"""


IIUC it configures the default version. But, if a MODULE.bazel file provides a specific version to the bindist module extension, then it will override this env-var, correct? If so, the docs should be clearer on this.

Correct, this will only effect the default GHC version used, and only for rules_haskell and rules_haskell_tests. Other modules are not effected at all. They will always just get the default GHC version if using version = None and the specified version otherwise.

I have added clearer docs to this file.

extensions/ghc_version.bzl

.bazelrc.common

haskell/private/ghc_ci.bzl

aherrmann · 2023-08-25T07:44:44Z

extensions/rules_haskell_dependencies.bzl

-        build_file_content = """
+    if GHC_VERSION and GHC_VERSION.startswith("9.4."):
+        http_archive(
+            name = "Cabal",


Where is @Cabal all used? From what I can tell it's mostly used for test dependencies and for the worker. So, it shouldn't directly affect users of rules_haskell, correct?

Side note, I noticed that some test-dependencies in rules_haskell's MODULE.bazel, e.g. some stack_snapshots, are not marked as dev-dependencies. It's out of scope for this PR, but something we should fix.

Where is @Cabal all used? From what I can tell it's mostly used for test dependencies and for the worker. So, it shouldn't directly affect users of rules_haskell, correct?

Yes.

Side note, I noticed that some test-dependencies in rules_haskell's MODULE.bazel, e.g. some stack_snapshots, are not marked as dev-dependencies. It's out of scope for this PR, but something we should fix.

Agreed, and I already tried that (setting dev_dependency = True) but that failed with some wild error later, since rules_haskell_tests depended on it.

aherrmann · 2023-08-25T07:46:35Z

extensions/rules_haskell_dependencies.bzl

-    http_archive(
-        name = "Cabal",
-        build_file_content = """
+    if GHC_VERSION and GHC_VERSION.startswith("9.4."):


So long as it's only for our CI, I'm fine with this approach. Generally speaking, a better route would be if we could figure out how to select based on resolved GHC version and then use select to switch to the correct versions. But, that seems quite difficult in this case.

avdv · 2023-09-01T12:34:22Z

@Mergifyio rebase

This makes the default GHC version publicly accessible and introduces a new repository which uses the default GHC version if not already defined.

Fixes #1900

This wrapper adds the `bin` folder of the cc toolchain from GHC's bindist to the `PATH` so that the protoc.exe is able to find its runtime DLLs, e.g. libc++.dll. Note, using a `sh_binary` rule with a cmd script does not create an extra executable contrary to using a Bash script.

These are not needed, since `executable = True` already demands a single executable file

We need to be able to use `--proto_compiler` to specify a custom protoc executable / script. The needed "proto_compiler" configuration field has been introduced in Bazel 5.3.0.

It is always the first one.

mergify · 2023-09-01T12:34:32Z

rebase

✅ Branch has been successfully rebased

avdv · 2023-09-01T19:00:03Z

@Mergifyio refresh

mergify · 2023-09-01T19:00:13Z

refresh

✅ Pull request refreshed

Add ghc version to CI matrix for bindists

avdv force-pushed the ci-matrix-ghc branch 5 times, most recently from 4ad86a7 to ccd7a43 Compare August 1, 2023 13:31

avdv mentioned this pull request Aug 2, 2023

fix include directories passed to hsc2hs #1922

Merged

avdv force-pushed the ci-matrix-ghc branch 3 times, most recently from 83e19de to 87e7f3e Compare August 8, 2023 17:25

avdv force-pushed the ci-matrix-ghc branch 7 times, most recently from 6288a69 to 734ffd4 Compare August 17, 2023 07:21

avdv force-pushed the ci-matrix-ghc branch 7 times, most recently from 907d710 to eb58ed1 Compare August 23, 2023 15:01

avdv requested a review from aherrmann August 24, 2023 07:26

avdv marked this pull request as ready for review August 24, 2023 07:26

aherrmann approved these changes Aug 25, 2023

View reviewed changes

avdv force-pushed the ci-matrix-ghc branch from c1ec1f1 to 7298993 Compare September 1, 2023 09:16

avdv added the merge-queue merge on green CI label Sep 1, 2023

avdv mentioned this pull request Sep 1, 2023

Simplify mergify config #1948

Merged

avdv added 21 commits September 1, 2023 12:34

Use GHC version in bzlmod

0be971c

Define default GHC version

5539b21

This makes the default GHC version publicly accessible and introduces a new repository which uses the default GHC version if not already defined.

Make sure GHC_VERSION is used

ee26fdc

Use protoc-update-bounds branch without submodules

634de02

Load ghc_version in rules_haskell_tests module too

9172508

Use Cabal 3.8.1 with GHC 9.4

f44b13a

Only use GHC_VERSION variable for rules_haskell modules

23789d3

Do not reverse order of plugins for GHC >= 9.4.1

f1acc5d

Fixes #1900

Remove allow_single_file parameter for protoc and plugin

9a8e464

These are not needed, since `executable = True` already demands a single executable file

Make the start script select the right stack snapshot

18cbb11

Add missing links

de47ca9

Make protoc able to be overridden by --proto_compiler

86c50d3

Set --proto_compiler to //tests:protoc.cmd on Windows

a4271eb

Pass on runtime manifests for protoc

d2b7117

Format

466d7ec

Bump minimal Bazel version to 5.3

569c7b2

We need to be able to use `--proto_compiler` to specify a custom protoc executable / script. The needed "proto_compiler" configuration field has been introduced in Bazel 5.3.0.

Pass on runtime manifests for protoc and plugin

9ced5ed

README: Add section about protoc errors on Windows to Troubleshooting

d755fad

Simplify determining the root module

cfbe06a

It is always the first one.

Clarify semantics of the ghc_default_version module extension

6821989

avdv force-pushed the ci-matrix-ghc branch from 7298993 to 6821989 Compare September 1, 2023 12:34

mergify bot merged commit 37e48d1 into master Sep 1, 2023
104 checks passed

mergify bot deleted the ci-matrix-ghc branch September 1, 2023 19:00

mergify bot removed the merge-queue merge on green CI label Sep 1, 2023

avdv pushed a commit that referenced this pull request Sep 4, 2023

Merge pull request #1923 from tweag/ci-matrix-ghc

f2b2f9f

Add ghc version to CI matrix for bindists

aherrmann mentioned this pull request Sep 26, 2023

Internal GHC version selection exposed to users #1967

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ghc version to CI matrix for bindists #1923

Add ghc version to CI matrix for bindists #1923

avdv commented Jul 31, 2023 •

edited

Loading

aherrmann left a comment

aherrmann Aug 24, 2023

avdv Sep 1, 2023

aherrmann Aug 24, 2023

avdv Sep 1, 2023 •

edited

Loading

aherrmann Aug 25, 2023

avdv Sep 1, 2023

aherrmann Aug 25, 2023

avdv commented Sep 1, 2023

mergify bot commented Sep 1, 2023

avdv commented Sep 1, 2023

mergify bot commented Sep 1, 2023

		@@ -0,0 +1,15 @@
		""" Module extension which configures the GHC version"""

Add ghc version to CI matrix for bindists #1923

Add ghc version to CI matrix for bindists #1923

Conversation

avdv commented Jul 31, 2023 • edited Loading

aherrmann left a comment

Choose a reason for hiding this comment

aherrmann Aug 24, 2023

Choose a reason for hiding this comment

avdv Sep 1, 2023

Choose a reason for hiding this comment

aherrmann Aug 24, 2023

Choose a reason for hiding this comment

avdv Sep 1, 2023 • edited Loading

Choose a reason for hiding this comment

aherrmann Aug 25, 2023

Choose a reason for hiding this comment

avdv Sep 1, 2023

Choose a reason for hiding this comment

aherrmann Aug 25, 2023

Choose a reason for hiding this comment

avdv commented Sep 1, 2023

mergify bot commented Sep 1, 2023

✅ Branch has been successfully rebased

avdv commented Sep 1, 2023

mergify bot commented Sep 1, 2023

✅ Pull request refreshed

avdv commented Jul 31, 2023 •

edited

Loading

avdv Sep 1, 2023 •

edited

Loading