Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Path mapping #70

Merged
merged 17 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 22 additions & 6 deletions .bazelrc_shared
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
common --announce_rc
common --color=yes
# Fix "Failed to fetch blobs because they do not exist remotely" errors
# https://github.com/bazelbuild/bazel/issues/18696#issuecomment-2175561503
common --remote_download_all
# TODO: remove this when we move to Bazel 8. This is the default in Bazel 8.
common --experimental_remote_cache_eviction_retries=5

# TODO: enable path mapping when we move to Bazel 8. There is a bug in 7. See:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once we enable path mapping, we should remove the outgoing transition on the scala_* rules.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I'll add that here as part of the comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

# https://github.com/bazelbuild/bazel/issues/23990
# In the meantime, if you need to test path mapping you should disable multiplex
# workers with the following build option:
#build --noworker_multiplex
#common --experimental_output_paths=strip
#common --modify_execution_info=CppCompile=+supports-path-mapping

build --experimental_strict_action_env
build --javacopt="-source 21 -target 21"
Expand All @@ -11,14 +18,23 @@ build --java_runtime_version="remotejdk_21"
build --tool_java_language_version="21"
build --tool_java_runtime_version="remotejdk_21"

build --strategy=ScalaCompile=worker
build --strategy=worker,sandboxed,local

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean we haven't been using the non-ScalaCompile workers as workers until now?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure to be honest. The doc says it is remote,worker,sandboxed,local by default, but the default in the code is blank. I figured I'd set it just to be safe.

https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/exec/ExecutionOptions.java#L63-L74

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense.


build --worker_max_instances=4
build --worker_sandboxing
build --experimental_worker_multiplex_sandboxing
# Disable multiplex sandboxing because there is a bug that causes files which
# should be in the sandbox to not be. Not sure if this is a Bazel bug or a rule
# set bug. Something we need to deal with either way.
# Once we enable path mapping, we should also be able to get rid of the outgoing
# transition that we're currently relying on to prevent an explosion in the number
# of builds.
#build --experimental_worker_multiplex_sandboxing
build --experimental_worker_cancellation
build --verbose_failures

build --experimental_worker_sandbox_hardening
# This is disabled as it was causing some issues with some javac targets when used
# with path mapping. If that is resolved, then we can turn it back on.
#build --experimental_worker_sandbox_hardening
build --experimental_use_hermetic_linux_sandbox

test --test_output=all
Expand Down
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
7.4.0
7.4.1
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-24.04]
bazel_version: [bazelbuild/7.4.0]
bazel_version: [bazelbuild/7.4.1]
steps:
- uses: actions/checkout@v3
- uses: bazel-contrib/[email protected]
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ load("@rules_scala_annex//rules/scala:workspace.bzl", "scala_register_toolchains
scala_repositories()
load("@annex//:defs.bzl", annex_pinned_maven_install = "pinned_maven_install")
annex_pinned_maven_install()
scala_register_toolchains(default_scala_toolchain_name = "zinc_3")
scala_register_toolchains(default_scala_toolchain_name = "annex_zinc_3")

load("@rules_scala_annex//rules/scalafmt:workspace.bzl", "scalafmt_default_config", "scalafmt_repositories")
scalafmt_repositories()
Expand Down Expand Up @@ -116,7 +116,7 @@ protobuf_deps()
# Specify the scala compiler we wish to use; in this case, we'll use the default one specified in rules_scala_annex
bind(
name = "default_scala",
actual = "@rules_scala_annex//src/main/scala:zinc_2_13",
actual = "@rules_scala_annex//src/main/scala:annex_zinc_2_13",
)
```

Expand Down
102 changes: 57 additions & 45 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@ workspace(name = "rules_scala_annex")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

# rules_license
rules_license_tag = "1.0.0"

http_archive(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are we using this for?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dependency depended upon this and without setting the version explicitly things were failing. I'm very much looking forward to moving to bzlmod, so we don't have to play this game as much.

name = "rules_license",
sha256 = "26d4021f6898e23b82ef953078389dd49ac2b5618ac564ade4ef87cced147b38",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_license/releases/download/{tag}/rules_license-{tag}.tar.gz".format(tag = rules_license_tag),
"https://github.com/bazelbuild/rules_license/releases/download/{tag}/rules_license-{tag}.tar.gz".format(tag = rules_license_tag),
],
)

# skylib

bazel_skylib_tag = "1.7.1"
Expand All @@ -21,11 +33,26 @@ load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")

bazel_skylib_workspace()

# rules_java
http_archive(
name = "rules_java",
sha256 = "a9690bc00c538246880d5c83c233e4deb83fe885f54c21bb445eb8116a180b83",
urls = [
"https://github.com/bazelbuild/rules_java/releases/download/7.12.2/rules_java-7.12.2.tar.gz",
],
)

load("@rules_java//java:repositories.bzl", "rules_java_dependencies")

rules_java_dependencies()

register_toolchains("//:repository_default_toolchain_21_definition")

# com_github_bazelbuild_buildtools

buildtools_tag = "7.1.2"
buildtools_tag = "7.3.1"

buildtools_sha256 = "4c63e823e6944c950401f92155416c631a65657dd32e1021451fc015faf22ecb"
buildtools_sha256 = "118602587d5804c720c1617db30f56c93ec7a2bdda5e915125fccf7421e78412"

http_archive(
name = "com_github_bazelbuild_buildtools",
Expand All @@ -47,48 +74,12 @@ http_archive(
urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.16/rules_cc-0.0.16.tar.gz"],
)

# io_bazel_rules_go

rules_go_tag = "v0.43.0"

rules_go_sha256 = "8e968b5fcea1d2d64071872b12737bbb5514524ee5f0a4f54f5920266c261acb"

http_archive(
name = "io_bazel_rules_go",
integrity = "sha256-1qtrV+SMCVI+kwUPE2mPcIQoz9XmGSUuNp03evZZdwc=",
urls = [
"https://storage.googleapis.com/bazel-mirror/github.com/bazelbuild/rules_go/releases/download/{tag}/rules_go-{tag}.zip".format(tag = rules_go_tag),
"https://github.com/bazelbuild/rules_go/releases/download/{tag}/rules_go-{tag}.zip".format(tag = rules_go_tag),
],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.17")

# rules_java
http_archive(
name = "rules_java",
sha256 = "a9690bc00c538246880d5c83c233e4deb83fe885f54c21bb445eb8116a180b83",
urls = [
"https://github.com/bazelbuild/rules_java/releases/download/7.12.2/rules_java-7.12.2.tar.gz",
],
)

load("@rules_java//java:repositories.bzl", "rules_java_dependencies")

rules_java_dependencies()

register_toolchains("//:repository_default_toolchain_21_definition")

# rules_jvm_external
rules_jvm_external_tag = "6.1"
rules_jvm_external_tag = "6.5"

http_archive(
name = "rules_jvm_external",
sha256 = "42a6d48eb2c08089961c715a813304f30dc434df48e371ebdd868fc3636f0e82",
sha256 = "c5b27928eeb8f0761f0805540587660f0ecc6946e9c04bf0d4c89e2f7d332b2b",
strip_prefix = "rules_jvm_external-{}".format(rules_jvm_external_tag),
url = "https://github.com/bazelbuild/rules_jvm_external/archive/{}.zip".format(rules_jvm_external_tag),
)
Expand All @@ -110,7 +101,7 @@ load("@annex//:defs.bzl", annex_pinned_maven_install = "pinned_maven_install")

annex_pinned_maven_install()

scala_register_toolchains(default_scala_toolchain_name = "zinc_2_13")
scala_register_toolchains(default_scala_toolchain_name = "annex_zinc_2_13")

# Scala 2.12

Expand Down Expand Up @@ -185,6 +176,27 @@ load("@rules_proto//proto:setup.bzl", "rules_proto_setup")

rules_proto_setup()

# rules_go

rules_go_tag = "v0.43.0"

rules_go_sha256 = "d6ab6b57e48c09523e93050f13698f708428cfd5e619252e369d377af6597707"

http_archive(
name = "io_bazel_rules_go",
sha256 = rules_go_sha256,
urls = [
"https://storage.googleapis.com/bazel-mirror/github.com/bazelbuild/rules_go/releases/download/{tag}/rules_go-{tag}.zip".format(tag = rules_go_tag),
"https://github.com/bazelbuild/rules_go/releases/download/{tag}/rules_go-{tag}.zip".format(tag = rules_go_tag),
],
)

load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")

go_rules_dependencies()

go_register_toolchains(version = "1.17")

# Stardoc

http_archive(
Expand All @@ -210,11 +222,11 @@ stardoc_pinned_maven_install()

# rules_pkg

rules_pkg_version = "1.0.0"
rules_pkg_version = "1.0.1"

http_archive(
name = "rules_pkg",
sha256 = "cad05f864a32799f6f9022891de91ac78f30e0fa07dc68abac92a628121b5b11",
sha256 = "d20c951960ed77cb7b341c2a59488534e494d5ad1d30c4818c736d57772a9fef",
urls = [
"https://github.com/bazelbuild/rules_pkg/releases/download/{v}/rules_pkg-{v}.tar.gz".format(v = rules_pkg_version),
],
Expand All @@ -225,11 +237,11 @@ load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
rules_pkg_dependencies()

# rules_python - this is needed by rules_jvm_external for some reason
rules_python_tag = "0.33.2"
rules_python_tag = "0.36.0"

http_archive(
name = "rules_python",
sha256 = "e3f1cc7a04d9b09635afb3130731ed82b5f58eadc8233d4efb59944d92ffc06f",
sha256 = "ca77768989a7f311186a29747e3e95c936a41dffac779aff6b443db22290d913",
strip_prefix = "rules_python-{}".format(rules_python_tag),
url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(rules_python_tag, rules_python_tag),
)
Expand Down
14 changes: 7 additions & 7 deletions docs/newdocs/scala_versions.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ compiler_classpath_2_13 = [
runtime_classpath_2_13 = ["@scala_library_2_13//jar"]

register_bootstrap_toolchain(
name = "bootstrap_2_13",
name = "annex_bootstrap_2_13",
compiler_classpath = compiler_classpath_2_13,
runtime_classpath = runtime_classpath_2_13,
version = "2.13.14",
Expand All @@ -46,7 +46,7 @@ register_bootstrap_toolchain(
scala_library(
name = "compiler_bridge_2_13",
srcs = ["@compiler_bridge_2_13//:src"],
scala_toolchain_name = "bootstrap_2_13",
scala_toolchain_name = "annex_bootstrap_2_13",
visibility = ["//visibility:public"],
deps = compiler_classpath_2_13 + [
"@scala_annex_org_scala_sbt_compiler_interface//jar",
Expand All @@ -56,7 +56,7 @@ scala_library(

# This augments the configuration to configure the zinc compiler
register_zinc_toolchain(
name = "zinc_2_13",
name = "annex_zinc_2_13",
compiler_bridge = ":compiler_bridge_2_13",
compiler_classpath = compiler_classpath_2_13,
runtime_classpath = runtime_classpath_2_13,
Expand All @@ -72,8 +72,8 @@ load("@rules_scala_annex//rules/scala:workspace.bzl", "scala_register_toolchains
...

scala_register_toolchains(
toolchains = ["//:bootstrap_2_13", "//:zinc_2_13"],
default_scala_toolchain_name = "zinc_2_13",
toolchains = ["//:annex_bootstrap_2_13", "//:annex_zinc_2_13"],
default_scala_toolchain_name = "annex_zinc_2_13",
)

...
Expand All @@ -90,13 +90,13 @@ For example:
scala_library(
name = "example_compiled_with_scalac",
srcs = glob(["**/*.scala"])
scala_toolchain_name = "bootstrap_2_13",
scala_toolchain_name = "annex_bootstrap_2_13",
)

scala_library(
name = "example_compiled_with_zinc",
srcs = glob(["**/*.scala"])
scala_toolchain_name = "zinc_2_13",
scala_toolchain_name = "annex_zinc_2_13",
)

# This would use the default toolchain, which we configured via `scala_register_toolchains` above
Expand Down
2 changes: 1 addition & 1 deletion docs/stardoc/scala_workspace.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ scala_repositories(<a href="#scala_repositories-java_launcher_version">java_laun

| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="scala_repositories-java_launcher_version"></a>java_launcher_version | <p align="center"> - </p> | `"7.4.0"` |
| <a id="scala_repositories-java_launcher_version"></a>java_launcher_version | <p align="center"> - </p> | `"7.4.1"` |
| <a id="scala_repositories-java_launcher_template_sha"></a>java_launcher_template_sha | <p align="center"> - </p> | `"ee4aa47ae5e639632c67be5cc0ccbc4e941a67a1b884a1ce0c4329357a4b62b2"` |


Loading
Loading