Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bash 0.19 no longer builds in Nix #734

Closed
toastal opened this issue Sep 10, 2024 · 6 comments
Closed

Bash 0.19 no longer builds in Nix #734

toastal opened this issue Sep 10, 2024 · 6 comments

Comments

@toastal
Copy link

toastal commented Sep 10, 2024

Describe the bug
Submodule is gone so it doesn’t clone correctly

To Reproduce
Try to build 0.4.0 from Nixpkgs (doesn’t yet exist… trying to update)

Expected behavior
Newer versions seem to at least build, but why hasn’t the lockfile been updated in ages?

Environment

  • OS name + version: NixOS unstable
  • Version of the code: 0.4.0

Additional context
Add any other context about the problem here.

1

Footnotes

  1. Please consider giving up MS GitHub or offering a non-proprietary, non-US-corporate-controlled mirror for this free software project. I wish to delete this Microsoft account in the future, but I need more projects like this to support alternative methods to send patches & contribute.

@toastal
Copy link
Author

toastal commented Sep 10, 2024

   > Cloning into '/nix/store/fxps707007r88i0dp3wb0693626avw2q-tree-sitter-bash-1b0321e/examples/bash-it'...
   > remote: Enumerating objects: 599, done.
   > remote: Counting objects: 100% (599/599), done.
   > remote: Compressing objects: 100% (472/472), done.
   > remote: Total 599 (delta 35), reused 435 (delta 26), pack-reused 0 (from 0)
   > Receiving objects: 100% (599/599), 1.30 MiB | 637.00 KiB/s, done.
   > Resolving deltas: 100% (35/35), done.
   > remote: Total 0 (delta 0), reused 0 (delta 0)
   > error: Server does not allow request for unadvertised object a99d905216cc0aac5de0c3050f4afc54e21c6bc5
   > fatal: Fetched in submodule path 'examples/bash', but it did not contain a99d905216cc0aac5de0c3050f4afc54e21c6bc5. Direct fetching of that commit failed.
   > error: Server does not allow request for unadvertised object a99d905216cc0aac5de0c3050f4afc54e21c6bc5
   > fatal: Fetched in submodule path 'examples/bash', but it did not contain a99d905216cc0aac5de0c3050f4afc54e21c6bc5. Direct fetching of that commit failed.

@toastal
Copy link
Author

toastal commented Sep 10, 2024

   last 25 log lines:
   > running 2 tests
   > test formatted_query_tester ... ok
   > test input_output_tester ... FAILED
   >
   > failures:
   >
   > ---- input_output_tester stdout ----
   > thread 'input_output_tester' panicked at /build/rustc-1.80.1-src/library/core/src/ops/function.rs:250:5:
   > Unexpected failure.
   > code=5
   > stderr=```"[2024-09-10T04:45:30Z ERROR topiary] Parsing error between line 4, column 1 and line 148, column 1\n"```
   > command=`TOPIARY_LANGUAGE_DIR="../topiary-queries/queries/" "/build/source/target/x86_64-unknown-linux-gnu/release/topiary" "fmt" "/build/.tmpkNkHOx/bash.sh"`
   > code=5
   > stdout=""
   > stderr="[2024-09-10T04:45:30Z ERROR topiary] Parsing error between line 4, column 1 and line 148, column 1\n"
   >
   > note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   >
   >
   > failures:
   >     input_output_tester
   >
   > test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.52s
   >
   > error: test failed, to rerun pass `-p topiary-cli --test sample-tester`

Builds but fails to test, but also these deps still need to be updated

@toastal
Copy link
Author

toastal commented Sep 10, 2024

--- a/Cargo.lock	2024-09-10 11:33:28.428829334 +0700
+++ b/Cargo.lock	2024-09-10 11:17:45.106205987 +0700
@@ -1382,8 +1382,8 @@
 
 [[package]]
 name = "tree-sitter-bash"
-version = "0.19.0"
-source = "git+https://github.com/tree-sitter/tree-sitter-bash?rev=1b0321ee85701d5036c334a6f04761cdc672e64c#1b0321ee85701d5036c334a6f04761cdc672e64c"
+version = "0.20.0"
+source = "git+https://github.com/tree-sitter/tree-sitter-bash?rev=c0f5797a728fc4ebd78a8b0e436b1494a8ab5f51#c0f5797a728fc4ebd78a8b0e436b1494a8ab5f51"
 dependencies = [
  "cc",
  "tree-sitter",

--- a/Cargo.toml	2024-09-10 11:42:13.451688919 +0700
+++ b/Cargo.toml	2024-09-10 11:42:37.337239550 +0700
@@ -71,7 +71,7 @@
 tokio-test = "0.4"
 toml = "0.8"
 tree-sitter = "=0.20.10"
-tree-sitter-bash = { git = "https://github.com/tree-sitter/tree-sitter-bash", rev = "1b0321ee85701d5036c334a6f04761cdc672e64c" }
+tree-sitter-bash = { git = "https://github.com/tree-sitter/tree-sitter-bash", rev = "c0f5797a728fc4ebd78a8b0e436b1494a8ab5f51" }
 tree-sitter-css = { git = "https://github.com/tree-sitter/tree-sitter-css.git", rev = "02b4ee757654b7d54fe35352fd8e53a8a4385d42" }
 tree-sitter-json = { git = "https://github.com/tree-sitter/tree-sitter-json.git", rev = "3b129203f4b72d532f58e72c5310c0a7db3b8e6d" }
 tree-sitter-nickel = { git = "https://github.com/nickel-lang/tree-sitter-nickel", rev = "43433d8477b24cd13acaac20a66deda49b7e2547" }

@toastal
Copy link
Author

toastal commented Sep 10, 2024

--- a/Cargo.toml	2024-09-10 11:42:13.451688919 +0700
+++ b/Cargo.toml	2024-09-10 11:42:37.337239550 +0700
@@ -71,7 +71,7 @@
 tokio-test = "0.4"
 toml = "0.8"
 tree-sitter = "=0.20.10"
-tree-sitter-bash = { git = "https://github.com/tree-sitter/tree-sitter-bash", rev = "1b0321ee85701d5036c334a6f04761cdc672e64c" }
+tree-sitter-bash = { git = "https://github.com/tree-sitter/tree-sitter-bash", rev = "b2959f8be16fa1e2ee88a560074527fdef05b328" }
 tree-sitter-css = { git = "https://github.com/tree-sitter/tree-sitter-css.git", rev = "02b4ee757654b7d54fe35352fd8e53a8a4385d42" }
 tree-sitter-json = { git = "https://github.com/tree-sitter/tree-sitter-json.git", rev = "3b129203f4b72d532f58e72c5310c0a7db3b8e6d" }
 tree-sitter-nickel = { git = "https://github.com/nickel-lang/tree-sitter-nickel", rev = "43433d8477b24cd13acaac20a66deda49b7e2547" }

--- a/Cargo.lock	2024-09-10 11:56:50.347199224 +0700
+++ b/Cargo.lock	2024-09-10 11:54:37.112528333 +0700
@@ -1383,7 +1383,7 @@
 [[package]]
 name = "tree-sitter-bash"
 version = "0.19.0"
-source = "git+https://github.com/tree-sitter/tree-sitter-bash?rev=1b0321ee85701d5036c334a6f04761cdc672e64c#1b0321ee85701d5036c334a6f04761cdc672e64c"
+source = "git+https://github.com/tree-sitter/tree-sitter-bash?rev=b2959f8be16fa1e2ee88a560074527fdef05b328#b2959f8be16fa1e2ee88a560074527fdef05b328"
 dependencies = [
  "cc",
  "tree-sitter",

@toastal
Copy link
Author

toastal commented Sep 10, 2024

   last 25 log lines:
   > test formatted_query_tester ... ok
   > test input_output_tester ... FAILED
   >
   > failures:
   >
   > ---- input_output_tester stdout ----
   > thread 'input_output_tester' panicked at topiary-core/src/test_utils.rs:6:9:
   >
   > ...
   > 130 ENV_VAR=123 ANOTHER=456 some_command
   > 131 cat <<-HEREDOC
   > 132     Here is
   >     Here is
   > 133     a
   > 134       heredoc
   > ...
   > note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
   >
   >
   > failures:
   >     input_output_tester
   >
   > test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.25s
   >
   > error: test failed, to rerun pass `-p topiary-cli --test sample-tester`

Parses but fails test

@ErinvanderVeen
Copy link
Collaborator

This has been a recurring theme with the bash tree-sitter grammar depending on bash itself. The lockfile hasn't been updated in ages because the bash grammar changed significantly, and we haven't had the time to update our queries.

The main issue regarding the current build failure is here: #702, I'll leave this issue open for the valuable regarding the bash update, but I think all further discussion about the savannah issue should go there.

Thank you!

toastal added a commit to toastal/nixpkgs that referenced this issue Sep 10, 2024
A messy update to say the least. Topiary 0.4.0 was released a while
back, but the update was non-trivial. Not only did a bunch of Git
outputHashes SHAs need to get updated & a new, revendored Cargo.lock,
but a Git submodule in tree-sitter-bash, which is by Topiary referenced
at an arbitrary point in 0.19.0’s history, is now no longer up & needed
patching to remove it. To circumvent needing to learn how patch to a
Cargo dependency for this, I have forked the tree-sitter-bash project @
Topiary 0.4.0’s commit + cherry-picked the
b2959f8be16fa1e2ee88a560074527fdef05b328 patch which removes the
submodule dependencies (to check that this is indeed the case, you may
check the commit graph
<https://codeberg.org/toastal/tree-sitter-bash/graph>). I also opened an
upstream issue with Topiary
<tweag/topiary#734> to hopefully in the future
start keeping these grammar queries updated more often than once every 3
years.

Co-Authored-By: Avi Dessauer <[email protected]>
@toastal toastal closed this as completed Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants