refactor: Improve script reuse

[febo]

Problem

There is too much duplication on the use of scripts. For example, there are 2 "versions" of each Rust specific script: one for the client and one for the interface.

Solution

** UPDATED **

The organization of the scripts is simplified by having multiple copies of the same script. Rust scripts expect a relative crate path, so the same script can be used to perform actions on the different crates. Further customization is possible by passing specific command line arguments.

The refactored scripts structure is as follows:

scripts
  |
  +- ci
  |
  +- helpers
  |
  +- js
  |
  +- rust

Since a program usually have multiple Rust crates, scripts for Rust actions expect a relative crate path – e.g., clients/rust (for the Rust client crate), interface (for the interface crate) and program (for the program crate):

"interface:format": "tsx ./scripts/rust/format.mts interface"

[buffalojoec]

I think this is looking nice and definitely heading in the right direction, I'm just a little worried about the scripts code continuing to swell. We're almost at the point where we need usage details (ie. --help).

I noticed the default args, like for example with clippy, are becoming increasingly obscured with this layout.

I wonder if there's an approach we can take here where we can condense to just a few scripts, maybe some to do test validator, client generation, and publishing, and then a single js.mjs and rust.mjs for any library operations. For example, these two lines are used across almost every Rust script:

const [env, cliArguments] = getCrateEnvironmentFromArgs();
const args = env.getTestArguments(cliArguments);

Maybe we could have a single JSON or YAML file where you can define default arguments, which can maybe be overridden on the command line. This could be a single file that captures all of the default args defined in the crate folder, and each script could check for an entry in this declaration file during execution.

Then you can just invoke the singular script (rust.mjs or js.mjs) with:

1. The command
2. The library path
3. Flags/arguments

I guess something like:

rust:
  lint:
    interface: [
          "--deny=warnings",
          "--deny=clippy::default_trait_access",
          "--deny=clippy::arithmetic_side_effects",
          "--deny=clippy::manual_let_else",
          "--deny=clippy::used_underscore_binding",
        ]
    program: [
          # ...
        ]

This way, the default args are crystal-clear and easy to configure, and then everything else in the scripts starts to look kind of similar except for the actual commands that are invoked. I'd bet you could wrap much of the setup steps from each script into a single setup, and then key on the provided command argument to handle the rest, within the same script.

What do you think?

[lorisleiva]

I think we still have ways to go to find the right balance between offering helpful versus overwhelming scripts but I do believe this is a step in the right direction.

[joncinque]

I'm a big fan of all the helpers/ scripts, since that's exactly what we want for language-specific tasks.

For the configuration, I'm leaning toward @buffalojoec's thinking. Are there any situations in which we have different args for the same job on different packages? For example, if I'm linting a rust package, when will I need different args than the defaults? If I truly need to do something different, then my top-level package.json can specify the different command to run.

That package.json file is essentially the CLI / --help output, since it can list all of actions that you might want to run.

If all the helpers just take a manifest path, or even more general, a directory, then I think we can remove all of the scripts/crate/*. But maybe I'm missing a use-case. I'm also happy to hear any other opinions!

[lorisleiva]

Love it! Jon was right, there's no need for crate variables at all. 🙌

[buffalojoec]

Nice, looking much smoother with the --manifest-path option.

Let me know what you think of this. I don't think a single script looks all that bad IMO.
#32

[febo]

Nice, looking much smoother with the --manifest-path option.

Let me know what you think of this. I don't think a single script looks all that bad IMO. #32

I like #32 a lot. We should do the same with the js one, so we don't need the js directory either.