Initial version of UI tests. #1115

Alexhuszagh · 2022-11-05T23:05:24Z

This is a very initial draft, but the idea is to add UI tests (which we'll add more as this gets further along).

UI tests still to add:

Any suggestions for additional UI tests to add, please suggest them below. This also needs to add ci/test-ui.sh into the CI. This will close #1037. I'm also waiting for #1103 to be closed so I can add it to the UI tests.

src/cli.rs

xtask/src/temp.rs

xtask/src/ui.rs

Emilgardis · 2022-11-05T23:18:31Z

I think instead of having methods to check the output we can have .stderr and .stdout files, i.e what trycmd does.

this way we can also present a diff and ensure every aspect is captured.

But this is definitely an interesting approach and may work very well 👍🏼

Alexhuszagh · 2022-11-20T00:34:19Z

I've added the initial versions of trycmd (and discovered some very minor bugs in the process). There's a few issues, but it's mostly working well. Due to the limitations of the markdown format, we're using the toml/stderr/stdout for everything, so to avoid duplicating everything numerous times, there's a lot of symlinks.

Alexhuszagh · 2022-11-20T21:01:23Z

I've completed reworked how CLI parsing works, and added an extensive set of integration tests. By default, these are ignored because they're very slow to run. I've also added in a regression test for #1141. I've ensured this only runs once per target.

The ignored unittests are also run in the main test section, so we don't run them except when needed (since they are very slow).

xtask/src/main.rs

xtask/src/temp.rs

tests/cli.rs

Emilgardis · 2022-11-21T00:39:17Z

Cargo.toml

+trycmd = "0.14.4"
+snapbox = "0.4"


We could have this as its own workspace member alongside cross and xtask, not needed but just a thought

Is there any major advantage to that over faster compile times for the test suite? Not saying that isn't a good goal, just wondering if there's something additional as well.

src/cli.rs

src/tests.rs

Alexhuszagh · 2022-11-21T01:37:02Z

This needs to handle escape sequences as well:
https://github.com/cross-rs/cross/actions/runs/3509739225/jobs/5879045993#step:6:238

I've used --color never for some of them, but that's not enough.

Alexhuszagh · 2022-11-21T01:42:16Z

ci/test-rest-args.sh

+ci_dir=$(realpath "${ci_dir}")
+. "${ci_dir}"/shared.sh
+
+main() {


This needs a clippy -- --deny warnings just to ensure that -Zbuild-std is getting correctly added as well.

Alexhuszagh added enhancement meta issues/PRs related to the maintenance of the crate. no changelog A valid PR without changelog (no-changelog) labels Nov 5, 2022