Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Testing and CI #106

Open
Meziu opened this issue Mar 27, 2023 · 5 comments
Open

Improve Testing and CI #106

Meziu opened this issue Mar 27, 2023 · 5 comments

Comments

@Meziu
Copy link
Member

Meziu commented Mar 27, 2023

We should mass produce tests now that the API is getting stabilized, and integrate with the CI to make sure the examples build after changes.

@ian-h-chamberlain
Copy link
Member

Definitely agree, unfortunately after doing a bit of research, I found out some things:

  • Our current CI does not appear to actually check any doctests at all, so we will want to fix that. related cargo test --all-targets does not run doc tests rust-lang/cargo#6669
  • Doc tests are... well, kind of an unfinished feature it seems like. A few gaps we will most likely run into:
  • For CI testing purposes, we can try using target.armv6k-nintendo-3ds.runner Cargo config but we don't really have a good way of verifying the tests actually pass. I have some ideas involving citra but it might take a bit of work

For now, we should probably keep all the examples as ```no_run, and try adding something to CI to make sure they at least compile. I might open a separate issue for the runner itself, since I think that will be a bit more involved and might even require changes to upstream citra to be effective.

@ian-h-chamberlain
Copy link
Member

With lots of doctests in #134 I've marked this "In Progress", at least. I haven't had a chance to revisit the citra runner to really test everything though, so I'd say let's leave it open for now.

@Meziu
Copy link
Member Author

Meziu commented Aug 4, 2023

I've noticed the CI still needs a little boost, and unit tests are very much lacking, so this issue needs to stay open for a while longer.

@ian-h-chamberlain
Copy link
Member

I've made some progress on a proof of concept for running tests using Citra:
https://github.com/ian-h-chamberlain/test-runner-3ds

So far I can get it to work with integration + unit tests, but it's not running doctests yet.

Example of passing: https://github.com/ian-h-chamberlain/test-runner-3ds/actions/runs/5781441190/job/15666535932
and failing: https://github.com/ian-h-chamberlain/test-runner-3ds/actions/runs/5781347357/job/15666313158

For now, I just copied all the test runner code out of ctru-rs — if this seems like a good direction to continue in, I can migrate the test-runner-3ds repo to rust3ds/test-runner (open to better names too) and start the process of using it in ctru-rs.

I wanted to have this as a separate repo so we can reuse it for e.g. citro3d-rs and make it usable for others if they want. I don't think we strictly need to publish the crate either, since cargo allows for git dev-dependencies when publishing.

@adryzz
Copy link
Contributor

adryzz commented Feb 19, 2024

I have recently made an app (in rust of course) to run unit tests back-to-back on physical consoles. it uses a reimplementation of 3dslink and the rosalina 3dsx loader, and almost works perfectly, just needs a few fixes for launch parameters.

It also supports toggling Luma3DS's GDB stub on a very hacked together branch.

This works when compiling the app as a .cia file to install, and a few additions to pass back the result to the app, to then log (both to a file and the bottom screen).

This is the end result, although this branch is very cursed and i'm adding things bit by bit as i clean them up

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants