Tools: Add RealFlight Autotest #28293
Conversation
This allows for autotests to run using the same UDP interface as the other vehicle frames.
There was a problem hiding this comment.
Not a great fan of needing to pass in the realflight address.
Ideally we:
- integrate this correctly with
vehicleinfo.py - get rid of "external: True" from vehicleinfo.py, instead specify which external simulator is involved
- factor out the code from FlyEachFrame which sets things up from the vehicleinfo definition
- read configuration for external simulator from a .json file - same code as we use for callisto.json but pointing out how to use the external simulator
- extreme stretch goal allow use of external simulator from sim-on-hardware
- use the same information from vehicleinfo.py in
sim_vehicle.pyto make it easy for it to use the same definitions.
| name = test.name | ||
| else: | ||
| name = test.__name__ | ||
| if name.lower().startswith("realflight"): |
There was a problem hiding this comment.
Yeah, very much not the right way to do things :-)
Unfortunately I see no good way around this. The address will be different for every person, unless they are crazy and running autotests with Cygwin. If they are using WSL (like me), I can have a shell command I get it automatically, but if someone uses separate machines (which I believe is more common in the core dev group), or uses a VM, we need to give them the option to override. If we want to force this to only work in WSL and Cygwin, I can remove that. |
robertlong13
force-pushed
the
pr/realflight-autotest
branch
from
30f7c30
to
873e1e6
Compare
|
Yes, self-testing could also be interesting. |
On the dev call, we were discussing testing #27839 in RealFlight. I thought it would be nice to have a framework for running optional automated tests in RealFlight. I ran the test before and after the PR and loaded both logs into the PID Review Tool and compared. They look the same (though @IamPete1, it would be really nice to be able to load two logs in here for this).
Draft PR because I'd like to get some advice on the right way to do add this. Do we do this by adding disabled tests to things like quadplane.py? Do we add separate files like quadplane_realflight.py? Not sure. This is my a-bit-hacky placeholder that I used to test the aformentioned PR (though I'm very late as it's been merged for a week).
(This PR is currently based on top of #28282, so only the last two commits in this will be here after the other is merged)