Guidance on debugging integration tests

[Jorge-Rodriguez]

We don't have any section in our guides on how to debug failing tests. Here's what I do:

• I have modified my ansible_test module to enable ANSIBLE_KEEP_REMOTE_FILES.
• I add a pause task right after the test case to be debugged, with a timeout long enough to allow me to run a debugger.
• I open a shell session on the running container and install Pudb.
• I navigate to the folder with the failing test and run Ansiballz explode.
• Debug the test with pudb Ansiballz execute
• ...
• profit?

@Andersson007 @bmalynovytch what's your approach in this case? How should we document a procedure to handle this workflow?

[Andersson007]

Adding this section sounds like a great idea @Jorge-Rodriguez ! Would be really helpful for us and new folks.
Thanks for sharing this!

I'll think about the structure of the doc later, added to my TODO.

I usually don't debug much (your tasks are usually much more technically complex then mine).

If you have time you could explain most possible scenarios and the methods you mentioned when they are applicable for the scenarios and explain how they can help (with short examples, maybe). My vision is several possible scenarios with the tools and short debugging algorithms. From simple things to more complex. Say

1. run your tests with -vvv > debug.log
2. find the place in debug.log where the module failed
3. first ramification: if the module failed itself because the wrong code, scrutinize the returned traceback. ...
4. second ramification: if the assertion failed, scrutinize the returned JSON and compare it with what you expected to see. ...

I think it makes sense to create a separate md called, say, TEST_DEBUGGING.md or something else. Then we could put the reference in the other guides.

I think we should keep it as laconic as possible and put references to the detailed guidelines of the mentioned tools, so if a reader wishes, they can dig into it.

I do like the idea:)