Bug: Output received only with specific commands

[Chezka109]

Description:
When running poetry run execexam . tests/, no output is displayed, even though the tests pass successfully. Unlike pytest, which provides verbose output when using the -v flag, execexam does not provide any feedback, making it difficult to confirm whether tests were executed or their results.

Steps to Reproduce:

1. Install the project dependencies with poetry install.
2. Run the tests using poetry run execexam . tests/.
3. Notice that no output is generated, even if the tests pass.
4. Run the tests using poetry run pytest -v for comparison, and observe that the test results are shown.

Expected Behavior:
Some form of output should be displayed when running tests with execexam, at least showing whether the tests passed or failed. Ideally, there should be a verbosity option or default feedback that informs the user about the test results.

Actual Behavior:
No output is provided by execexam, even though the tests execute and pass in the background. There is no indication of the test status unless pytest is used directly.

Additional Information:

• pytest provides verbose output using the -v flag, but execexam doesn't seem to have any verbosity option or built-in output.
• The issue might stem from how execexam wraps or integrates with pytest and handles test execution.

[gkapfham]

Hello @Chezka109 is this still an issue or has it been resolved with recent releases?

[Chezka109]

@gkapfham, sorry for the late reply. it is still an issue but i believe @hemanialaparthi is working on it.

[gkapfham]

Hi @hemanialaparthi thanks for raising this issue, I appreciate it! Yes, I agree that this is a problem that we should resolve. Ideally, the tool should at least provide the same output as --report status every time the tool is run. When we refactored and enhanced the tool to use the --report command-line argument we neglected to make sure that the tool always produces at least some minimal output by default. Who plans to work on this? What is the timeline?

[hemanialaparthi]

Hi @gkapfham! I'd be happy to take on this issue, but I have a few other PRs to complete first. I'll make sure to get to it soon, though!

[gkapfham]

Hi @hemanialaparthi, sure that sounds fine. I have marked this as future for now. I think that we could resolve this issue by deciding what is the default --report that execexam will always run. Perhaps this would be an easy issue to resolve if we simply configure the tool such that --report status is the default argument for this command-line argument? Please let me know what you think, okay?

[rebekahrudd]

I think that this issue is also impacting the output of gatorgrade, because sometimes when execexam gives no output gatorgrade will show that execexam failed but then not give any feedback to the user about what the failure was. Is this the same issue you are talking about @Chezka109 and @hemanialaparthi , or is this issue only saying that execexam isn't showing when tests pass?

[hemanialaparthi]

Hi! This isn’t quite what @Chezka109 and I were discussing. When you run execexam, it doesn’t generate an output; instead, it shows red when it fails and blue when it passes, but no output is displayed. We have to put in echo $? to see what the return code is. The issue you're mentioning is another one that also needs to be addressed.