Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests fail if run in a terminal with more or less than 80-82 columns #237

Open
Michionlion opened this issue Sep 13, 2021 · 0 comments
Open
Assignees
Labels

Comments

@Michionlion
Copy link
Member

Michionlion commented Sep 13, 2021

Describe the bug

Many test cases in GatorGrader check the newline count of generated output. It turns out that this count can change depending on an external variable: the number of columns in the terminal used to run pytest. With a $COLUMNS value fewer than 80, certain tests will fail, with more failures the smaller the terminal becomes. The same is true in reverse, with a $COLUMNS value of greater than 82.

To Reproduce

Steps to reproduce the behavior:

  1. Resize a terminal such that echo $COLUMNS prints a number less than 80 or greater than 82
  2. Execute pipenv run cover to see how many tests fail at that given size.

Expected behavior

Tests should not depend on terminal size for correctness.

Environment

GitHub Actions terminal, Elementary OS 5 (based on Ubuntu 18.04) gnome-terminal.
Python 3.9.7 (also reproducible in 3.8.12).

I can confirm that this bug does not appear in Python 3.7.12.

Proposed Solution

This bug is caused by asserting an exact amount of new lines in expected produced output; instead of asserting an exact amount, the minimum amount possible should be "at least" (with >=) asserted. Alternatively, new line count should be dropped as a measure of correctness for produced output -- this may be the desired solution, since new line count can be influenced by many factors, and often does not indicate regressions or errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant