Test coverage fails when running on a network filesystem in Azure Machine Learning

[FaridDelgadoTR]

Summary

pytest-cov fails when running from the context of a mounted filesystem in Azure Machine Learning. This may relate to the lack of support to the CIFS filesystem reported by @adamjstewart here.

Expected vs actual result

Command:

$ pytest --cov=src --cov-fail-under=80 --cov-report=term

Expected:

Actual result:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/_pytest/main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pytest_cov/plugin.py", line 271, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pytest_cov/engine.py", line 44, in ensure_topdir_wrapper
INTERNALERROR>     return meth(self, *args, **kwargs)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/pytest_cov/engine.py", line 230, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/control.py", line 680, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/control.py", line 749, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/collector.py", line 473, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(self.data))
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/sqldata.py", line 236, in _wrapped
INTERNALERROR>     return method(self, *args, **kwargs)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/sqldata.py", line 463, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/sqldata.py", line 522, in _choose_lines_or_arcs
INTERNALERROR>     with self._connect() as con:
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/sqldata.py", line 318, in _connect
INTERNALERROR>     self._create_db()
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/sqldata.py", line 268, in _create_db
INTERNALERROR>     db.executescript(SCHEMA)
INTERNALERROR>   File "/mnt/batch/tasks/shared/LS_root/mounts/clusters/some_compute_name/code/Users/some_user/labs-thoughttrace_sample-prioritization/.venv/lib/python3.6/site-packages/coverage/sqldata.py", line 1160, in executescript
INTERNALERROR>     self.con.executescript(script)
INTERNALERROR> sqlite3.OperationalError: database is locked

Reproducer

Run pytest --cov=. on a network-mounted filesystem in Azure ML.

Versions

$ python -V
Python 3.6.2 :: Anaconda, Inc.
$ pip list | grep pytest
pytest                         5.4.3
pytest-cov                     2.12.1
pytest-html                    3.2.0
pytest-metadata                1.11.0

Config

pytest.ini:

[pytest]
minversion = 5.0

testpaths = tests

junit_family = xunit2
junit_logging = all

log_auto_indent = True
log_format = %(asctime)s %(levelname)s %(message)s
log_date_format = %Y-%m-%d %H:%M:%S

addopts =
    --cov=src
    --cov-fail-under=80
    --cov-report=term

Code

Should fail with any test code, eg:

def test_foobar():
    assert True

[FaridDelgadoTR]

This is not the correct repo for this issue. The right place is coverage. Will create an issue when I have documented a more straightforward way to reproduce it.

See #476 for more info.