Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fail to run tests with --coverage flag #335

Open
palage4a opened this issue Nov 14, 2023 · 2 comments
Open

Fail to run tests with --coverage flag #335

palage4a opened this issue Nov 14, 2023 · 2 comments

Comments

@palage4a
Copy link

palage4a commented Nov 14, 2023 •
edited
Loading

There is a problem with test suites with helper instances when we are using --coverage flag to run luatest on MacOS.

Tarantool version: Tarantool Enterprise 2.11.0-0-gc9673ebb7-r575-gc64
OS: MacOS Sonoma 14.1.1
Arch: x86_64

Steps to reproduce:

$ cartridge create --name coverage-fail
$ cd coverage-fail
$ tarantoolctl rocks install luacov
$ # git clone https://github.com/tarantool/luatest <path-to-luatest-workdir>
$ tarantoolctl rocks make --chdir <path-to-luatest-workdir>
$ cartridge build
$ .rocks/bin/luatest --coverage

And we get output like this:

Tests with errors:
------------------

1) integration_api.test_sample
.../coverage-fail/.rocks/share/tarantool/luatest/server.lua:513: connect, called on fd 32, aka 127.0.0.1:59706: Connection refused
stack traceback:
	....rocks/share/tarantool/cartridge/test-helpers/server.lua:165: in function 'connect_net_box'
	....rocks/share/tarantool/cartridge/test-helpers/server.lua:175: in function 'retrying'
	....rocks/share/tarantool/cartridge/test-helpers/server.lua:174: in function 'start'
	...rocks/share/tarantool/cartridge/test-helpers/cluster.lua:265: in function 'start'
	...ode/palage4a/coverage-fail/test/integration/api_test.lua:8: in function <...ode/palage4a/coverage-fail/test/integration/api_test.lua:6>
	...
	[C]: in function 'xpcall'
Captured stdout:
api | 2023-11-14 15:54:36.838 [71043] main/103/init.lua/cartridge I> Using advertise_uri "localhost:13301"
api | 2023-11-14 15:54:36.839 [71043] main/103/init.lua/membership.options I> Membership encryption enabled
api | 2023-11-14 15:54:36.843 [71043] main/103/init.lua/cartridge I> Probe uri was successful
api | 2023-11-14 15:54:36.844 [71043] main/103/init.lua/membership I> Membership BROADCAST sent to 127.0.0.1:13302
...
@ylobankov
Copy link
Contributor

ylobankov commented Nov 17, 2023 •
edited
Loading

I don't have an x86 mac, but I cannot reproduce this issue on m1.

ylobankov@ylobankov:~/Workspace$ sw_vers
ProductName:		macOS
ProductVersion:		14.1.1
BuildVersion:		23B81
ylobankov@ylobankov:~/Workspace$ uname -m
arm64
ylobankov@ylobankov:~/Workspace$ cartridge create --name coverage-fail
   • Create application coverage-fail
   • Generate application files
   • Initialize application git repository
   • Application "coverage-fail" created successfully
ylobankov@ylobankov:~/Workspace$ cd coverage-fail
ylobankov@ylobankov:~/Workspace/coverage-fail (master)$ tarantoolctl rocks install luacov
Installing http://rocks.tarantool.org/luacov-0.13.0-1.rockspec

Cloning into 'luacov'...
remote: Enumerating objects: 1800, done.
remote: Counting objects: 100% (258/258), done.
remote: Compressing objects: 100% (128/128), done.
remote: Total 1800 (delta 130), reused 222 (delta 114), pack-reused 1542
Receiving objects: 100% (1800/1800), 824.26 KiB | 3.31 MiB/s, done.
Resolving deltas: 100% (1077/1077), done.
Note: switching to '637c48d59f722050d718d0c398f655bc7fe1707a'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

No existing manifest. Attempting to rebuild...
luacov 0.13.0-1 is now installed in /Users/ylobankov/Workspace/coverage-fail/.rocks (license: MIT)

ylobankov@ylobankov:~/Workspace/coverage-fail (master)$ git clone https://github.com/tarantool/luatest
Cloning into 'luatest'...
remote: Enumerating objects: 1963, done.
remote: Counting objects: 100% (652/652), done.
remote: Compressing objects: 100% (230/230), done.
remote: Total 1963 (delta 508), reused 451 (delta 420), pack-reused 1311
Receiving objects: 100% (1963/1963), 489.83 KiB | 2.00 MiB/s, done.
Resolving deltas: 100% (1408/1408), done.
ylobankov@ylobankov:~/Workspace/coverage-fail (master)$ tarantoolctl rocks make --chdir luatest/
Missing dependencies for luatest scm-1:
   checks >= 3.0.0 (not installed)

luatest scm-1 depends on checks >= 3.0.0 (not installed)
Installing http://rocks.tarantool.org/checks-scm-1.rockspec

Cloning into 'checks'...
remote: Enumerating objects: 523, done.
remote: Counting objects: 100% (102/102), done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 523 (delta 52), reused 47 (delta 40), pack-reused 421
Receiving objects: 100% (523/523), 122.54 KiB | 988.00 KiB/s, done.
Resolving deltas: 100% (266/266), done.
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /var/folders/7k/zkm6llyj7n51mqz_lgb2ct4w0000gn/T/luarocks_checks-scm-1-cm56oo/checks/build.luarocks
Install the project...
-- Install configuration: ""
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/checks/scm-1/lua/checks.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/checks/scm-1/lua/checks
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/checks/scm-1/lua/checks/version.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/checks/scm-1/lua/override/checks.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/checks/scm-1/lua/override/checks
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/checks/scm-1/lua/override/checks/version.lua
checks scm-1 is now installed in /Users/ylobankov/Workspace/coverage-fail/.rocks (license: MIT)

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Warning (dev) at /opt/homebrew/Cellar/cmake/3.27.8/share/cmake/Modules/GNUInstallDirs.cmake:243 (message):
  Unable to determine default CMAKE_INSTALL_LIBDIR directory because no
  target architecture is known.  Please enable at least one language before
  including GNUInstallDirs.
Call Stack (most recent call first):
  cmake/FindTarantool.cmake:2 (include)
  CMakeLists.txt:9 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found Tarantool: /opt/homebrew/Cellar/tarantool/2.11.1/include (found version "2.11.1-0-g96877bd")
-- Tarantool LUADIR is /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua
-- Tarantool LIBDIR is lib/tarantool
-- Tarantool BINDIR is /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/bin
-- Configuring done (0.1s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    LUAROCKS


-- Build files have been written to: /Users/ylobankov/Workspace/coverage-fail/luatest/build.luarocks
Install the project...
-- Install configuration: ""
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/loader.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/hooks.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/class.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/init.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/server.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/group.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output_beautifier.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/replica_conn.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/sandboxed_runner.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/tarantool.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/capturing.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output/tap.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output/nil.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output/generic.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output/junit.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/output/text.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/cli_entrypoint.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/assertions.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/server_instance.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/monitor.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/parametrizer.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/VERSION.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/pp.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/test_instance.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/comparator.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/ffi_io.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/coverage.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/utils.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/mismatch_formatter.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/capture.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/http_response.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/helpers.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/coverage_utils.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/replica_proxy.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/runner.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/sorted_pairs.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/process.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/replica_set.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luarocks
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luarocks/test
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luarocks/test/luatest.lua
-- Up-to-date: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/lua//luatest/VERSION.lua
-- Installing: /Users/ylobankov/Workspace/coverage-fail/.rocks/share/tarantool/rocks/luatest/scm-1/bin/luatest
luatest scm-1 is now installed in /Users/ylobankov/Workspace/coverage-fail/.rocks (license: MIT)

ylobankov@ylobankov:~/Workspace/coverage-fail (master)$ cartridge build
   • Build application in /Users/ylobankov/Workspace/coverage-fail
   • Running `cartridge.pre-build`
   • Running `tarantoolctl rocks make`
   • Application was successfully built
ylobankov@ylobankov:~/Workspace/coverage-fail (master)$ .rocks/bin/luatest --coverage
Tarantool version is 2.11.1-0-g96877bd
...
Ran 3 tests in 1.085 seconds, 3 succeeded, 0 failed

@palage4a
Copy link
Author

I've tried to use tarantool CE and there was no probs with it. It seems to me that issues might be related to tarantool enterprise.

$ .rocks/bin/luatest --coverage
Tarantool version is 2.11.2-0-g1bac2d2
...
Ran 3 tests in 1.289 seconds, 3 succeeded, 0 failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ylobankov @palage4a