Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wavebank concurrent reads on CentOS #248

Open
shawnboltz opened this issue Dec 10, 2021 · 2 comments
Open

Wavebank concurrent reads on CentOS #248

shawnboltz opened this issue Dec 10, 2021 · 2 comments
Labels
bug Something isn't working

Comments

@shawnboltz
Copy link
Collaborator

Description
Just installed a fresh copy of obsplus on a CentOS machine with python 3.9 and ran the tests. All of the tests passed except for two in the WaveBank tests relating to performing concurrent reads on the WaveBank:
tests/test_bank/test_wavebank.py::TestConcurrentReads::test_concurrent_get_waveforms and ...::test_concurrent_update_index , which give the following error:

concurrent.futures.process.BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore

To Reproduce
Install obsplus (from master) in a new install of conda (Anaconda3-2021.11-Linux-x86_64.sh to be exact)
Execute the following:

pytest tests/test_bank/test_wavebank.py::TestConcurrentReads

Note that these are parametrized tests and that if you only run one of the two variations of either test it passes, so it is very much something related to the concurrency aspect itself.

I also created a new environment using python 3.8 to try and isolate whether it was related to the python version or to the OS, and the tests failed there as well.

Expected behaviour
The tests should pass.

Versions (please complete the following information):

  • OS: CentOS 7
  • ObsPlus Version: 0.2.2 (Note: this is directly from master)
  • Python Version: 3.9/3.8 (conda 4.11.0; Anaconda3-2021.11-Linux-x86_64.sh)
@shawnboltz shawnboltz added the bug Something isn't working label Dec 10, 2021
@d-chambers
Copy link
Member

This looks like a difference in python multiprocessing in the CentOS operating system. Moreover, it looks like it is a failure of the test setup and not so much the functionality of ObsPlus. Any idea how to fix it?

@shawnboltz
Copy link
Collaborator Author

I'm honestly not sure what the fix would look like. It was one of those things that happened at the end of the day and I wanted to make a note of it, but haven't had an opportunity to really delve into it. There was a bunch of other pytest stuff that got spewed into the terminal that wasn't part of the actual error message that I was going to see if I could capture and get a better understanding of what went wrong.

While it does indeed seem to be a problem with the test setup, I also want to make sure there are no ramifications to how the WaveBank itself behaves.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants