Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove cache folder #2927

Merged
merged 83 commits into from
Jun 6, 2024
Merged

Conversation

paulrignanese
Copy link
Contributor

removed the setting of the cache folder as global variable in test_phase_shift in /preprocessing/tests, to see if it survives the CI tests

… in sortingcomponents/benchmark/tests folder
… tmp_path for test_filter_gaussian in preprocessing/tests folder
…one, replace cache_folder with pytest fixture for test_numpy_extractors in core/tests
… function for test_node_pipeline.py in core/tests/ folder
…ation function for test_npyfoldersnippets.py in core/tests/ folder
…ation function for test_motion.py in preprocessing/tests/ folder
…ate_cache_folder function for test_silence.py in preprocessing/tests/
…he_folder function for test_binaryrecordingextractor.py in core/tests/
…ate_cache_folder function for test_whiten.py in preprocessing/tests/
…he_folder function for test_unitsaggregationsorting.py in core/tests/
# Conflicts:
#	src/spikeinterface/core/tests/test_core_tools.py
#	src/spikeinterface/preprocessing/tests/test_motion.py
@paulrignanese paulrignanese force-pushed the remove_cache_folder branch from 91c15b3 to e7d55e1 Compare May 30, 2024 10:54
pre-commit-ci bot and others added 8 commits May 30, 2024 10:56
…he_folder function for test_time_handling.py in core/tests/
…he_folder function for test_time_handling.py in core/tests/
…he_folder function for test_basesorting.py in core/tests/
…reation for test_benchmark_peak_localization.py in sortingcomponents/benchmark/tests/ folder, even if here the tests are skipped
…_creation for test_sorting_folder.py in core/tests/ folder
… cache_folder by pytest fixture for test_docker_containers.py in sorters/external/tests (fails possibly because of docker not working on my machine)
…w inherited as self.cache_folder in Kilosort4SorterCommonTestSuite for test_kilosort4.py in sorters/external/tests folder
…eplace cache_folder by pytest fixture for test_singularity_containers.py in sorters/external/tests
…st_singularity_containers_gpu.py in sorters/external/tests
… need to be replaced by pytest fixtures however there are not trivial
@alejoe91 alejoe91 added the hackathon-24 Contributions during the SpikeInterface Hackathon May 24 label Jun 1, 2024
…for test_deepinterpolation.py in deepinterpolation/tests folder (I reverted those changes because we thought that it was producing a bug but it was unrelated, it is back now - Still 2 pytest fixtures TODO : test_launcher.py and test_runsorter.py) Last commit for the Hackathon 2024
@JoeZiminski
Copy link
Collaborator

Hey @paulrignanese thanks a lot for this, that's a lot of work! I agree that breaking this up into smaller PRs is a good idea, and will help with fixing some of the conflicts. I have rights on your fork so if it is convenient for you I'm happy to cherry-pick your commits onto new branches on your fork and fix any merge conflicts, then you can open new PRs from these branches. I can let you know the details of this process. However if this will mess with your workflow or you'd like to go ahead with this yourself of course that's no problem!

@alejoe91
Copy link
Member

alejoe91 commented Jun 4, 2024

I agree that this is huge, but it might make sense to fix this since it only involves tests. If @JoeZiminski and @h-mayorquin think that breaking up into smaller pieces, then let's go for it. In that case, we might need some of @JoeZiminski 's git magic! @paulrignanese what do you think?

@paulrignanese
Copy link
Contributor Author

paulrignanese commented Jun 4, 2024 via email

@JoeZiminski
Copy link
Collaborator

JoeZiminski commented Jun 4, 2024

Great, thanks Paul! @alejoe91 yes I'm also a little conflicted as if the tests are passing then it should be okay. On balance although it will be more work to break up I think because of the number of conflicts it will be safer to break it up to avoid accidentally reverting any recent merges. I think updating the fork's main main branch to the most recent version, branching off that and then cherry-picking commits will avoid these merge conflicts or make them easier to resolve. I also wanted to go through the test suite anyways to improve my understanding so this seems like a good chance to do it!

If everyone is happy I will:

  1. sync @paulrignanese forks main branch with SpikeInterface main
  2. split branches off one per-module and cherry-pick relevant commits over while checking for conflicts. @paulrignanese at any time in this process feel free to message me, jump in, anything like that! as it might be disruptive for you if I'm working on your fork.
  3. @paulrignanese I'll let you know when each branch is ready for a PR to be created.

@samuelgarcia
Copy link
Member

Hi Paul.
Thanks a lot for this tedious refactoring.
We are OK to merge this in one unique PR.
We will fix conflicts ourself.

@paulrignanese
Copy link
Contributor Author

Ok!

@JoeZiminski
Copy link
Collaborator

Awesome, cheers @alejoe91 @samuelgarcia!

@JoeZiminski
Copy link
Collaborator

Hey @alejoe91 just to check I don't need to do anything on this PR anymore right?

@alejoe91
Copy link
Member

alejoe91 commented Jun 6, 2024

Thanks @JoeZiminski

No I think this is ok, will give it a final read today ;)

conftest.py Outdated

for mark_name in mark_names:
(pytest.global_test_folder / mark_name).mkdir()
@pytest.fixture
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paulrignanese @JoeZiminski moving the fixture here allows us to remove it from everywhere else, since it's available to the entire pytest session ;)

@alejoe91
Copy link
Member

alejoe91 commented Jun 6, 2024

Ok, should be done now :)

@alejoe91 alejoe91 merged commit a9fe858 into SpikeInterface:main Jun 6, 2024
10 of 11 checks passed
@JoeZiminski
Copy link
Collaborator

Awesome cheers, and thanks a lot @paulrignanese !!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hackathon-24 Contributions during the SpikeInterface Hackathon May 24 testing Related to test routines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants