Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add plexon2 recording, sorting and event support #1736

Merged
merged 16 commits into from
Sep 13, 2023

Conversation

JuliaSprenger
Copy link
Contributor

This PR is building on top of NeuralEnsemble/python-neo#1214 and can only be merged once the pl2 format is supported by Neo.

@alejoe91 alejoe91 added NEO Problem related to NEO IO extractors Related to extractors module labels Jun 22, 2023
@JuliaSprenger JuliaSprenger marked this pull request as ready for review June 29, 2023 09:00
@JuliaSprenger
Copy link
Contributor Author

It looks like not all required packages are installed for the CI tests. As far as I can see you are installing the latest version of the neo package, but not all dependencies, as this would include the missing package zugbruecke here. Is this intended or would you prefer to install the missing package explicitly in the workflow? @CodyCBakerPhD

@samuelgarcia samuelgarcia added this to the 0.98.0 milestone Jul 6, 2023

mode = "file"
NeoRawIOClass = "Plexon2RawIO"
handle_spike_frame_directly = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you have a look to #1626 @h-mayorquin made some variable name changes.

@samuelgarcia
Copy link
Member

Thanks a lot @JuliaSprenger
Can you have a look when #1626 will be merge you will need a variable name change

@samuelgarcia
Copy link
Member

@alejoe91 do we include this also in the relase no ?

@alejoe91
Copy link
Member

alejoe91 commented Jul 6, 2023

Is it released on neo?

@alejoe91 alejoe91 removed this from the 0.98.0 milestone Jul 7, 2023
@alejoe91
Copy link
Member

@JuliaSprenger tests seem to fail because of zugbruecke (had to add it in the extractors dependencies). Would you have time to look into this?

@alejoe91
Copy link
Member

@samuelgarcia @h-mayorquin good to merge for me!

I included the missing dependencies in CI and updated after #1626

Comment on lines 43 to 49
- name: Install wine (needed for Plexon2)
run: |
sudo rm -f /etc/apt/sources.list.d/microsoft-prod.list
sudo dpkg --add-architecture i386
sudo apt-get update -qq
sudo apt-get install -yqq --allow-downgrades libc6:i386 libgcc-s1:i386 libstdc++6:i386 wine
shell: bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be quite heavy and not needed (unless plexon2 wrapper changes). We could make a manual workflow that installs wine and tests plexon2 only.

Otherwise we can make an install_wine.yml action and install + run plexon2 tests only if plexon2 has changed.

@alejoe91
Copy link
Member

@samuelgarcia done with the suggestions!

@@ -290,6 +290,32 @@ def test_pickling(self):
pass


# We mark plexon2 tests as they require additional dependencies (wine)
@pytest.mark.plexon2
Copy link
Collaborator

@h-mayorquin h-mayorquin Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In POSIX the dependencies are wine and https://pypi.org/project/zugbruecke/ right?

In windows what are the dependecies?

I don't like that if I don't have wine or this experimental packages my test will fail if run pytest -m "extractors" locally.

Can we add skipif tests to accomplish this? (this can be another PR if you think this is too much)

I am suggesting:

import subprocess
import platform

def has_dependencies():
    """
    Check if required dependencies are installed.
    """
    
    os_type = platform.system()

    if os_type == "Windows":
        # On Windows, check for dependencies if any (probably just return True)
        return True

    elif os_type == "Linux":
        # Check for 'wine' using dpkg
        try:
            result_wine = subprocess.run(['dpkg', '-l', 'wine'], 
                                         stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True)
        except subprocess.CalledProcessError:
            return False
        
        # Check for 'zugbruecke' using pip
        try:
            import zugbruecke
            return True
        except subprocess.CalledProcessError:
            return False

    else:
        raise ValueError(f"Unsupported OS: {os_type}")

# Use pytest.mark.skipif to conditionally skip the test class
@pytest.mark.skipif(not has_dependencies(), reason="Required dependencies not installed")
@pytest.mark.plexon2
class Plexon2RecordingTest(RecordingCommonTestSuite, unittest.TestCase):
    ExtractorClass = Plexon2RecordingExtractor
    downloads = ["plexon"]
    entities = [
        ("plexon/4chDemoPL2.pl2", {"stream_id": "3"}),
    ]
        ```

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think that maybe this will help us avoid adding -some- complexity to the testing in the CI but less sure about it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point @h-mayorquin , I'll do something along these lines!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. One last question about the CI logic.

- name: Test core
run: ./.github/run_tests.sh core
- name: Test extractors
if: ${{ steps.modules-changed.outputs.EXTRACTORS_CHANGED == 'true' || steps.modules-changed.outputs.CORE_CHANGED == 'true' }}
run: ./.github/run_tests.sh "extractors and not streaming_extractors"
run: ./.github/run_tests.sh "extractors and not streaming_extractors and not plexon2"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need not here with the new condition that you added? Will it not be skipped automatically when wine is not installed and test it with all the other extractor when wine is installed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nope, removed!

@@ -157,3 +164,6 @@ jobs:
- name: Test internal sorters
if: ${{ steps.modules-changed.outputs.SORTERS_INTERNAL_CHANGED == 'true' || steps.modules-changed.outputs.SORTINGCOMPONENTS_CHANGED || steps.modules-changed.outputs.CORE_CHANGED == 'true' }}
run: ./.github/run_tests.sh sorters_internal
- name: Test plexon2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That way, we would not need this, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

@samuelgarcia samuelgarcia merged commit d050bb4 into SpikeInterface:main Sep 13, 2023
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
extractors Related to extractors module NEO Problem related to NEO IO
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants