Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there any plan to update the code or the document #200

Closed
anewlearner opened this issue Sep 30, 2019 · 8 comments
Closed

Is there any plan to update the code or the document #200

anewlearner opened this issue Sep 30, 2019 · 8 comments

Comments

@anewlearner
Copy link

No description provided.

@cbrnr
Copy link
Contributor

cbrnr commented Sep 30, 2019

Do you have any features in mind that you would like to see in SCoT? Currently, I don't have much time to work on this project, but I am certainly maintaining the package and planning for the future. At some point, I'd like to get this into MNE (or at least make it much easier to interact with MNE).

@mbillingr
Copy link
Member

Same here. Currently, I do not work with EEG/MEG so I have no personal use for the package. I'd like to keep it running, though. If there are any issues with recent versions of Python or dependencies I'm happy to fix them. Feel free to open issues. Pull requests are welcome too! :)

@anewlearner
Copy link
Author

This project is a great work. I'm new to EEG and used it to do some experiment(comparing with other types of features). The AUC of this method is good. But the models obtained on training data cannot be saved. When I want to get prediction results on the test data at other times, I have to retrain the model. What's more, the speed is not very fast with a lot of training data.
Do you have any suggestion about how to fix the problem? I need to solve this problem urgently. Thanks a lot.

@cbrnr
Copy link
Contributor

cbrnr commented Oct 3, 2019

I'm not sure if I understood your issues correctly, but I assume you use scikit-learn to train and test your model. Saving models should therefore be handled by scikit-learn and you should consult the corresponding documentation. Computation speed is also related to scikit-learn (unless feature extraction, i.e. calculation of connectivity measures is the bottleneck).

@anewlearner
Copy link
Author

I did not explain my problem clearly and will illustrate it using an example. The first figure shows current process. I want to change the process to something showed in the second figure. That is saving the fitted scot models(fitted var model and so on) and then loading it at any time I want without fitting again(ws.do_cspvarica()). In this way, the training and testing process can be separated totally. The problem now is that 'ws' can not saved/pickled.
image

image

@mbillingr
Copy link
Member

@anewlearner It sounds like the bottleneck is feature extraction, so I'd suggest to pre-compute both, train and test features for later reuse. Does the following workflow solve your problem?

for train, test in kf:
    ws.set_data(data[train], classes[train])
    ws.do_cspvarica()

    features_train = get_features(ws, data, train)
    features_test = get_features(ws, data, test)

    # TODO: store train and test features for current fold

for train, test in kf:
    # TODO: load train and test features for current fold

    lda.fit(features_train, classids[train])
    lda.score(features_test, classids[test])

@anewlearner
Copy link
Author

@mbillingr It solved the problem in the above case that used for illustration. We need to save the 'ws' which can not used to get features in reality.

@mbillingr
Copy link
Member

Since this is essentially a request for a save/restore feature the Workspace class I'm closing this in favor of #201 to keep things tidy. Please discuss the feature in that thread.

I think we have addressed the other issues raised here. Feel free to reopen if this was not the case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants