Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LogisticRegressionClassifier out of memory #553

Open
Dadibom opened this issue Oct 23, 2020 · 2 comments
Open

LogisticRegressionClassifier out of memory #553

Dadibom opened this issue Oct 23, 2020 · 2 comments
Labels
Help/Questions

Comments

@Dadibom
Copy link

Dadibom commented Oct 23, 2020

Trying to train LogisticRegressionClassifier with a large dataset. LogisticRegressionClassifier.train() causes the heap to run out of memory.

The data in question is too large to be able to simply increase the heap size, so I am looking for a solution to train in batches. However, I cannot find any references in the documentation to this.

What is the intended way to train a classifier in batches?
@SirPhemmiey
Copy link

Please any solution to this yet?

@Hugo-ter-Doest
Copy link
Collaborator

You could try training in parallel with the functions implemented in classifier_train_parallel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Help/Questions
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Hugo-ter-Doest @Dadibom @SirPhemmiey