Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change sampling strategy #15

Open
tribhuvanesh opened this issue Nov 21, 2014 · 0 comments
Open

Change sampling strategy #15

tribhuvanesh opened this issue Nov 21, 2014 · 0 comments
Assignees

Comments

@tribhuvanesh
Copy link
Contributor

Right now, by setting solverOptions.sampleFrac, say 0.5, the master node samples 50% of the training data and hands each of the K workers 50/K% of the data.

This does not scale well. Since, adding more workers forces the users to change the sampling fraction in order to scale.
Rather, each worker should independently sample the data and then perform the computation.
Doing so implies adding more workers results in faster convergence.

@tribhuvanesh tribhuvanesh self-assigned this Nov 21, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant