Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using PYTHONPATH or alternative in streamparse #474

Open
z11373 opened this issue Jan 27, 2020 · 1 comment
Open

using PYTHONPATH or alternative in streamparse #474

z11373 opened this issue Jan 27, 2020 · 1 comment

Comments

@z11373
Copy link

z11373 commented Jan 27, 2020

We have our python module that is called from our streamparse topology, right now we dpkg the module in the site-packages folder, but we want to have a way to override the location, for example if engineer wants to try his change, he/she can just redirect streamparse to look for the module at specific location (instead of in site-packages folder).
Will setting PYTHONPATH work for our case? I think it'll work for submitting topology only, but then likely fail when Storm spawn the spout/bolt process (unless the env var is being propagated). If setting PYTHONPATH is not right solution for our case, is there a recommended solution? Thanks!

@z11373
Copy link
Author

z11373 commented Jan 30, 2020

Our case is we have our python modules (used in spout/bolt code) automatically installed on the machines, and on dev machine engineer sometimes want to run their own version for quick test, etc. We want to tell streamparse to temporary look for the modules in that override location, instead of the installed location. I really appreciate if someone can share their experience on this matter, or provide us guidelines. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant