Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ETL job errors if there are no videos to process and parallel processing is requested #19

Closed
galenlynch opened this issue Nov 22, 2024 · 1 comment

Comments

@galenlynch
Copy link
Collaborator

I would expect this to create the output directory and exit instead of erroring.

$ python -m aind_behavior_video_transformation.etl -j '{"input_source": "foo", "output_directory": "bar",  "compression_requested": {"compresison_enum": "gamma fix colorspace"}}'
<frozen runpy>:128: RuntimeWarning: 'aind_behavior_video_transformation.etl' found in sys.modules after import of package 'aind_behavior_video_transformation', but prior to execution of 'aind_behavior_video_transformation.etl'; this may result in unpredictable behaviour
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/galen.lynch/Documents/Code/aind-behavior-video-transformation/src/aind_behavior_video_transformation/etl.py", line 183, in <module>
    job_response = job.run_job()
  File "/home/galen.lynch/Documents/Code/aind-behavior-video-transformation/src/aind_behavior_video_transformation/etl.py", line 153, in run_job
    self._run_compression(convert_video_args)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/home/galen.lynch/Documents/Code/aind-behavior-video-transformation/src/aind_behavior_video_transformation/etl.py", line 87, in _run_compression
    with ProcessPoolExecutor(max_workers=num_jobs) as executor:
         ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/concurrent/futures/process.py", line 666, in __init__
    raise ValueError("max_workers must be greater than 0")
ValueError: max_workers must be greater than 0

with ProcessPoolExecutor(max_workers=num_jobs) as executor:

@galenlynch
Copy link
Collaborator Author

Closed by #20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant