Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

edr 'dbt deps' step running into Windows file path length limits #1679

Open
JennaFlan2426 opened this issue Aug 23, 2024 · 0 comments
Open
Labels
Bug Something isn't working Triage 👀

Comments

@JennaFlan2426
Copy link

Describe the bug
edr commands such as 'edr report' fail with a FileNotFound error regarding "dbt_packages\dbt-data-reliability-0.14.0\integration_tests\deprecated_tests\macros\unit_tests\test_adapter_specific_macros_have_default_implementation.sql". (dbt-data-reliability-0.14.0 folder name may change with different Elementary version numbers)

Failure occurs during the 'dbt deps' preparatory step when EDR attempts to install required packages within the folder structure of the Elementary Python module. When I run "dbt deps --project-dir " the install of dbt-utils package is successful (therefore I can confirm write permissions to the directory structure) but install of elementary package fails during untar of the above file.

Complete path from Python environment folder is "\Lib\site-packages\elementary\monitor\dbt_project\dbt_packages\dbt-data-reliability-0.14.0\integration_tests\deprecated_tests\macros\unit_tests\test_adapter_specific_macros_have_default_implementation.sql" which is 204 characters long and risks exceeding Windows 256 character filepath limits. Therefore, untar of the elementary package in the desired location fails with a misleading "File Not Found" error (the problem actually is untar can't WRITE the file because the path is too long).

Possible cause of #1261 which was closed as a "local environment" issue. I wish to leave this as at least an information breadcrumb for other users... and also ask the team to consider filepath depth in future naming choices for those of us stuck with Windows. Thanks!

To Reproduce
Steps to reproduce the behavior:

  1. Create a Python environment with a Windows file path greater than 52 characters, such as "C:\Users\JenniferFlanagan\GitHub\analytics-dbt-project.dbtenv"
  2. pip install dbt and elementary
  3. Navigate to <environment_path>\Lib\site-packages\elementary\monitor\dbt_project\
  4. Attempt to run dbt deps (edr commands do this as a preparatory step)
  5. Logs should indicate that untar of the elementary package fails on file "\dbt_packages\dbt-data-reliability-0.14.0\integration_tests\deprecated_tests\macros\unit_tests\test_adapter_specific_macros_have_default_implementation.sql" with a "File Not Found" error
  6. To confirm, retry these steps using a shorter environment path such as "C:\Users\JenniferFlanagan.testenv" (34 characters; long path will total 238), dbt deps will run without error

Expected behavior
EDR commands succeed on dbt deps step

Environment (please complete the following information):

  • Elementary CLI (edr) version: 0.14.0
  • Elementary dbt package version: 0.14.1
  • dbt version: 1.7.9 (also tested after 1.8.5 upgrade)
  • Data warehouse: Postgres
  • Infrastructure details: Windows OS

Additional context
Running 'dbt deps' manually from the <environment_path>\Lib\site-packages\elementary\monitor\dbt_project\ also resolved a less-obvious "failed to run dbt command" error encountered while trying to get 'edr report' to succeed.

Would you be willing to contribute a fix for this issue?
Willing, not sure about able.

@JennaFlan2426 JennaFlan2426 added Bug Something isn't working Triage 👀 labels Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Triage 👀
Projects
None yet
Development

No branches or pull requests

1 participant