Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storing into Kestra Internal Storage does not take into account illegal characters #5589

Closed
johnkm516 opened this issue Oct 22, 2024 · 3 comments · Fixed by #5593
Closed
Assignees
Labels
area/backend Needs backend code changes bug Something isn't working kind/customer-request Requested by one or more customers

Comments

@johnkm516
Copy link

Describe the issue

I'm trying to download a file into Kestra's internal storage using the io.kestra.plugin.core.http.Download task, but this task fails because the filename of the downloaded file has illegal characters for the URI. Here's the error log for the download task :

Illegal character in path at index 122: kestra:///plm/complm-cm-dd-005-worker/executions/1QE1b2wn7AS3yoJTtqKKmm/tasks/centric-downloadfile/1E8tP0Znk8oTzN33AEVRYv/[TEST용]-Adobe-Artboard-for-APP_Sarah-23.png

I have no control over the endpoint and the endpoint itself does not support renaming the file in the request. Kestra fails to add the file to the internal storage because [,] are illegal characters for URI.

I suspect this issue will occur for any download of files or any time a file is stored in internal storage with the filename set by external factors, not just exclusive to HTTP Download.

My proposed solution would be to automatically url encode the filename any time Kestra stores something in internal storage, and automatically decode it during retrieval.

Environment

  • Kestra Version: 0.19.4
  • Operating System (OS/Docker/Kubernetes): Docker
@johnkm516 johnkm516 added the bug Something isn't working label Oct 22, 2024
@MilosPaunovic MilosPaunovic added area/backend Needs backend code changes kind/customer-request Requested by one or more customers labels Oct 22, 2024
@Malaydewangan09
Copy link
Contributor

Hey @MilosPaunovic @johnkm516
Can I pick this up?
Thanks!

@MilosPaunovic
Copy link
Member

It's fine by me, but let's first close #5590, as you're already assigned there. After, you can switch to this issue.

@Malaydewangan09
Copy link
Contributor

Sure, Thanks!

Skraye added a commit that referenced this issue Oct 22, 2024
Skraye added a commit that referenced this issue Oct 22, 2024
Skraye added a commit that referenced this issue Oct 22, 2024
Skraye added a commit that referenced this issue Oct 22, 2024
Skraye added a commit that referenced this issue Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backend Needs backend code changes bug Something isn't working kind/customer-request Requested by one or more customers
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

4 participants