Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More defensive access of extra props in filesources #17445

Merged

Conversation

nuwang
Copy link
Member

@nuwang nuwang commented Feb 8, 2024

Based on bug and solution described in: #17442

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

@github-actions github-actions bot added this to the 24.0 milestone Feb 8, 2024
@mvdbeek
Copy link
Member

mvdbeek commented Feb 8, 2024

That doesn't look right, FileProps probably needs some work.

Copy link
Member

@natefoo natefoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nuwang!

I'm not sure why adding an extra attr to FilesSourceProperties caused the existing one not to be present when unset? But assuming extra_props is still set properly when it should be, this solution works for me.

@nuwang
Copy link
Member Author

nuwang commented Feb 9, 2024

@mvdbeek Does this look better?
@natefoo I think there were a few other recent changes that were made to filesources, so the combination might have caused the problem?


# Property overrides for values initially configured through the constructor. For example
# the HTTPFilesSource passes in additional http_headers through these properties, which
# are merged with constructor defined http_headers. The interpretation of these properties
# are filesystem specific.
extra_props: Optional[FilesSourceProperties]
extra_props: Optional[FilesSourceProperties] = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine if you make the class a dataclass ... which may have been the intention from the start ?
Otherwise it's error prone to have this be a class var.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. fixed.

@mvdbeek
Copy link
Member

mvdbeek commented Feb 9, 2024

You can use make format to fix the import linting

@davelopez
Copy link
Contributor

Cool thanks! Should we target a release branch (23.1?) since it is a fix?

@nuwang nuwang force-pushed the make_extra_props_more_defensive branch from c54f5bb to 6a3575d Compare February 9, 2024 11:19
@nuwang nuwang force-pushed the make_extra_props_more_defensive branch from 6a3575d to a644028 Compare February 9, 2024 11:30
@nuwang nuwang changed the base branch from dev to release_23.1 February 9, 2024 11:31
@nuwang
Copy link
Member Author

nuwang commented Feb 9, 2024

I've retargeted 23.1, but it looks like the writable property was introduced in 23.2. Should I target that instead?

@mvdbeek mvdbeek closed this Feb 9, 2024
@mvdbeek mvdbeek reopened this Feb 9, 2024
@mvdbeek
Copy link
Member

mvdbeek commented Feb 9, 2024

I think this is fine as a fix

@nuwang nuwang force-pushed the make_extra_props_more_defensive branch from a644028 to 4a47a43 Compare February 9, 2024 13:47
Co-authored-by: Marius van den Beek <[email protected]>
@nuwang nuwang force-pushed the make_extra_props_more_defensive branch from 4a47a43 to c7a5e4f Compare February 9, 2024 14:08
@mvdbeek mvdbeek merged commit 9d0a200 into galaxyproject:release_23.1 Feb 10, 2024
33 of 37 checks passed
Copy link

This PR was merged without a "kind/" label, please correct.

@nuwang nuwang deleted the make_extra_props_more_defensive branch February 20, 2024 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants