Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HUDI-7262][0.x] Validate checksum only if it exists #10764

Merged
merged 1 commit into from
Feb 28, 2024

Conversation

stream2000
Copy link
Contributor

@stream2000 stream2000 commented Feb 27, 2024

Change Logs

Only validate the checksum if it exists.
Otherwise, some operations would throw exceptions. For example:

If a HUDI table was created with 0.10 Hudi version and read by 0.14.0, an IllegalArgumentExeption would be thrown out because checksum property doesn't exist in hoodie.properties.
Fail to upgrade a table created with 0.10 HUDI version using 0.14 version, an IllegalArgumentExeption would be thrown out because checksum property doesn't exist in hoodie.properties.
See more information in #10404.

Origin PR on master by @beyond1920 : #10417

Impact

None

Risk level (write none, low medium or high below)

None

Documentation Update

None

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@stream2000
Copy link
Contributor Author

@beyond1920 @yihua Hi, this fix is important for reading 0.10.x or earlier version of hudi table using 0.x-branch. I pick it from 1.x branch, PTAL~

@stream2000
Copy link
Contributor Author

Oh, the conflict is wired. @yihua Or could you help pick this commit directly to 0.x branch? Thanks~

@yihua
Copy link
Contributor

yihua commented Feb 27, 2024

Hi @stream2000 thanks for the contribution. I've updated branch-0.x by cherry-picking features and fixes till last Thursday. Could you help rebase and fix the conflict? I'm also going to enable CI on PRs targeting branch-0.x.

@stream2000
Copy link
Contributor Author

Could you help rebase and fix the conflict? I'm also going to enable CI on PRs targeting branch-0.x.

Sure! Already rebased fixed the conflicts.

@stream2000 stream2000 changed the title [HUDI-7262] Validate checksum only if it exists (#10417) [HUDI-7262] Validate checksum only if it exists Feb 27, 2024
@yihua
Copy link
Contributor

yihua commented Feb 27, 2024

Thanks! Here's the PR to enable CI on branch-0.x and PRs targeting it: #10765

@yihua
Copy link
Contributor

yihua commented Feb 27, 2024

@stream2000 now CI works. I have to cherry-pick my changes on GH workflows on master to branch-0.x too.

@github-actions github-actions bot added the size:S PR with lines of changes in (10, 100] label Feb 27, 2024
Copy link
Contributor

@yihua yihua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Could you also add the link to the original PR targeting master?

@stream2000
Copy link
Contributor Author

LGTM. Could you also add the link to the original PR targeting master?

Sure. Added the link in the Pr description.

@hudi-bot
Copy link

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@danny0405 danny0405 merged commit 1bc3e41 into apache:branch-0.x Feb 28, 2024
22 of 32 checks passed
@yihua yihua changed the title [HUDI-7262] Validate checksum only if it exists [HUDI-7262][0.x] Validate checksum only if it exists Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:S PR with lines of changes in (10, 100]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants