Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Check referencedDataFile existence for DV #12088

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ebyhr
Copy link
Contributor

@ebyhr ebyhr commented Jan 24, 2025

This field is required for DV. I missed setting this field when preparing deletion vector's changes in Trino, and it resulted in incorrect results even though DELETE statement succeeded. It would be nice to fail early.

/**
* Returns the location of a data file that all deletes reference.
*
* <p>The referenced data file is required for deletion vectors and may be optionally captured for
* position delete files that apply to only one data file. This method always returns null for
* equality delete files.
*/
default String referencedDataFile() {
return null;
}

@github-actions github-actions bot added the core label Jan 24, 2025
@@ -255,6 +255,8 @@ public DeleteFile build() {
if (format == FileFormat.PUFFIN) {
Preconditions.checkArgument(contentOffset != null, "Content offset is required for DV");
Preconditions.checkArgument(contentSizeInBytes != null, "Content size is required for DV");
Preconditions.checkArgument(
referencedDataFile != null, "Referenced data file is required for DV");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, could you please add a small test for this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding and fixing this @ebyhr! +1 for adding a simple test

Copy link
Contributor Author

@ebyhr ebyhr Jan 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your reviews! Added a test to TestDeleteFiles because there is no dedicated test class for FileMetadata & DeleteFile. Let me know if there is a better place or I should create a new test class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants