Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HUDI-6787] Integrate the new file group reader with Hive query engine #10422

Merged
merged 43 commits into from
Jun 9, 2024
Merged

[HUDI-6787] Integrate the new file group reader with Hive query engine #10422

merged 43 commits into from
Jun 9, 2024
+1,470 −128

Commits on May 31, 2024

  1. squash commits

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    aa6e1c2 View commit details
    Browse the repository at this point in the history

  2. refactor a bit and add some comments

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    3674390 View commit details
    Browse the repository at this point in the history

  3. make build properly

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    edad863 View commit details
    Browse the repository at this point in the history

  4. fix some of the failing tests

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    b764026 View commit details
    Browse the repository at this point in the history

  5. revert to old impl when schema evolution enabled

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    6647869 View commit details
    Browse the repository at this point in the history

  6. disable fg reader for stupid test

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    b43949b View commit details
    Browse the repository at this point in the history

  7. fix some failing tests

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    2bc441f View commit details
    Browse the repository at this point in the history

  8. assigned the ports backwards

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    1bcbab8 View commit details
    Browse the repository at this point in the history

  9. verbose output bundle validation

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    6e5fadd View commit details
    Browse the repository at this point in the history

  10. add volume to docker compose

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    dc4ac6f View commit details
    Browse the repository at this point in the history

  11. need to lowercase the field names omg

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    8669cce View commit details
    Browse the repository at this point in the history

  12. don't remove partition if it's listed in the schema

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    fdeea22 View commit details
    Browse the repository at this point in the history

  13. put partition cols at end of output

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    60fe6fa View commit details
    Browse the repository at this point in the history

  14. invert filter

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    ad3f3d3 View commit details
    Browse the repository at this point in the history

  15. support no base file, only log. support read from non-hudi table

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    6a77480 View commit details
    Browse the repository at this point in the history

  16. disable for skip merge as well

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    51f91c7 View commit details
    Browse the repository at this point in the history

  17. fix non hoodie path read

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    1da1e76 View commit details
    Browse the repository at this point in the history

  18. revert setting precombine

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    2b1f172 View commit details
    Browse the repository at this point in the history

  19. fix no meta cols table

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    7b45027 View commit details
    Browse the repository at this point in the history

  20. check if no requested fields

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    16cff40 View commit details
    Browse the repository at this point in the history

  21. create empty schema properly

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    e115fd5 View commit details
    Browse the repository at this point in the history

  22. check if metadata folder exists

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    211bcb9 View commit details
    Browse the repository at this point in the history

  23. handle mor with no meta fields

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    73bedb4 View commit details
    Browse the repository at this point in the history

  24. disable reader for a test because mor seems to work different

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    3f412a1 View commit details
    Browse the repository at this point in the history

  25. delete partition column from the jobconf if it is written in the file

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    51a47e4 View commit details
    Browse the repository at this point in the history

  26. modify data schema due to partition column madness

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    a8020bd View commit details
    Browse the repository at this point in the history

  27. remove unused import

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    25078f3 View commit details
    Browse the repository at this point in the history

  28. add some comments

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    c225f52 View commit details
    Browse the repository at this point in the history

  29. don't add partition fields when the data schema doesn't have them

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    069a5ad View commit details
    Browse the repository at this point in the history

  30. address review feedback

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    185217d View commit details
    Browse the repository at this point in the history

  31. accidently put remove in for loop for combine reader

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    6d58742 View commit details
    Browse the repository at this point in the history

  32. get building again

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    325a8b7 View commit details
    Browse the repository at this point in the history

  33. address some review comments

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    a012522 View commit details
    Browse the repository at this point in the history

  34. add reviewer suggested change

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    50bd39f View commit details
    Browse the repository at this point in the history

  35. add missing params fg reader

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    50a308d View commit details
    Browse the repository at this point in the history

  36. address some comments

    @codope
    Jonathan Vexler authored and codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    5a33e85 View commit details
    Browse the repository at this point in the history

  37. Fix post rebase

    @codope
    codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    18fbd92 View commit details
    Browse the repository at this point in the history

  38. fix hudi table check

    @codope
    codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    9487e12 View commit details
    Browse the repository at this point in the history

  39. Address self feedback

    @codope
    codope committed May 31, 2024
    Configuration menu
    Copy the full SHA
    2201cb0 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. Merge branch 'master' into use_fg_reader_hive

    Jonathan Vexler committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    b29ff63 View commit details
    Browse the repository at this point in the history

  2. fix failing test

    Jonathan Vexler committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    33249cc View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2024

  1. Merge branch 'master' into use_fg_reader_hive

    Jonathan Vexler committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    89a4a8c View commit details
    Browse the repository at this point in the history

  2. fix failing test

    Jonathan Vexler committed Jun 8, 2024
    Configuration menu
    Copy the full SHA
    e95bcb8 View commit details
    Browse the repository at this point in the history