[GOBBLIN-2165] Support partial commit semantic from CopyDataPublisher

[Will-Lo]

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-2165

Description

• Here are some details about my PR, including screenshots (if applicable):
  When Gobblin is using partial commit semantics, it is possible for files to be missing on the destination after the job is completed (in line with the semantic where any number of files may be missing, as long as one file is successfully published it can still be considered successful).

Currently, Gobblin will raise a FileNotFoundException when this file is being published if its performs a metadata sync, which prevents the job from running through its postpublish steps. We still want the postpublish steps to go through as they may contain important steps for the job to be considered at least partially successful (permission sync, table registration).

This PR allows one to add the configurations

job.commit.policy: successful // or job.commit.policy=partial
job.commit.partial.fail.task.fails.job.commit=true

And have a job publish all of its successful files but still mark the job as failed.
It is important that certain table copies (e.g. iceberg distcp) should not be supporting partial copy semantics in the same way as it would break the table manifest. Also care should be taken with hive copies so that subsequent copies will still copy the missing files (turn off fast partition skip predicate).

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[phet]

nit: the commit message is confusing - This PR allows one to add the configurations... and requires really close reading to realize it's not really about those two partial success params, but more about preventing internal corruption when they're enabled.

it does however raise a good point about how the params would be problematic for iceberg-distcp. would we want the IcebergDatasetFinder to fail-fast if it finds those params set?

[phet]

I'm glad the fix was a mere reordering. still, that is so subtle, it definitely deserves a comment (e.g. about what would transpire if preserving attrs prior to verifying the associate file was COMMITTED)

[Will-Lo]

Added a comment