We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change data feed generates three columns: _commit_timestamp, _commit_version, and _change_type. These columns are generated by TableChanges.
_commit_timestamp
_commit_version
_change_type
TableChanges
TableChanges assumes that these column names do not exist. Moreover spark disallows enabling CDF on tables with these column names.
When writing tables, we must ensure that CDF cannot be enabled on tables with the reserved column names.
Ensure that when writing tables or metadata updates, a table may not have CDF enabled and have the reserved columns.
No response
The text was updated successfully, but these errors were encountered:
Also we need to block adding those columns to a CDF-enabled table
Sorry, something went wrong.
No branches or pull requests
Please describe why this is necessary.
Change data feed generates three columns:
_commit_timestamp
,_commit_version
, and_change_type
. These columns are generated byTableChanges
.TableChanges
assumes that these column names do not exist. Moreover spark disallows enabling CDF on tables with these column names.When writing tables, we must ensure that CDF cannot be enabled on tables with the reserved column names.
Describe the functionality you are proposing.
Ensure that when writing tables or metadata updates, a table may not have CDF enabled and have the reserved columns.
Additional context
No response
The text was updated successfully, but these errors were encountered: