Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict enabling CDF on tables with reserved column names #524

Open
OussamaSaoudi-db opened this issue Nov 22, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@OussamaSaoudi-db
Copy link
Collaborator

Please describe why this is necessary.

Change data feed generates three columns: _commit_timestamp, _commit_version, and _change_type. These columns are generated by TableChanges.

TableChanges assumes that these column names do not exist. Moreover spark disallows enabling CDF on tables with these column names.

When writing tables, we must ensure that CDF cannot be enabled on tables with the reserved column names.

Describe the functionality you are proposing.

Ensure that when writing tables or metadata updates, a table may not have CDF enabled and have the reserved columns.

Additional context

No response

@scovich
Copy link
Collaborator

scovich commented Nov 22, 2024

Also we need to block adding those columns to a CDF-enabled table

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants