Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

delete some intermediate tables #633

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Conversation

gyafdxis-k
Copy link

Whether we try to delete some intermediate tables. Deletion using regular matching, such as tables starting with ^tmp can be deleted in the blood relationship. The upstream and downstream of tables starting with ^tmp can be directly connected according to the Cartesian product.

1234

add del lineage which is table and table name contain tmp or temp
add del lineage which is table and table name contain tmp or temp
@gyafdxis-k
Copy link
Author

del some intermediate tables function but I think this feature can be config

@delphisharp
Copy link
Contributor

tmp or temp is a business definition.
I don't think it's a good idea to introduce business definitions.
Business processing should be done at the business level.
What if my temporary table definition is based on the schema name? Or other logic?

@delphisharp
Copy link
Contributor

Is there any problem in processing business logic on the parsing results?
I mean, a tool should have its own boundaries, so I don't think this business processing should be done at the tool level.

If you want to do it, you should use an interface or other more compatible methods.
Scenarios I encountered

  1. Create Session Table in teradata
  2. Schema is Tmp or ods_temp, is temp table
    In this case, I process the parsing results at the business level.

For the current metadata_provider, I also think that providing a callback function is enough, and how to implement it is up to the developer to handle it.

@gyafdxis-k
Copy link
Author

mp or temp is a business definition.
I don't think it's a good idea to introduce business definitions.
Business processing should be done at the business level.
What if my temporary table definition is based on the schema name? Or other logic?

It is only a example, I think the callback function is a good idea which is friendly to the user
thanks for your replying, I think the sqlineage is very good opensource project

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants