-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[hotfix-#1799][connector][jdbc] Fix related issues when the write mode is replace/update #1800
Conversation
主要是为了让插件的行为和SQL语义保持一致 |
…ite mode is replace/update [hotfix-DTStack#1799][connector][jdbc] Fixed related issues when the write mode is replace/update
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -52,4 +54,10 @@ public class JdbcSinkOptions { | |||
.stringType() | |||
.noDefaultValue() | |||
.withDescription("sink.post-sql"); | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里的WriteMode看起来只是决定了OuputFormat#OpenInternal里面是否会从元数据中获取UniqueKey,不会导致WriteMode真正的变更,看起来不太恰当
解释一下为何之前没有地方设置写入方式:之前WriteMode会取决于是否有主键,一般来说有主键的情况下基本就是回撤流,需要upsert语义,而UpsertDeleteWrapper也能兼容appendOnly的场景,这里应该是没有问题的(我们没法好的办法判断当前connector是否会接受回撤流,所以只好用主键来决策)
@@ -103,6 +106,23 @@ protected void openInternal(int taskNumber, int numTasks) { | |||
log.info("updateKey = {}", JsonUtil.toJson(tableIndex)); | |||
} | |||
} | |||
if (!CollectionUtil.isNullOrEmpty(jdbcConfig.getUniqueKey())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里会强制获取表中的主键、唯一索引信息。 SQL作业应该尽量遵从SQL本身的语义。比如SQL中只写了一个主键,前面的groupby等操作一般都是基于这个唯一的主键去做的。但是在这里获取了额外的信息,那么实际写操作的时候,就会和SQL本身语义不符
LGTM |
…
Purpose of this pull request
Which issue you fix
Fixes #1799
Checklist: