[Feature] Support clickhouse jdbc datasource(#40894) #42930

DataScientistSamChan · 2024-03-21T12:04:59Z

Why I'm doing:

Enhance current clickhouse JDBC datasource support.

What I'm doing:

Support more fine-grained clickhouse JDBC datasource with partition information detection and more fine grained data types.

Fixes #issue
#40894

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

CLAassistant · 2024-03-21T12:05:09Z

All committers have signed the CLA.

starrocks-cr · 2024-03-21T12:09:10Z

fe/fe-core/src/main/java/com/starrocks/connector/jdbc/ClickhouseSchemaResolver.java

+        }
+
+    }
+}


How can I help you today?

be/src/exec/jdbc_scanner.cpp

zombee0

clang-format

zombee0 · 2024-03-21T12:19:25Z

be/src/exec/jdbc_scanner.cpp

        if (type != TYPE_DATE) {
            return Status::NotSupported(
                    fmt::format("Type mismatches on column[{}], JDBC result type is Date, please set the type to date",
                                slot_desc->col_name()));
        }
-        return TYPE_VARCHAR;
+        return TYPE_DATE;


is this ok?

Should be TYPE_VARCHAR, fixed.

zombee0 · 2024-03-22T03:31:35Z

fe/fe-core/src/main/java/com/starrocks/connector/jdbc/ClickhouseSchemaResolver.java

+        }
+    }
+
+    public static void main(String[] args) {


testing code?

Sorry about that, I am kind of using the git for synchronizing the code between local mac and my dev server, hence, the main for test. It's all cleaned up in the latest commit. With unittest added, and function working just fine in my environment.

zombee0 · 2024-03-28T10:56:42Z

.github/workflows/approve-checker.yml

@@ -97,7 +97,7 @@ jobs:
  feature-review:
    needs: info
    runs-on: [ self-hosted, normal ]
-    if: github.event_name == 'pull_request_review' && contains(needs.info.outputs.LABELS, 'feature') && !contains(needs.info.outputs.LABELS, 'sync') && !startsWith(needs.info.outputs.HEAD_REF, 'mergify/')


why is this?

Signed-off-by: samcchen <[email protected]>

sonarcloud · 2024-04-18T03:22:12Z

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

github-actions · 2024-04-18T05:25:14Z

[FE Incremental Coverage Report]

✅ pass : 53 / 66 (80.30%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/connector/jdbc/ClickhouseSchemaResolver.java	49	62	79.03%	[56, 57, 66, 67, 68, 70, 73, 74, 75, 76, 79, 136, 144]
🔵	com/starrocks/catalog/JDBCTable.java	2	2	100.00%	[]
🔵	com/starrocks/connector/jdbc/JDBCMetadata.java	2	2	100.00%	[]

github-actions · 2024-04-18T05:30:33Z

[BE Incremental Coverage Report]

❌ fail : 6 / 34 (17.65%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	be/src/exec/jdbc_scanner.cpp	6	34	17.65%	[199, 201, 202, 204, 206, 207, 209, 211, 212, 215, 217, 227, 228, 231, 233, 250, 251, 253, 255, 271, 272, 274, 276, 327, 328, 329, 331, 333]

zombee0 · 2024-04-19T02:57:06Z

https://github.com/Mergifyio backport branch-3.3

mergify · 2024-04-19T02:57:11Z

backport branch-3.3

✅ Backports have been created

#44415 [Feature] Support clickhouse jdbc datasource(#40894) (backport #42930) has been created for branch branch-3.3

Why I'm doing: Enhance current clickhouse JDBC datasource support. What I'm doing: Support more fine-grained clickhouse JDBC datasource with partition information detection and more fine grained data types. Fixes #issue #40894 Signed-off-by: samcchen <[email protected]> (cherry picked from commit 36501fe)

#44415) Co-authored-by: Sam Chan <[email protected]>

…cks#42930) Why I'm doing: Enhance current clickhouse JDBC datasource support. What I'm doing: Support more fine-grained clickhouse JDBC datasource with partition information detection and more fine grained data types. Fixes #issue StarRocks#40894 Signed-off-by: samcchen <[email protected]>

DataScientistSamChan requested review from a team as code owners March 21, 2024 12:05

mergify bot assigned DataScientistSamChan Mar 21, 2024

starrocks-cr bot reviewed Mar 21, 2024

View reviewed changes

fe/fe-core/src/main/java/com/starrocks/connector/jdbc/ClickhouseSchemaResolver.java

}

}

}

Copy link

starrocks-cr bot Mar 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I help you today?

starrocks-cr bot reviewed Mar 21, 2024

View reviewed changes

be/src/exec/jdbc_scanner.cpp Show resolved Hide resolved

zombee0 reviewed Mar 21, 2024

View reviewed changes

github-actions bot added the behavior_changed label Mar 21, 2024

DataScientistSamChan force-pushed the feature/support_jdbc_clickhouse branch from 2ae27f0 to d64eb2c Compare March 21, 2024 14:12

zombee0 reviewed Mar 22, 2024

View reviewed changes

DataScientistSamChan force-pushed the feature/support_jdbc_clickhouse branch from 9072fbf to 250e2f8 Compare March 25, 2024 10:11

github-actions bot added the feature label Mar 26, 2024

DataScientistSamChan force-pushed the feature/support_jdbc_clickhouse branch from c43d28a to 9f0c483 Compare March 27, 2024 06:30

DataScientistSamChan requested review from a team as code owners March 28, 2024 03:36

wanpengfei-git added the PROTO-REVIEW label Mar 28, 2024

github-actions bot added the documentation Improvements or additions to documentation label Mar 28, 2024

wanpengfei-git requested a review from a team March 28, 2024 03:37

github-actions bot removed the behavior_changed label Mar 28, 2024

DataScientistSamChan force-pushed the feature/support_jdbc_clickhouse branch from 362508c to 7228a86 Compare April 8, 2024 06:35

wanpengfei-git removed the PROTO-REVIEW label Apr 8, 2024

github-actions bot removed the documentation Improvements or additions to documentation label Apr 8, 2024

wanpengfei-git removed the request for review from a team April 8, 2024 06:36

wangsimo0 previously approved these changes Apr 8, 2024

View reviewed changes

zombee0 approved these changes Apr 8, 2024

View reviewed changes

Youngwb previously approved these changes Apr 11, 2024

View reviewed changes

dirtysalt previously approved these changes Apr 12, 2024

View reviewed changes

wangsimo0 previously approved these changes Apr 12, 2024

View reviewed changes

DataScientistSamChan dismissed stale reviews from wangsimo0, dirtysalt, and Youngwb via 9c4379b April 12, 2024 06:38

DataScientistSamChan force-pushed the feature/support_jdbc_clickhouse branch 9 times, most recently from 0f9f70a to 4014e46 Compare April 18, 2024 01:35

[Feature] Support clickhouse jdbc datasource(StarRocks#40894)

8850898

Signed-off-by: samcchen <[email protected]>

DataScientistSamChan force-pushed the feature/support_jdbc_clickhouse branch from 4014e46 to 8850898 Compare April 18, 2024 02:13

wangsimo0 approved these changes Apr 18, 2024

View reviewed changes

imay merged commit 36501fe into StarRocks:main Apr 18, 2024
47 of 49 checks passed

mergify bot mentioned this pull request Apr 19, 2024

[Feature] Support clickhouse jdbc datasource(#40894) (backport #42930) #44415

Merged

41 tasks

wanpengfei-git pushed a commit that referenced this pull request Apr 19, 2024

[Feature] Support clickhouse jdbc datasource(#40894) (backport #42930) (

dd0f6a9

#44415) Co-authored-by: Sam Chan <[email protected]>

github-actions bot added the 3.3-merged label Apr 19, 2024

Dshadowzh mentioned this pull request Apr 22, 2024

Starrocks Can I use catalog to query ClickHouse? If so, what should I do? #38890

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support clickhouse jdbc datasource(#40894) #42930

[Feature] Support clickhouse jdbc datasource(#40894) #42930

DataScientistSamChan commented Mar 21, 2024 •

edited by Youngwb

Loading

CLAassistant commented Mar 21, 2024 •

edited

Loading

starrocks-cr bot Mar 21, 2024

zombee0 left a comment

zombee0 Mar 21, 2024

DataScientistSamChan Mar 21, 2024

zombee0 Mar 22, 2024

DataScientistSamChan Mar 25, 2024

zombee0 Mar 28, 2024

sonarcloud bot commented Apr 18, 2024

github-actions bot commented Apr 18, 2024

github-actions bot commented Apr 18, 2024

zombee0 commented Apr 19, 2024

mergify bot commented Apr 19, 2024 •

edited

Loading

+                      }
+                  }
+              }

[Feature] Support clickhouse jdbc datasource(#40894) #42930

[Feature] Support clickhouse jdbc datasource(#40894) #42930

Conversation

DataScientistSamChan commented Mar 21, 2024 • edited by Youngwb Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

CLAassistant commented Mar 21, 2024 • edited Loading

starrocks-cr bot Mar 21, 2024

Choose a reason for hiding this comment

zombee0 left a comment

Choose a reason for hiding this comment

zombee0 Mar 21, 2024

Choose a reason for hiding this comment

DataScientistSamChan Mar 21, 2024

Choose a reason for hiding this comment

zombee0 Mar 22, 2024

Choose a reason for hiding this comment

DataScientistSamChan Mar 25, 2024

Choose a reason for hiding this comment

zombee0 Mar 28, 2024

Choose a reason for hiding this comment

sonarcloud bot commented Apr 18, 2024

Quality Gate passed

github-actions bot commented Apr 18, 2024

[FE Incremental Coverage Report]

file detail

github-actions bot commented Apr 18, 2024

[BE Incremental Coverage Report]

file detail

zombee0 commented Apr 19, 2024

mergify bot commented Apr 19, 2024 • edited Loading

✅ Backports have been created

DataScientistSamChan commented Mar 21, 2024 •

edited by Youngwb

Loading

CLAassistant commented Mar 21, 2024 •

edited

Loading

mergify bot commented Apr 19, 2024 •

edited

Loading