Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #513] Support hologres connector #514

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ferrirW
Copy link

@ferrirW ferrirW commented Jun 2, 2023

support hologres connectors (both source and sink)

link: #513

Data flow build, send and receive data success in hologres 1.3.42 version.
But the data must be structured, Otherwise, the type conversion will fail, such as: fileSource -> holoSink

Alternatively, perhaps fileSource could be modified to support JSON-type structured data instead of being transmitted in rows

@ferrirW ferrirW force-pushed the support-hologres-connector-dev branch from e67758f to e10e266 Compare June 2, 2023 05:01
@ferrirW ferrirW force-pushed the support-hologres-connector-dev branch from e10e266 to 44b80d8 Compare June 7, 2023 03:29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gitignore already exists, is this necessary?

- **connector-class**: connector的类名
- **tasks.num**: 启动的task数目

##### parameter configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感觉可以标注一下是source还是sink。比如使用三级标题

return null;
}

if (++count % 1000 == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

或许count需要reset一下,避免溢出

} catch (Exception e) {
log.error("Error while polling data from Hologres", e);
}
return records;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

每次poll只拉一个record下来吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants