Release 1.2.10

What's Changed

Features

• Supports reading JSON columns. #334
• Supports reading ARRAY, STRUCT, and MAP columns. #347
• Supports LZ4 compression when sinking data with the JSON format. #354
• Supports Flink 1.19. #379

Improvements

• Supports configuring socket timeout. #319
• The Stream Load transaction interface supports asynchronous prepare and commit operations. #328
• Supports mapping a subset of columns in a StarRocks table to a Flink source table. #352
• Supports setting a specific warehouse when using the Stream Load transaction interface. #361

Bug Fixes

• StarRocksSourceBeReader in StarRocksDynamicLookupFunction is not closed after data reading completes. #351
• An exception was thrown when loading an empty JSON string into a JSON column. #380

Contributors

banmoy, chenhaifengkeda, DanRoscigno, Lyang77

Release 1.2.9

What's Changed

This release includes some features and bug fixes. The notable change is that the connector can be integrated with Flink CDC 3.0 to easily build a streaming ELT pipeline from CDC sources (such as MySQL, Kafka) to StarRocks. You can see Flink CDC Synchronization for details.

Features

• Implement catalog to support Flink CDC 3.0. #295
• Implement new sink api in FLP-191 to support Flink CDC 3.0. #301
• Support Flink 1.18. #305

Bug Fixes

• Fix misleading thread name and log. #290
• Fix wrong configurations when writing to multiple tables . #298

Contributors

Jin-H, zaorangyang, banmoy

Release 1.2.8

What's Changed

This release includes some improvements and bug fixes. The notable changes are as follows:

• Support Flink 1.16 and 1.17.
• Recommend to set sink.label-prefix when the sink is configured to guarantee the exactly-once semantics. For the specific instructions, see Exactly Once.

Improvements

• Support to configure whether to use Stream Load transaction interface to guarantee at-least-once. #228
• Add retry metrics for sink V1. #229
• No need to getLabelState when EXISTING_JOB_STATUS is FINISHED. #231
• Remove useless stack trace log for sink V1. #232
• [Refactor] Move StarRocksSinkManagerV2 to stream-load-sdk. #233
• Automatically detect partial updates according to a Flink table's schema instead of the sink.properties.columns parameter explicitly specified by users. #235
• [Refactor] Move probeTransactionStreamLoad to stream-load-sdk. #240
• Add git-commit-id-plugin for stream-load-sdk. #242
• Use info log for DefaultStreamLoader#close. #243
• Support to generate stream-load-sdk JAR file without dependencies. #245
• Replace fastjson with jackson in stream-load-sdk. #247
• Support to process update_before record. #250
• Add the Apache license into files. #251
• Support to get the exception in stream-load-sdk. #252
• Enable strip_outer_array and ignore_json_size by default. #259
• Try to cleanup lingering transactions when a Flink job restores and the sink semantics is exactly-once. #271
• Return the first exception after the retrying fails. #279

Bug Fixes

• Fix typos in StarRocksStreamLoadVisitor. #230
• Fix the fastjson classloader leak. #260

Tests

• Add the test framework for loading from Kafka to StarRocks. #249

Doc

• Refactor the docs. #262
• Improve the doc for the sink. #268 #275
• Add examples of DataStream API for the sink. #253

Contributors

EMsnap, szza, zaorangyang, hellolilyliuyi, banmoy

Release 1.2.7

What's Changed

Improvements

• Support to configure 100-continue wait timeout #208
• Add pull request template #204
• Exit getLabelState() if retry for more than a certain time #212
• Use git-commit-id-plugin to get git information at runtime #213

Bug Fixes

• Check the http response status code for (transaction) stream load #203
• The returned result for transaction rollback should be "OK" #205
• The frequency of loading in a transaction should only be controlled by maxCacheBytes #206
• Fix that transaction stream load can't configure timeout #210
• Don't commit if the data is never flushed #216

Contributors

PengFei Li

Release 1.2.6

What's Changed

Improvements

• Optimized the performance of sink V2 by using batch processing, making write and stream load parallel, and using HeapByteBuffer.get() to make data copy more efficient. #181

Features

• Added the ability to use transaction stream load for at-least-once mode, which requires StarRocks version 2.4 or higher. #185

Bug Fixes

• Fixed an issue with column names containing SQL keywords in the StarRocks Connector for Apache Flink by adding backticks to column names in the HTTP header. #173
• Correctly call the complete() method on a Region object after checking the state of the label. #176
• Fixed the issue of not using HTTP chunk to send JSON format data for StarRocks versions <=2.2. #174
• Resolved the issue of adding backticks when sink.properties.columns are already configured. #183
• Fixed the issue of adding the first and end new line delimiters for CSV format. #186
• Aborting transactions if the flush operation fails, helping save resources. #188
• Removed the HTTP prefix when probing transaction load, allowing the transaction load probing to pass. #189
• Appended the CSV row delimiter to the end in the transaction load. #190
• Fixed the name mismatch when deserializing JSON string responses. #192
• Resolved the dead loop issue in the getAvailableHost() method. #193
• Improved the commit process by checking the label state if the commit fails, regardless of the reason for the failure. #195

Tests

• [ITTest] Add ITTest for sink and source. #180

Contributors

Husky Zeng, Kui Yuan, PengFei Li, xlfjcg,

v1.2.5

Release 1.2.5

What's Changed

• [Enhancement] Make transaction stream load default timeout to 600s #157
• [Bugfix] Fail to delete data when using json format #158
• [Bugfix] Can't load data to StarRocks json column #161
• [Enhancement] Only use maven-gpg-plugin when releasing package #164
• [Bugfix] Fix CancellationException #147
• [Enhancement] Support https load url #152
• [Enhancement] Fix shade jackson #163
• Bump version to 1.2.5 #165

v1.2.4

release v1.2.4

v1.2.3

release v1.2.3

v1.2.2

release v1.2.2

release v1.2.2

v1.2.1

release 1.2.1