Skip to content

Commit

Permalink
Update javadocs for PscSink and PscSource to highlight difference com…
Browse files Browse the repository at this point in the history
…pared to FlinkPscProducer and Consumer
  • Loading branch information
jeffxiang committed Nov 7, 2024
1 parent a6efb43 commit 20535c0
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,14 @@
/**
* Flink Sink to produce data into a PSC topicUri. The sink supports all delivery guarantees
* described by {@link DeliveryGuarantee}.
*
* A PscSink using EXACTLY_ONCE delivery guarantee requires a {@link com.pinterest.flink.connector.psc.PscFlinkConfiguration#CLUSTER_URI_CONFIG}
* to be set in the producer config. This is used to pre-construct the {@link com.pinterest.psc.producer.PscBackendProducer} upon
* creation of a top-level {@link com.pinterest.psc.producer.PscProducer} to perform transactional operations. For this reason,
* a single PscSink instance cannot be used to write to topics spanning multiple different clusters. This limitation does not
* exist in the AT_LEAST_ONCE and NONE delivery guarantees, nor does it exist in {@link com.pinterest.flink.streaming.connectors.psc.FlinkPscProducer}
* which can be used to write to multiple clusters regardless of the delivery guarantee.
*
* <li>{@link DeliveryGuarantee#NONE} does not provide any guarantees: messages may be lost in case
* of issues on the PubSub broker and messages may be duplicated in case of a Flink failure.
* <li>{@link DeliveryGuarantee#AT_LEAST_ONCE} the sink will wait for all outstanding records in the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,17 @@

/**
* The Source implementation of PSC. Please use a {@link PscSourceBuilder} to construct a {@link
* PscSource}. The following example shows how to create a PscSource emitting records of <code>
* PscSource}
*
* This is different from {@link com.pinterest.flink.streaming.connectors.psc.FlinkPscConsumer}
* in that it is a source implementation for Flink's new source API. Due to the difference in implementation,
* the two classes are not compatible with each other. Most notably, PscSource currently can only support
* reading from topics from a single backend cluster, while FlinkPscConsumer can read from multiple clusters.
* This limitation is due to the fact that a {@link com.pinterest.flink.connector.psc.PscFlinkConfiguration#CLUSTER_URI_CONFIG}
* is required to be set in the configuration for the PscSource to perform metadata queries against the cluster using
* a {@link com.pinterest.psc.metadata.client.PscMetadataClient}.
*
* The following example shows how to create a PscSource emitting records of <code>
* String</code> type.
*
* <pre>{@code
Expand Down

0 comments on commit 20535c0

Please sign in to comment.