Skip to content

Commit

Permalink
W-15672495 added docs for new kinesis changes
Browse files Browse the repository at this point in the history
  • Loading branch information
sf-vishwastyagi committed May 8, 2024
1 parent 60ce81e commit e230061
Show file tree
Hide file tree
Showing 12 changed files with 983 additions and 0 deletions.
17 changes: 17 additions & 0 deletions amazon-kinesis/2.0/antora.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: amazon-kinesis-connector
version: '2.0'
display_version: 2.0 (Mule 4)
title: Amazon Kinesis Connector
nav:
- modules/ROOT/nav.adoc
asciidoc:
attributes:
page-component-desc: Provides access to standard Amazon Kinesis Data Stream operations using Anypoint Platform.
page-connector-type: Connector
page-connector-level: Select
page-exchange-group-id: com.mulesoft.connectors
page-exchange-asset-id: amazon-kinesis-data-streams-connector
page-runtime-version: 4.3.0
page-release-notes-page: release-notes::connector/amazon-kinesis-connector-release-notes-mule-4.adoc
page-vendor-name: amazon
page-vendor-title: Amazon
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 6 additions & 0 deletions amazon-kinesis/2.0/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.xref:index.adoc[Amazon Kinesis Connector]
* xref:index.adoc[Amazon Kinesis Connector Overview]
* xref:amazon-kinesis-connector-studio.adoc[Using Anypoint Studio to Configure Amazon Kinesis Connector]
* xref:amazon-kinesis-connector-xml-maven.adoc[Amazon Kinesis Connector XML and Maven Support]
* xref:amazon-kinesis-connector-examples.adoc[Amazon Kinesis Connector Examples]
* xref:amazon-kinesis-connector-reference.adoc[Amazon Kinesis Connector Reference]
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
= Amazon Kinesis Data Streams Connector 1.0 Examples - Mule 4

The following example shows how to use Anypoint Connector for Amazon Kinesis Data Streams (Amazon Kinesis Data Streams Connector) to put data records and listen from an Amazon Kinesis data stream.

Before you try the example, access Anypoint Studio (Studio) and verify that the *Mule Palette* view displays entries for Kinesis. If not, follow the instructions in xref:amazon-kinesis-connector-studio.adoc#add-connector-to-project[Add the Connector to Your Project].

== Flows for This Example

The following screenshots show the Anypoint Studio app flows for this example:

* This flow uses the *Put Records* operation to put data into the Amazon Kinesis data stream. In this flow, *HTTP > Listener* receives a query parameter named *data* that sets the payload for the *Put Record* operation. It uses a query parameter named *partitionKey* to set the partition key value for the *Put Record* operation.
+
image::amazon-kinesis-example-put-record.png[Put Record operation flow]
+
* This flow uses the *Listener* source to listen for new data records:
+
image::amazon-kinesis-example-listener.png[Listener source flow]
+
* This flow uses the *Listener* source to listen for new data records. After receiving the new records, it calls the *Checkpoint* operation to set a manual checkpoint:
+
image::amazon-kinesis-example-checkpoint.png[Checkpoint operation flow]

== XML Code for This Example

[source,xml,linenums]
----
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:os="http://www.mulesoft.org/schema/mule/os" xmlns:ee="http://www.mulesoft.org/schema/mule/ee/core"
xmlns:http="http://www.mulesoft.org/schema/mule/http"
xmlns:kinesis="http://www.mulesoft.org/schema/mule/kinesis" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/kinesis http://www.mulesoft.org/schema/mule/kinesis/current/mule-kinesis.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd
http://www.mulesoft.org/schema/mule/os http://www.mulesoft.org/schema/mule/os/current/mule-os.xsd">
<configuration-properties doc:name="Configuration properties" file="mule-artifact.properties" />
<kinesis:config name="Kinesis_config" doc:name="Kinesis config">
<kinesis:connection accessKey="${config.accessKey}" secretKey="${config.secretKey}" dynamoDBEndPoint="${config.dynamoDBEndpoint}" cloudWatchEndPoint="${config.cloudWatchEndpoint}"/>
</kinesis:config>
<kinesis:config name="Amazon_Kinesis_Data_Streams_Connector_Config_FIPS" doc:name="Amazon Kinesis Data Streams Connector Config FIPS" >
<kinesis:connection accessKey="${config.accessKey}" secretKey="${config.secretKey}" dynamoDBEndPoint="${config.dynamoDBEndpoint}" cloudWatchEndPoint="${config.cloudWatchEndpoint}" kinesisServiceEndPoint="${config.kinesisUrl}" customSTSEndPoint="${config.stsEndpoint}" />
</kinesis:config>
<http:listener-config name="HTTP_Listener_config" doc:name="HTTP Listener config" >
<http:listener-connection host="0.0.0.0" port="8081" />
</http:listener-config>
<flow name="kinesis-demo-put-record" >
<http:listener doc:name="Listener" config-ref="HTTP_Listener_config" path="/putrecord"/>
<kinesis:put-record doc:name="Put Record" config-ref="Kinesis_config" streamName="${streamName}"
partitionKey="#[attributes.queryParams.partitionKey]">
<kinesis:data ><![CDATA[#[attributes.queryParams.data]]]></kinesis:data>
</kinesis:put-record>
<ee:transform doc:name="Transform Message" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" message="Sent message: #[payload]"/>
</flow>
<!-- Stop the following flow and enable/start the other one in order to run it -->
<flow name="kinesis-demo-listener-latest">
<kinesis:listener doc:name="Listener" config-ref="Kinesis_config"
streamName="${streamName}" applicationName="${applicationName}"
absolutePosition="LATEST"/>
<ee:transform doc:name="Transform Message" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" message="Received message: #[payload]" />
</flow>
<flow name="kinesis-demo-checkpoint" initialState="stopped">
<kinesis:listener doc:name="Listener" config-ref="Kinesis_config"
streamName="${streamName}" applicationName="${applicationName}"
absolutePosition="LATEST" checkpointOnComplete="false"/>
<ee:transform doc:name="Transform Message" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" message="Received message: #[payload]" />
<kinesis:checkpoint doc:name="Checkpoint" config-ref="Kinesis_config" applicationName="${applicationName}" streamName="${streamName}"/>
</flow>
<flow name="kinesisdemoFlow">
<http:listener doc:name="Listener" config-ref="HTTP_Listener_config" path="/kinesis"/>
<kinesis:put-record doc:name="Put Record" config-ref="Amazon_Kinesis_Data_Streams_Connector_Config_FIPS" streamName="${streamName}"/>
<ee:transform doc:name="Transform Message">
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/java
---
payload]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" message="Sent message: #[payload]"/>
</flow>
</mule>
----

=== See Also

* xref:connectors::introduction/introduction-to-anypoint-connectors.adoc[Introduction to Anypoint Connectors]
* https://help.mulesoft.com[MuleSoft Help Center]
Loading

0 comments on commit e230061

Please sign in to comment.