Skip to content

Commit

Permalink
"Created by snbot"
Browse files Browse the repository at this point in the history
  • Loading branch information
streamnativebot committed Sep 23, 2024
1 parent 7a2b056 commit 106585f
Show file tree
Hide file tree
Showing 4 changed files with 33 additions and 43 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
PULSAR_VERSION=`mvn -q -Dexec.executable=echo -Dexec.args='${pulsar.version}' --non-recursive exec:exec 2>/dev/null`
REPO=`mvn -q -Dexec.executable=echo -Dexec.args='${project.artifactId}' --non-recursive exec:exec 2>/dev/null`
IMAGE_REPO=streamnative/${REPO}
RUNNER_IMAGE=streamnative/pulsar-functions-java-runner:${PULSAR_VERSION}
RUNNER_IMAGE=snstage/pulsar-functions-java-runner:${PULSAR_VERSION}
docker pull ${RUNNER_IMAGE}
docker build --build-arg PULSAR_VERSION="$PULSAR_VERSION" -t ${IMAGE_REPO}:${CONNECTOR_VERSION} -f ./image/Dockerfile ./
docker push ${IMAGE_REPO}:${CONNECTOR_VERSION}
16 changes: 8 additions & 8 deletions docs/cloud-storage-sink.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ You can get the Cloud Storage sink connector using one of the following methods:

## Use it with Function Worker

- Download the NAR package from [here](https://github.com/streamnative/pulsar-io-cloud-storage/releases/download/v{{connector:version}}/pulsar-io-cloud-storage-{{connector:version}}.nar).
- Download the NAR package from [here](https://github.com/streamnative/pulsar-io-cloud-storage/releases/download/v4.0.0-ursa-10-SNAPSHOT/pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar).

- Build it from the source code.

Expand All @@ -33,7 +33,7 @@ You can get the Cloud Storage sink connector using one of the following methods:

```bash
ls target
pulsar-io-cloud-storage-{{connector:version}}.nar
pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar
```

## Use it with Function Mesh
Expand Down Expand Up @@ -282,7 +282,7 @@ kind: Sink
metadata:
name: cloud-storage-sink-sample
spec:
image: streamnative/pulsar-io-cloud-storage:{{connector:version}}
image: streamnative/pulsar-io-cloud-storage:4.0.0-ursa-10-SNAPSHOT
className: org.apache.pulsar.io.jcloud.sink.CloudStorageGenericRecordSink
replicas: 1
maxReplicas: 1
Expand Down Expand Up @@ -314,7 +314,7 @@ spec:
cpu: "0.1"
memory: 1G
java:
jar: connectors/pulsar-io-cloud-storage-{{connector:version}}.nar
jar: connectors/pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar
clusterName: test-pulsar
```
Expand All @@ -334,7 +334,7 @@ This example shows how to create an Cloud Storage sink connector on a Pulsar clu
```
PULSAR_HOME/bin/pulsar-admin sinks create \
--archive pulsar-io-cloud-storage-{{connector:version}}.nar \
--archive pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar \
--sink-config-file cloud-storage-sink-config.yaml \
--classname org.apache.pulsar.io.jcloud.sink.CloudStorageGenericRecordSink \
--name cloud-storage-sink
Expand Down Expand Up @@ -433,7 +433,7 @@ This example explains how to create a Cloud Storage sink connector in an on-prem
1. Copy the NAR package of the Cloud Storage connector to the Pulsar connectors directory.
```
cp pulsar-io-cloud-storage-{{connector:version}}.nar $PULSAR_HOME/connectors/pulsar-io-cloud-storage-{{connector:version}}.nar
cp pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar $PULSAR_HOME/connectors/pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar
```
2. Reload all [built-in connectors](https://pulsar.apache.org/docs/en/next/io-connectors/).
Expand Down Expand Up @@ -481,7 +481,7 @@ This example demonstrates how to create Cloud Storage sink connector through Fun
metadata:
name: cloud-storage-sink-sample
spec:
image: streamnative/pulsar-io-cloud-storage:{{connector:version}}
image: streamnative/pulsar-io-cloud-storage:4.0.0-ursa-10-SNAPSHOT
className: org.apache.pulsar.io.jcloud.sink.CloudStorageGenericRecordSink
replicas: 1
maxReplicas: 1
Expand Down Expand Up @@ -513,7 +513,7 @@ This example demonstrates how to create Cloud Storage sink connector through Fun
cpu: "0.1"
memory: 1G
java:
jar: connectors/pulsar-io-cloud-storage-{{connector:version}}.nar
jar: connectors/pulsar-io-cloud-storage-4.0.0-ursa-10-SNAPSHOT.nar
clusterName: test-pulsar
```
Expand Down
2 changes: 1 addition & 1 deletion image/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,5 @@
#

ARG PULSAR_VERSION
FROM streamnative/pulsar-functions-java-runner:${PULSAR_VERSION}
FROM snstage/pulsar-functions-java-runner:${PULSAR_VERSION}
COPY --chown=$UID:$GID target/*.nar /pulsar/connectors/
56 changes: 23 additions & 33 deletions pom.xml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?><!--
<?xml version='1.0' encoding='UTF-8'?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
Expand All @@ -18,22 +19,18 @@
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.apache</groupId>
<artifactId>apache</artifactId>
<version>18</version>
</parent>

<modelVersion>4.0.0</modelVersion>

<groupId>io.streamnative.ecosystem</groupId>
<artifactId>pulsar-io-cloud-storage</artifactId>
<version>2.9.0-rc-202110221101</version>
<version>4.0.0-ursa-10-SNAPSHOT</version>
<name>Pulsar Ecosystem :: IO Connector :: Cloud Storage Project</name>
<description>Cloud Storage Connector integrates Apache Pulsar with Cloud Storage.</description>

<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
Expand All @@ -43,12 +40,11 @@
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<spotbugs-annotations.version>4.2.2</spotbugs-annotations.version>
<testRetryCount>2</testRetryCount>

<!-- connector dependencies -->
<jackson.version>2.13.2</jackson.version>
<jackson-databind.version>2.13.4.2</jackson-databind.version>
<lombok.version>1.18.32</lombok.version>
<pulsar.version>3.2.2.1</pulsar.version>
<pulsar.version>4.0.0-ursa-10-SNAPSHOT</pulsar.version>
<avro.version>1.11.3</avro.version>
<hadoop.version>3.3.6</hadoop.version>
<parquet.version>1.13.1</parquet.version>
Expand All @@ -64,14 +60,12 @@
<grpc.version>1.42.1</grpc.version>
<protoc-gen-grpc-java.version>${grpc.version}</protoc-gen-grpc-java.version>
<azure-sdk-bom.version>1.2.22</azure-sdk-bom.version>

<!-- test dependencies -->
<junit.version>4.13.1</junit.version>
<awaitility.version>4.2.0</awaitility.version>
<mockito.version>3.7.7</mockito.version>
<powermock.version>2.0.2</powermock.version>
<testcontainers.version>1.15.2</testcontainers.version>

<!-- build plugin dependencies -->
<license.plugin.version>3.0</license.plugin.version>
<maven-checkstyle-plugin.version>3.1.1</maven-checkstyle-plugin.version>
Expand All @@ -88,15 +82,13 @@
<netty.version>4.1.86.Final</netty.version>
<libthrift.version>0.17.0</libthrift.version>
</properties>

<licenses>
<license>
<name>Apache License, Version 2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
<distribution>repo</distribution>
</license>
</licenses>

<!-- keep all the dependencies used by all modules here -->
<dependencyManagement>
<dependencies>
Expand Down Expand Up @@ -211,7 +203,6 @@
<artifactId>avro</artifactId>
<version>${avro.version}</version>
</dependency>

<!-- test dependencies -->
<dependency>
<groupId>junit</groupId>
Expand Down Expand Up @@ -268,45 +259,37 @@
<artifactId>sts</artifactId>
<version>${aws.java.sdk2.version}</version>
</dependency>

<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>${snakeyaml.version}</version>
</dependency>

<dependency>
<groupId>ch.qos.reload4j</groupId>
<artifactId>reload4j</artifactId>
<version>${reload4j.version}</version>
</dependency>

<dependency>
<groupId>com.fasterxml.woodstox</groupId>
<artifactId>woodstox-core</artifactId>
<version>${woodstox-core.version}</version>
</dependency>

<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-bom</artifactId>
<version>${netty.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>

<dependency>
<groupId>org.apache.thrift</groupId>
<artifactId>libthrift</artifactId>
<version>${libthrift.version}</version>
</dependency>

</dependencies>
</dependencyManagement>

<!-- include the dependencies -->
<dependencies>

<!-- provided dependencies (available at compilation and test classpaths and *NOT* packaged) -->
<dependency>
<groupId>org.projectlombok</groupId>
Expand All @@ -318,7 +301,6 @@
<artifactId>spotbugs-annotations</artifactId>
<scope>provided</scope>
</dependency>

<!-- runtime dependencies -->
<dependency>
<groupId>io.streamnative</groupId>
Expand All @@ -336,7 +318,6 @@
<groupId>io.streamnative</groupId>
<artifactId>pulsar-client-original</artifactId>
</dependency>

<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
Expand Down Expand Up @@ -544,7 +525,6 @@
<groupId>com.azure</groupId>
<artifactId>azure-core-http-okhttp</artifactId>
</dependency>

<!-- test dependencies -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
Expand Down Expand Up @@ -605,9 +585,7 @@
<version>${pulsar.version}</version>
<scope>test</scope>
</dependency>

</dependencies>

<build>
<pluginManagement>
<plugins>
Expand Down Expand Up @@ -764,7 +742,6 @@
<version>${os-maven-plugin.version}</version>
</extension>
</extensions>

<plugins>
<!-- compile -->
<plugin>
Expand All @@ -789,31 +766,26 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>

<!-- test -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>

<!-- package -->
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
</plugin>

<!-- license -->
<plugin>
<groupId>com.mycila</groupId>
<artifactId>license-maven-plugin</artifactId>
</plugin>

<!-- checkstyle -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
</plugin>

<!-- spotbugs -->
<plugin>
<groupId>com.github.spotbugs</groupId>
Expand All @@ -832,5 +804,23 @@
<name>bintray</name>
<url>https://dl.bintray.com/streamnative/maven</url>
</repository>
<repository>
<id>ossrh</id>
<url>https://s01.oss.sonatype.org/service/local/repositories/0/content</url>
</repository>
<repository>
<id>nexus-snapshot</id>
<url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
</repository>
</repositories>
<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
<repository>
<id>ossrh</id>
<url>https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>
</project>

0 comments on commit 106585f

Please sign in to comment.