Publish packages to Github packages. Maven central or similar?

[josteinbf]

Is your feature request related to a problem? Please describe.
We have successfully set up the connector in AWS using MSK Connect, and this is working well. However, we are unable to use the zip files released by this repo directly because we need to add some AWS-specific classes for interacting with an AWS schema registry and some standard single message transforms to the zip or jar file we use with MSK Connect. This leads to our deployment pipeline containing of manual steps to get the right artefacts created and deployed, which is of course undesirable.

In more general terms, this problem is one of bundling more (compiled) code together with the Clickhouse Connect code provided by this repo.

Describe the solution you'd like
If this repo would provide standard maven packages, either on maven central, github packages or some other public package repository, that would meet our needs. It would allow us to build our own jar packages for use with MSK Connect (or any other Kafka Connect setup for that matter) using a super simple build script. The essence of such a build.gradle.kts would be:

dependencies {
    implementation("software.amazon.glue:schema-registry-kafkaconnect-converter:1.1.22")
    implementation("org.apache.kafka:connect-transforms:2.7.0")

    // This last line is currently not possible since no packages are published
    implementation("com.clickhouse:clickhouse-kafka-connect:v1.2.6")
}

I'll share a fully functional example of such a script in the comments below.

Describe alternatives you've considered
We have currently made this work by forking this repo and building the maven packages there. I guess combining zip files from the various projects we need to combine code from might also work, but that sounds messy and potentially fragile.

Additional context
As mentioned above, we are currently working around this issue by having our own fork of this repo where we have added package generation (for github packages). The essence of it is something along the lines of what is done in this commit (auth etc is not done correctly in that commit alone, but it gives a good idea of what's required): josteinbf@083818a

Since we already have most of the code required to publish packages to either github packages or maven central done in this fork, we'd of course be happy to attempt a PR to get that into this repo. However, we would require some assistance setting up credentials for publishing packages etc -- that's something only the owners of the repo (and potentially the maven central account) can do.

[josteinbf]

Here's a full example build.gradle.kts that we currently use to build a jar for our MSK Connect instance. This pulls packages from a fork of this repo, as well as other some other packages:

plugins {
    id("java")
    id("co.uzzu.dotenv.gradle") version "4.0.0" apply true
}

group = "com.airthings.clickhouse"
version = "1.0-SNAPSHOT"

fun RepositoryHandler.josteinbf(repository: String) = maven {
    name = "com.airthings.github.packages"
    url = uri("https://maven.pkg.github.com/josteinbf/${repository}")
    credentials {
        //env.fetchOrNull("GITHUB_USERNAME")
        username = System.getenv("GITHUB_USERNAME")
            ?: System.getenv("GITHUB_ACTOR")
        password = System.getenv("GITHUB_TOKEN")
    }
}

repositories {
    mavenCentral()
    maven(url = "https://packages.confluent.io/maven/")
    josteinbf(repository = "clickhouse-kafka-connect")
}

dependencies {
    // Connector classes we use from other places; these just need to be in the JAR we generate
    val kafkaVersion = "2.7.0"
    implementation("software.amazon.glue:schema-registry-kafkaconnect-converter:1.1.22")
    implementation("com.airthings.clickhouse:clickhouse-kafka-connect:v1.2.6")
    implementation("org.apache.kafka:connect-transforms:${kafkaVersion}")
}

tasks.jar {
    val dependencies = configurations
        .runtimeClasspath
        .get()
        .map(::zipTree)
    from(dependencies)
    duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}