KafkaConnect fails to load plugin

[hunghip95]

I'm deploying KafkaCluster, KafkaConnect on K8s by using Strimzi.

• Strimzi Operator Cluster image: quay.io/strimzi/operator:0.26.1
• Kafka image: quay.io/strimzi/kafka:0.26.1-kafka-3.0.0.

I'm trying to setting KafkaConnector which connects to Postgres DB.

These are my steps:

1. Build a new KafkaConnect from this image 'quay.io/strimzi/kafka:0.26.1-kafka-3.0.0' (I followed this guide). This new image has plugin which I download from this link

2. I run this new image on K8s, then exec into KafkaConnect container. It has plugin correctly ('/opt/kafka/plugins/debezium/debezium-connector-postgres'):
   

3. I create KafkaConnector which refers to class 'io.debezium.connector.postgresql.PostgresConnector':

kind: "KafkaConnector"
metadata:
  name: dev-approvalflowservice-connector
  labels:
    strimzi.io/cluster: postgres-connect-cluster-1
spec:
  class: io.debezium.connector.postgresql.PostgresConnector
  tasksMax: 2
  config:
    database.hostname: 10.14.101.204
    database.port: "5432"
    database.user: "debezium"
    database.password: "debezium"
    database.dbname: "approvalflowservice"
    database.server.name: "approvalflowservice"
    plugin.name: "pgoutput"
    slot.name: "approvalflowserviceslot"
    truncate.handling.mode: "include"
    table.include.list: "public.approval_flow"

4. The problem is KafkaConnect fail to load the Plugin which I already added in to new KafkaConnect image. Log from KafkaConnect: log

Update

5. I tried to change the location of plugin inside the new KafkaConnect image ('/opt/kafka/plugins/debezium-connector-postgres') :
   
6. The problem is the same: log

Thank you all.

[scholzj]

The second way is the right way how to do this. The first one should work as well, but it might cause problems if you add additional connectors into /opt/kafka/plugins/debezium/. For whatever reason, it seems that your connector files have read permissions only for the owner which is root. But the Connect cluster does not run under root. When I check my images, the files are always readable by everyone:

-rw-r--r--. 1 root root 7624 Dec 22 19:28 ....jar

I guess without that, Connect might not see the connector. So I guess you should try to change the permissions on these files.

[hunghip95]

This is my Dockerfile which I followed link.

FROM quay.io/strimzi/kafka:0.26.1-kafka-3.0.0
USER root:root
COPY ./debezium-connector-postgres/ /opt/kafka/plugins/
USER 1001

Now, ill try to add some more permission grants inside this Dockerfile

[scholzj]

Yeah, that looks good in general and it normally works. But I guess it might depend on the permissions you have locally in the ./debezium-connector-postgres/ directory

[hunghip95]

Actually, my local files has permissions.

[scholzj]

Well, it does not show the JARs. Either way, I don't know why it in your case ended up with the permissions it did. It does not seem to happen to me.