Facing issues while sending Databricks Lineage (Spline) to kafka

[B09306]

Hi Team,
we have enabled spline on databricks cluster and we are able to connect from databricks to kafka using SSL certificate but spline is not able to connect to kafka that's why it is not able to send lineage to kafka & got below error .

SparkLineageInitializer: Spline successfully initialized. Spark Lineage tracking is ENABLED.
23/11/13 11:07:49 WARN NetworkClient: [Producer clientId=producer-1] Bootstrap broker xxxxxxxxxxxxx:9092 (id: -1 rack: null) disconnected

could anyone please look into this and help us.

[cerveada]

You have to configure the Kafka producer the same way you provide bootstrap servers you can provide all other Kafka properties. Just be sure to use the right prefix.

For example for bootstrap.servers and ssl.key.password

spline.lineageDispatcher.kafka.producer.bootstrap.servers=localhost:9092
spline.lineageDispatcher.kafka.producer.ssl.key.password=xxx
//and so on

https://docs.confluent.io/platform/7.5/installation/configuration/producer-configs.html