You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are running several Kafka clusters managed by operator (v 0.20.0). We don't have any custom TLS certs. A few days ago we started seeing multiple active controllers in one of our clusters, and the problem seems to boil down to the fact that two brokers (it's a 3 broker cluster) are not able to talk to each other. The error is
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
We are running several Kafka clusters managed by operator (v 0.20.0). We don't have any custom TLS certs. A few days ago we started seeing multiple active controllers in one of our clusters, and the problem seems to boil down to the fact that two brokers (it's a 3 broker cluster) are not able to talk to each other. The error is
{"exception":{"stacktrace":"org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed\nCaused by: javax.net.ssl.SSLProtocolException: Unexpected handshake message: server_hello\n\tat java.base\/sun.security.ssl.Alert.createSSLException(Alert.java:129)\n\tat java.base\/sun.security.ssl.Alert.createSSLException(Alert.java:117)\n\tat java.base\/sun.security.ssl.TransportContext.fatal(TransportContext.java:313)\n\tat java.base\/sun.security.ssl.TransportContext.fatal(TransportContext.java:269)\n\tat java.base\/sun.security.ssl.TransportContext.fatal(TransportContext.java:260)\n\tat java.base\/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:438)\n\tat java.base\/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1074)\n\tat java.base\/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1061)\n\tat java.base\/java.security.AccessController.doPrivileged(Native Method)\n\tat java.base\/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1008)\n\tat org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:425)\n\tat org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:509)\n\tat org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:363)\n\tat org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:286)\n\tat org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:174)\n\tat org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)\n\tat org.apache.kafka.common.network.Selector.poll(Selector.java:485)\n\tat org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:549)\n\tat org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:74)\n\tat kafka.controller.RequestSendThread.brokerReady(ControllerChannelManager.scala:288)\n\tat kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:242)\n\tat kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)","exception_class":"org.apache.kafka.common.errors.SslAuthenticationException","exception_message":"SSL handshake failed"},"source_host":"x-platform-kafka-0","method":"warn","level":"WARN","message":"[RequestSendThread controllerId=0] Controller 0's connection to broker x-platform-kafka-1.x-platform-kafka-brokers.kafka.svc:9091 (id: 1 rack: null) was unsuccessful","mdc":{},"@timestamp":"2021-12-23T18:39:14.654Z","file":"Logging.scala","line_number":"72","thread_name":"Controller-0-to-broker-1-send-thread","@version":1,"logger_name":"kafka.controller.RequestSendThread","class":"kafka.utils.Logging"}
I am assuming the operator should be updating the certificates automatically. What could be going wrong here?
Beta Was this translation helpful? Give feedback.
All reactions