Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ISSUE] Cannot start a self managed cluster with latest release due to java.nio.file.AccessDeniedException: /usr/local/zeebe/data/.topology.meta #2436

Open
fenix-hub opened this issue Oct 7, 2024 · 1 comment
Open

[ISSUE] Cannot start a self managed cluster with latest release due to java.nio.file.AccessDeniedException: /usr/local/zeebe/data/.topology.meta #2436

fenix-hub opened this issue Oct 7, 2024 · 1 comment
Labels
kind/issue Unidentified issue, it could be a bug, misconfig, or anything in between

Comments

@fenix-hub
Copy link

Describe the issue:
I'm trying to configure a Self Managed cluster using kubernetes, following the instructions at https://docs.camunda.io/docs/self-managed/setup/install.

When running

helm install camunda camunda/camunda-platform -f camunda-platform-core-kind-values.yaml

where camunda-platform-core-kind-values.yaml is defined as:

# camunda-platform-core-kind-values.yaml

global:
  identity:
    auth:
      # Disable the Identity authentication for local development
      # it will fall back to basic-auth: demo/demo as default user
      enabled: false

# Disable identity as part of the Camunda core
identity:
  enabled: false

# Disable keycloak
identityKeycloak:
  enabled: false

optimize:
  enabled: false

# Reduce for Zeebe and Gateway the configured replicas and with that the required resources
# to get it running locally
zeebe:
  clusterSize: 1
  partitionCount: 1
  replicationFactor: 1
  pvcSize: 1Gi

zeebeGateway:
  replicas: 1

connectors:
  enabled: true
  inbound:
    mode: disabled

elasticsearch:
  master:
    replicaCount: 1
    # Request smaller persistent volumes.
    persistence:
      size: 2Gi

the cluster is not able to start due to camunda-platform-zeebe-master failures:

$ kubectl logs camunda-platform-zeebe-0

2024-10-07 10:19:00.396 [] [main] [] ERROR
      org.springframework.boot.SpringApplication - Application run failed
io.camunda.zeebe.util.exception.UncheckedExecutionException: Failed to start broker
        at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:112) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) ~[zeebe-util-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.Broker.start(Broker.java:86) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:100) ~[camunda-zeebe-8.5.4.jar:8.5.4]
        at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774) [spring-boot-3.2.5.jar:3.2.5]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) ~[?:?]
        at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source) ~[?:?]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774) [spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:341) [spring-boot-3.2.5.jar:3.2.5]
        at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:82) [camunda-zeebe-8.5.4.jar:8.5.4]
Caused by: java.util.concurrent.ExecutionException: Startup failed in the following steps: [Cluster Topology Manager]. See suppressed exceptions for details.
        at io.camunda.zeebe.scheduler.future.CompletableActorFuture.get(CompletableActorFuture.java:147) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.future.CompletableActorFuture.get(CompletableActorFuture.java:114) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.FutureUtil.join(FutureUtil.java:21) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.future.CompletableActorFuture.join(CompletableActorFuture.java:203) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:103) ~[zeebe-broker-8.5.4.jar:8.5.4]
        ... 21 more
Caused by: io.camunda.zeebe.scheduler.startup.StartupProcessException: Startup failed in the following steps: [Cluster Topology Manager]. See suppressed exceptions for details.
        at io.camunda.zeebe.scheduler.startup.StartupProcess.aggregateExceptionsSynchronized(StartupProcess.java:283) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.startup.StartupProcess.completeStartupFutureExceptionallySynchronized(StartupProcess.java:184) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.startup.StartupProcess.lambda$proceedWithStartupSynchronized$3(StartupProcess.java:168) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.future.FutureContinuationRunnable.run(FutureContinuationRunnable.java:33) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorJob.invoke(ActorJob.java:96) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorJob.execute(ActorJob.java:47) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorTask.execute(ActorTask.java:122) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorThread.executeCurrentTask(ActorThread.java:130) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorThread.doWork(ActorThread.java:108) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorThread.run(ActorThread.java:227) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        Suppressed: io.camunda.zeebe.scheduler.startup.StartupProcessStepException: Bootstrap step Cluster Topology Manager failed
                at io.camunda.zeebe.scheduler.startup.StartupProcess.completeStartupFutureExceptionallySynchronized(StartupProcess.java:186) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.startup.StartupProcess.lambda$proceedWithStartupSynchronized$3(StartupProcess.java:168) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.future.FutureContinuationRunnable.run(FutureContinuationRunnable.java:33) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorJob.invoke(ActorJob.java:96) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorJob.execute(ActorJob.java:47) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorTask.execute(ActorTask.java:122) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorThread.executeCurrentTask(ActorThread.java:130) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorThread.doWork(ActorThread.java:108) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorThread.run(ActorThread.java:227) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        Caused by: java.nio.file.AccessDeniedException: /usr/local/zeebe/data/.topology.meta
                at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[?:?]
                at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
                at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
                at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(Unknown Source) ~[?:?]
                at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source) ~[?:?]
                at java.base/java.nio.file.Files.newOutputStream(Unknown Source) ~[?:?]
                at java.base/java.nio.file.Files.write(Unknown Source) ~[?:?]
                at io.camunda.zeebe.topology.PersistedClusterTopology.writeToFile(PersistedClusterTopology.java:118) ~[zeebe-cluster-topology-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.topology.PersistedClusterTopology.update(PersistedClusterTopology.java:74) ~[zeebe-cluster-topology-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.topology.ClusterTopologyManagerImpl.lambda$initialize$4(ClusterTopologyManagerImpl.java:153) ~[zeebe-cluster-topology-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.future.FutureContinuationRunnable.run(FutureContinuationRunnable.java:28) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                ... 6 more
2024-10-07 10:19:00.401 [] [main] [] WARN
      org.springframework.boot.web.reactive.context.AnnotationConfigReactiveWebServerApplicationContext - Exception thrown from ApplicationListener handling ContextClosedEvent
java.util.concurrent.CompletionException: io.camunda.zeebe.util.exception.UncheckedExecutionException: Failed to start broker
        at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.CompletableFuture.thenAccept(Unknown Source) ~[?:?]
        at io.camunda.zeebe.broker.Broker.lambda$close$1(Broker.java:170) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) ~[zeebe-util-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.Broker.close(Broker.java:165) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.StandaloneBroker.onApplicationEvent(StandaloneBroker.java:106) ~[camunda-zeebe-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.StandaloneBroker.onApplicationEvent(StandaloneBroker.java:34) ~[camunda-zeebe-8.5.4.jar:8.5.4]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:185) ~[spring-context-6.1.10.jar:6.1.10]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.10.jar:6.1.10]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:156) ~[spring-context-6.1.10.jar:6.1.10]
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451) ~[spring-context-6.1.10.jar:6.1.10]
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384) ~[spring-context-6.1.10.jar:6.1.10]
        at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1130) [spring-context-6.1.10.jar:6.1.10]
        at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.doClose(ReactiveWebServerApplicationContext.java:149) [spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:1093) [spring-context-6.1.10.jar:6.1.10]
        at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:816) [spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:344) [spring-boot-3.2.5.jar:3.2.5]
        at io.camunda.zeebe.broker.StandaloneBroker.main(StandaloneBroker.java:82) [camunda-zeebe-8.5.4.jar:8.5.4]
Caused by: io.camunda.zeebe.util.exception.UncheckedExecutionException: Failed to start broker
        at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:112) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) ~[zeebe-util-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.Broker.start(Broker.java:86) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:100) ~[camunda-zeebe-8.5.4.jar:8.5.4]
        at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774) ~[spring-boot-3.2.5.jar:3.2.5]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) ~[?:?]
        at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source) ~[?:?]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:341) ~[spring-boot-3.2.5.jar:3.2.5]
        ... 1 more
Caused by: java.util.concurrent.ExecutionException: Startup failed in the following steps: [Cluster Topology Manager]. See suppressed exceptions for details.
        at io.camunda.zeebe.scheduler.future.CompletableActorFuture.get(CompletableActorFuture.java:147) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.future.CompletableActorFuture.get(CompletableActorFuture.java:114) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.FutureUtil.join(FutureUtil.java:21) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.future.CompletableActorFuture.join(CompletableActorFuture.java:203) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.Broker.internalStart(Broker.java:103) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.util.LogUtil.doWithMDC(LogUtil.java:23) ~[zeebe-util-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.Broker.start(Broker.java:86) ~[zeebe-broker-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.broker.StandaloneBroker.run(StandaloneBroker.java:100) ~[camunda-zeebe-8.5.4.jar:8.5.4]
        at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:790) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:83) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:88) ~[spring-core-6.1.10.jar:6.1.10]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:774) ~[spring-boot-3.2.5.jar:3.2.5]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) ~[?:?]
        at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source) ~[?:?]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:774) ~[spring-boot-3.2.5.jar:3.2.5]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:341) ~[spring-boot-3.2.5.jar:3.2.5]
        ... 1 more
Caused by: io.camunda.zeebe.scheduler.startup.StartupProcessException: Startup failed in the following steps: [Cluster Topology Manager]. See suppressed exceptions for details.
        at io.camunda.zeebe.scheduler.startup.StartupProcess.aggregateExceptionsSynchronized(StartupProcess.java:283) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.startup.StartupProcess.completeStartupFutureExceptionallySynchronized(StartupProcess.java:184) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.startup.StartupProcess.lambda$proceedWithStartupSynchronized$3(StartupProcess.java:168) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.future.FutureContinuationRunnable.run(FutureContinuationRunnable.java:33) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorJob.invoke(ActorJob.java:96) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorJob.execute(ActorJob.java:47) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorTask.execute(ActorTask.java:122) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorThread.executeCurrentTask(ActorThread.java:130) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorThread.doWork(ActorThread.java:108) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        at io.camunda.zeebe.scheduler.ActorThread.run(ActorThread.java:227) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        Suppressed: io.camunda.zeebe.scheduler.startup.StartupProcessStepException: Bootstrap step Cluster Topology Manager failed
                at io.camunda.zeebe.scheduler.startup.StartupProcess.completeStartupFutureExceptionallySynchronized(StartupProcess.java:186) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.startup.StartupProcess.lambda$proceedWithStartupSynchronized$3(StartupProcess.java:168) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.future.FutureContinuationRunnable.run(FutureContinuationRunnable.java:33) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorJob.invoke(ActorJob.java:96) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorJob.execute(ActorJob.java:47) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorTask.execute(ActorTask.java:122) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorThread.executeCurrentTask(ActorThread.java:130) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorThread.doWork(ActorThread.java:108) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.ActorThread.run(ActorThread.java:227) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
        Caused by: java.nio.file.AccessDeniedException: /usr/local/zeebe/data/.topology.meta
                at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source) ~[?:?]
                at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
                at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source) ~[?:?]
                at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(Unknown Source) ~[?:?]
                at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source) ~[?:?]
                at java.base/java.nio.file.Files.newOutputStream(Unknown Source) ~[?:?]
                at java.base/java.nio.file.Files.write(Unknown Source) ~[?:?]
                at io.camunda.zeebe.topology.PersistedClusterTopology.writeToFile(PersistedClusterTopology.java:118) ~[zeebe-cluster-topology-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.topology.PersistedClusterTopology.update(PersistedClusterTopology.java:74) ~[zeebe-cluster-topology-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.topology.ClusterTopologyManagerImpl.lambda$initialize$4(ClusterTopologyManagerImpl.java:153) ~[zeebe-cluster-topology-8.5.4.jar:8.5.4]
                at io.camunda.zeebe.scheduler.future.FutureContinuationRunnable.run(FutureContinuationRunnable.java:28) ~[zeebe-scheduler-8.5.4.jar:8.5.4]
                ... 6 more
2024-10-07 10:19:00.404 [] [main] [] INFO
      org.springframework.boot.web.embedded.netty.GracefulShutdown - Commencing graceful shutdown. Waiting for active requests to complete
2024-10-07 10:19:00.409 [] [netty-shutdown] [] INFO
      org.springframework.boot.web.embedded.netty.GracefulShutdown - Graceful shutdown complete
2024-10-07 10:19:00.843 [] [atomix-cluster-heartbeat-sender] [] INFO
      io.atomix.cluster.protocol.swim - 0 - Member added Member{id=camunda-platform-zeebe-gateway-d974d7dcf-9mr44, address=10.244.2.17:26502, properties={event-service-topics-subscribed=KIIDAGpvYnNBdmFpbGFibOU=}}
2024-10-07 10:19:01.480 [] [main] [] INFO
      io.atomix.cluster.messaging.impl.DefaultClusterCommunicationService - Stopped
2024-10-07 10:19:01.481 [] [atomix-cluster-0] [] INFO
      io.atomix.cluster.messaging.impl.DefaultClusterEventService - Stopped
2024-10-07 10:19:01.482 [] [atomix-cluster-0] [] INFO
      io.atomix.cluster.protocol.swim - 0 - Member deactivated: Member{id=0, address=camunda-platform-zeebe-0.camunda-platform-zeebe.default.svc:26502, properties={}}
2024-10-07 10:19:01.483 [] [atomix-cluster-0] [] INFO
      io.atomix.cluster.protocol.swim - Stopped
2024-10-07 10:19:01.484 [] [atomix-cluster-0] [] INFO
      io.atomix.cluster.discovery.BootstrapDiscoveryProvider - Local node Member{id=0, address=camunda-platform-zeebe-0.camunda-platform-zeebe.default.svc:26502, properties={}} left the bootstrap servide
2024-10-07 10:19:01.485 [] [atomix-cluster-0] [] INFO
      io.atomix.cluster.impl.DefaultClusterMembershipService - Stopped cluster membership service for member Member{id=0, address=camunda-platform-zeebe-0.camunda-platform-zeebe.default.svc:26502, properties={}}
2024-10-07 10:19:01.597 [] [Thread-19] [] INFO
      io.atomix.cluster.messaging.impl.NettyUnicastService - Stopped plaintext unicast service bound to 0.0.0.0:26502, advertising camunda-platform-zeebe-0.camunda-platform-zeebe.default.svc:26502
2024-10-07 10:19:01.707 [] [Thread-20] [] INFO
      io.atomix.cluster.messaging.impl.NettyMessagingService - Stopped messaging service bound to [0.0.0.0:26502], advertising camunda-platform-zeebe-0.camunda-platform-zeebe.default.svc:26502, and using plaintext
2024-10-07 10:19:01.709 [] [atomix-cluster-0] [] INFO
      io.atomix.cluster.AtomixCluster - Stopped

I also tried using latest camunda 8 helm charts following the guide:

# This will install the latest Camunda Helm chart with the latest applications/dependencies.
helm install camunda camunda/camunda-platform \
    --values https://helm.camunda.io/camunda-platform/values/values-latest.yaml

but the cluster failes due to elasticsearch write/lock issues.

Actual behavior:

Expected behavior:

How to reproduce:

  1. start a cluster named camunda8 with 3 nodes: minikube start -p camunda8 --nodes 3
  2. activate camunda8 as default namespace minikube profile camunda8
  3. alias kubectl="minikube kubectl --"
  4. kubectl proxy --address="192.168.60.65" --disable-filter=true &
  5. minikube dashboard --url &
  6. helm install camunda-platform camunda/camunda-platform -f camunda-platform-core-kind-values.yaml

Logs:

kubernetes cluster dashboard:
image

minikube status:

$ minikube status

camunda8
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured

camunda8-m02
type: Worker
host: Running
kubelet: Running

camunda8-m03
type: Worker
host: Running
kubelet: Running

nodes:
image

kubectl cluster-info dump:
dump.txt

Environment:

-----------------------------------------------
OS: Ubuntu 22.04.3 LTS x86_64
Host: acloud 1
Kernel: 5.15.0-91-generic
Uptime: 266 days, 45 mins
Packages: 808 (dpkg), 7 (snap)
Shell: bash 5.1.16
Resolution: 1024x768
Terminal: /dev/pts/1
CPU: Intel Core 2 Duo T7700 (4) @ 2.893GHz
GPU: 00:02.0 Vendor 1234 Device 1111
Memory: 7164MiB / 15989MiB
  • minikube version:
$ minikube version
minikube version: v1.33.1
commit: 5883c09216182566a63dff4c326a6fc9ed2982ff
  • kubectl version:
$ kubectl version
Client Version: v1.30.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.0
  • helm version:
$ helm version
version.BuildInfo{Version:"v3.16.1", GitCommit:"5a5449dc42be07001fd5771d56429132984ab3ab", GitTreeState:"clean", GoVersion:"go1.22.7"}
@fenix-hub fenix-hub added the kind/issue Unidentified issue, it could be a bug, misconfig, or anything in between label Oct 7, 2024
@aabouzaid
Copy link
Member

@fenix-hub Thanks for reporting this issue.
Did you check the guide for local K8s setup?
https://docs.camunda.io/docs/self-managed/setup/deploy/local/local-kubernetes-cluster/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/issue Unidentified issue, it could be a bug, misconfig, or anything in between
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aabouzaid @fenix-hub