need help with gcp-pubsub event source: no active event server running

[comtihon]

Hi, I am setting up example gcp event source and sensor from argo-events examples.
Argo-events and argo-workflows were installed with helm, example workflow I took from the argo-events github is also packaged with helm and installed via argocd.
However, event source can't run with the error:

2023-11-24T12:06:11.142Z FATAL argo-events.eventsource cmd/start.go:67 failed to start eventsource server {"eventSourceName": "my-event-source", "error": "no active event server running"}
github.com/argoproj/argo-events/eventsources/cmd.Start
/home/runner/work/argo-events/argo-events/eventsources/cmd/start.go:67
github.com/argoproj/argo-events/cmd/commands.NewEventSourceCommand.func1
/home/runner/work/argo-events/argo-events/cmd/commands/eventsource.go:14
github.com/spf13/cobra.(*Command).execute
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:944
github.com/spf13/cobra.(*Command).ExecuteC
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068
github.com/spf13/cobra.(*Command).Execute
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
github.com/argoproj/argo-events/cmd/commands.Execute
/home/runner/work/argo-events/argo-events/cmd/commands/root.go:19
main.main
/home/runner/work/argo-events/argo-events/cmd/main.go:8
runtime.main
/opt/hostedtoolcache/go/1.20.7/x64/src/runtime/proc.go:250

It's not quite informative, but as far as I understand - no active event server running means that it failed to start (not because I need to run some event server separately).
So the reason can be in event source -> pubsub connection.
I tried 2 ways:

• create a k8s secret, mount it to pod and reference it in credentialSecret as said here https://github.com/argoproj/argo-events/blob/stable/examples/event-sources/gcp-pubsub.yaml#L36
• create a k8s secret with a special name, as said here
  Both didn't work.
  My event-source:

apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
  name: my-pubsub-event-source
  namespace: argo-events
spec:
  template:
    container:
      env:
        - name: DEBUG_LOG
          value: "true"
  pubsub:
    example:
      jsonBody: true
      projectID: {{ .Values.config.project_id }}
      topic: {{ .Values.config.topic }}
      subscriptionID: {{ .Values.config.subscription_id }}

secret is created via terraform, here is it's yaml:

apiVersion: v1
data:
  key.json: <google secret base64 encoded> 
kind: Secret
metadata:
  creationTimestamp: <TS>
  name: gcp-credentials
  namespace: argo-events
  resourceVersion: <version>
  uid: <uuid>
type: Opaque

In my environment I see these pods:

1. argo-events-controller-manager [argo-events]
2. argo-workflows-server [argo]
3. argo-workflows-workflow-controller [argo]
4. eventbus-default-stan [argo-events]
5. events-webhook [argo-events]
6. my-pubsub-event-source-eventsource-4mnx6 [argo-events] <-- crashloopbackoff
7. myi-pubsub-sensor-sensor-bpdtl [argo-events]

I have two questions:

1. is there a way to get more logs? DEBUG_LOG didn't really improved the situation
2. how to fix it? It is a pubsub hello world, not a rocket science, but I am fighting with it for quite a while