Webhook: create k8s resources selected by a headless service (DNS resolution expected)

[ProtossGP32]

Hello everyone! I'm pretty new to the Argo Events and Workflow ecosystem but I think they could help me implement an experimental architecture I'm working on.

Brief description: I'd like to create an Argo Events architecture that spawns workloads, each one of them then selected by a headless service so I can ensure communication between these pods. Something along the lines of this diagram:

graph TB
   CLI -- curl ... http://localhost:12000/example --> EventSource
   subgraph hs [Headless Service]
     direction LR
     EventSource --> Sensor
     Sensor -- /example --> PodA("Pod A")
     Sensor -- /example --> PodB("Pod B")
     PodA <-- can reach --> PodB
   end

Loading

Also, Pod A and Pod B should be able to reach the Event Source to send it payloads.

Tested until now: I've followed the Argo Events Getting Started guide and already deployed a Webhook EventSource and a Source, and configured the triggers and endpoints so each time I do an HTTP request to an endpoint http://localhost:12000/example an argo-workflow template is created (k8s resource) and a pod is spawned. What I would like now is to make all these pods internally reachable via DNS using a headless service (no need for external communication, only inter-pod).

I've tried assiging labels to the pods via podMetadata in the Argo Workflow template but the service with the appropriate selector doesn't seem to care about this. The only thing that I've achieved is to include the EventSource to the external headless service using the eventsource-name label.

Any thoughts on this?

[ProtossGP32]

So, I've been doing some additional tests:

• I've exposed the EventSource using its own Service --> This is reachable through DNS resolution by any pod within the namespace
• I've created a headless service that uses a label app=my-app as the selector (any further Pod shall be created with this label)
• In the Sensor, I've replaced Workflows resource creations with Pods
  • This allows me to configure both spec.hostname and spec.subdomain fields, mandatory to let the headless service create the appropriate A/AAA records to reach the pods from within the same namespace

This is the current representation of what I've accomplished so far (sorry for the diagram mess, can't do better with mermaid):

graph BT
   subgraph es-svc [Event Source service]
     direction TB
     EventSource(Event Source)
   end

   subgraph hs [Headless Service]
     direction TB
     PodA("Pod A")
     PodB("Pod B")
   end
   CLI -- curl ... http://localhost:12000/example --> EventSource
   EventSource --> Sensor
   PodA <-- can reach --> PodB
   Sensor -- creates --> PodA
   Sensor -- creates --> PodB
   PodA -- /example --> es-svc
   PodB -- /example --> es-svc

Loading

Pros:

• Any pod can send new events to the Event Source and it can spawn new pods based on the received payloads

Cons:

• As it isn't using Workflows, there's no surveilance on the pod's execution nor deletion when completed
• Also, I'm not sure how to pass input parameters to the pods based on the sent payload. With workflows that would be done by defining input and output parameters, but with Pods I don't know if I should define environment variables or mount some kind of custom configMap each time.
  • Is it possible to configure Pods' spec fields (not podMetadata fields) in the Workflows? I've tried multiple times to include them either via Parameterization or directly in the template but they don't translate in the final Pod manifest...