[lagoon-logging] cdn-logs-collector metrics missing

[xantrix]

We found with @anothertobi that the Servicemonitor tries to scrape a non-existent port on the service
https://github.com/uselagoon/lagoon-charts/blob/main/charts/lagoon-logging/templates/cdn-logs-collector.servicemonitor.yaml#L13

The metrics port is available on the pod
https://github.com/uselagoon/lagoon-charts/blob/main/charts/lagoon-logging/templates/cdn-logs-collector.statefulset.yaml#L49

Possible solution:

• Change the ServiceMonitor manifest to use TargetPort instead of port
  https://doc.crds.dev/github.com/prometheus-operator/kube-prometheus/monitoring.coreos.com/ServiceMonitor/[email protected]#spec-endpoints-targetPort

• Create an additional service to expose the metrics internally. The existing service is an internet-facing load balancer, so we don't expose the metrics port publicly.

[smlx]

Nice find! IMO the additional service makes the most sense since it matches the approach we already have with other logging charts. e.g. https://github.com/uselagoon/lagoon-charts/blob/main/charts/lagoon-logs-concentrator/templates/metrics-service.yaml