Skip to content

Log forwarder for SumoLogic OTel collector

License

Notifications You must be signed in to change notification settings

dpc-sdp/tide_logs

Folders and files

NameName
Last commit message
Last commit date
Aug 15, 2022
Oct 8, 2024
Nov 1, 2022
Jul 29, 2022
Nov 1, 2022
Nov 17, 2023
Nov 17, 2023
Feb 18, 2022
Nov 17, 2023
Feb 18, 2022
Feb 18, 2022

Repository files navigation

Tide Logs

Provides a SumoLogic handler for Monolog.

Requirements

There is also a soft dependency on the following modules

Activation

  1. The module requires a working Sumo Logic OTEL Collector to which it can send logs.
    1. A helm chart has been created here for easy installation on Kubernetes clusters.
    2. To test locally, in a test project:
      1. Create a file at .docker/sumologic-otel/config.yaml with the following content:
        extensions:
          sumologic:
            # Get from https://service.au.sumologic.com/ui/#/security/installation-tokens
            install_token: <collector-install-token>
             # Highly recommend adding a unique suffix, otherwise you might run into conflicts with other collector instances.
            collector_name: SDP Syslog - OTEL - <random-suffix>
            clobber: true
        
        receivers:
          udplog:
            listen_address: "0.0.0.0:514"
            attributes:
              source_name: "foobar/otel/test"
            operators:
              - type: json_parser
              - type: metadata
                id: metadata/source_host
                if: '"source_host" in $$body'
                attributes:
                  source_host: 'EXPR($$body.source_host)'
              - type: metadata
                id: metadata/source_category
                if: '"source_category" in $$body'
                attributes:
                  source_category: 'EXPR($$body.source_category)'
        
        processors: {}
        
        exporters:
          logging:
            loglevel: debug
          sumologic:
            auth:
              authenticator: sumologic
            source_category: "%{source_category}"
            source_name: "%{source_name}"
            source_host: "%{source_host}"
            metadata_attributes:
              - source.*
        
        service:
          extensions: [sumologic]
          pipelines:
            logs:
              receivers: [udplog]
              exporters: [sumologic]
        
      2. Add a service to docker-compose.yml as follows:
        sumo-otel:
          image: "public.ecr.aws/sumologic/sumologic-otel-collector:${SUMO_OTEL_RELEASE_VERSION:-0.47.0-sumo-0}"
          volumes:
            - .docker/sumologic-otel:/etc/otel
          ports:
            - 514
          networks:
            - amazeeio-network
            - default
          labels:
            lagoon.type: none
      3. Running ahoy up will register the instance with Sumo Logic; it's now ready to start collecting logs. Run docker-compose logs -f sumo-otel to make sure there were no errors. The last line should be something like
        Everything is ready. Begin running and processing data.
        
  2. Enable the Tide Logs module. Go to /admin/config/development/tide_logs and ensure UDPlog host and UDPlog port correspond to the service's name and the port in the config.yaml respectively, if running with docker-compose. The module's default is udp://logger.default.svc:5514 (see the default config here).
  3. SUMOLOGIC_CATEGORY can also be set if a different category from the default (sdp/dev/tide) is required.
  4. The following search query can then be used in SumoLogic to view the logs:
    _collector="SDP Syslog - OTEL - <random-suffix>" and _sourceCategory="sdp/dev/tide"
    

Debug

Some very basic debug messages can be printed locally (or remotely if you have drush access) by setting the following config variable:

drush config:set tide_logs.settings debug 1