Memory leak in Crowdsec addon?

[Matthias-vdE]

I recently moved all three components (npmplus, crowdsec, and geoipupdate) from three separate docker-compose files into one to make things a bit more manageable:

services:
  npmplus:
    container_name: npmplus
    image: zoeyvid/npmplus:latest
    restart: always
    network_mode: host
    volumes:
      - "/opt/npm:/data"
    environment:
      - "TZ=Europe/Brussels" # set timezone, required
      - "NGINX_LOG_NOT_FOUND=true" # Allow logging of 404 errors, default false
      - "LOGROTATE=true" # Enables writing http access logs to /opt/npm/nginx/access.log, stream access logs to /opt/npm/nginx/stream.log and enables daily logrotation, default false
      - "LOGROTATIONS=7" # Set how often the access.log should be rotated until it is deleted, default 3
      - "GOA=true" # Enables goaccess, requires LOGROTATE, default false --- if you download the GeoLite2-Country.mmdb, GeoLite2-City.mmdb AND GeoLite2-ASN.mmdb file from MaxMind and place them in /opt/npm/etc/goaccess/geoip it will automatically enable GeoIP in goaccess after restarting NPMplus (no need to change GOACLA below), you may also use the compose.geoip.yaml

  crowdsec:
    container_name: crowdsec
    image: crowdsecurity/crowdsec
    restart: always
    network_mode: bridge
    ports:
      - "127.0.0.1:7422:7422"
      - "127.0.0.1:8080:8080"
    environment:
      - "TZ=Europe/Brussels"
      - "COLLECTIONS=ZoeyVid/npmplus"
    volumes:
      - "/opt/crowdsec/conf:/etc/crowdsec"
      - "/opt/crowdsec/data:/var/lib/crowdsec/data"
      - "/opt/npm/nginx:/opt/npm/nginx:ro"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

  geoipupdate:
    container_name: geoipupdate
    image: maxmindinc/geoipupdate
    restart: always
    network_mode: bridge
    environment:
      - "TZ=Europe/Brussels"
      - "GEOIPUPDATE_EDITION_IDS=GeoLite2-Country GeoLite2-City GeoLite2-ASN"
      - "GEOIPUPDATE_ACCOUNT_ID=SECRET"
      - "GEOIPUPDATE_LICENSE_KEY=SUPERSECRET"
      - "GEOIPUPDATE_FREQUENCY=24"
    volumes:
      - "/opt/npm/etc/goaccess/geoip:/usr/share/GeoIP"

Everything seems to be working, but for some reason the crowdsec container's memory usage keeps growing over time:

Looking at the crowdsec log, it's also being flooded with these messages:

time="2024-09-06T08:40:04+02:00" level=info msg="start tail for container npmplus" container_name=npmplus type=docker

time="2024-09-06T08:40:04+02:00" level=info msg="container acquisition stopped for container 'npmplus'"

time="2024-09-06T08:40:04+02:00" level=info msg="container acquisition stopped for container 'npmplus'"

time="2024-09-06T08:40:05+02:00" level=info msg="start tail for container npmplus" container_name=npmplus type=docker

time="2024-09-06T08:40:05+02:00" level=info msg="start tail for container npmplus" container_name=npmplus type=docker

time="2024-09-06T08:40:05+02:00" level=info msg="container acquisition stopped for container 'npmplus'"

time="2024-09-06T08:40:05+02:00" level=info msg="container acquisition stopped for container 'npmplus'"

time="2024-09-06T08:40:06+02:00" level=info msg="start tail for container npmplus" container_name=npmplus type=docker

time="2024-09-06T08:40:06+02:00" level=info msg="start tail for container npmplus" container_name=npmplus type=docker

time="2024-09-06T08:40:06+02:00" level=info msg="container acquisition stopped for container 'npmplus'"

time="2024-09-06T08:40:06+02:00" level=info msg="container acquisition stopped for container 'npmplus'"

Which I haven't seen before. Are these logs expected and/or normal behavior? Perhaps the memory increase is just due to the logs being filled so quickly? And if that's the case, is there a way to disable these info logs?

I completely reset my configuration (removed all config files from /opt/crowdsec as well as /opt/npm/nginx/etc/crowdsec) and re-registered the crowdsec container using the README https://github.com/ZoeyVid/NPMplus?tab=readme-ov-file#crowdsec but the issue remains. Memory usage grew to 6GB and counting overnight.

[Zoey2936]

so is it npmplus or crowdsec itself which uses this much ram?

[Matthias-vdE]

It's the crowdsec container (so the issue might be better suited for the crowdsec github) but if it only starts acting this way when registering the bouncer in npmplus. Currently, crowdsec is running without any problems or memory issues. But the moment I follow the README to register the bouncer and restart everything it starts spewing these logs and fills the memory incredibly fast.

[Zoey2936]

Yes, the reason could be the huge log, but I would recommand asking crowedsec

[Matthias-vdE]

I pulled the latest version of crowdsec and tried again. The issue appears to be gone now.