fix: kafka.errors.MessageSizeTooLargeError

[jk2K]

Problem

fix #19633

solution

1. kafka producer config: max.request.size set to 20971520
2. kafka server config: message.max.bytes set to 20971520
3. posthog kafka client, SESSION_RECORDING_KAFKA_MAX_REQUEST_SIZE_BYTES set to 20971520

related to bitnami/containers#54458

another thing

1. put Caddy's data into caddy-data docker volume, I use Local HTTPS, Caddy will generate local CA, when recreating the container, avoid reinstalling Caddy's root CA on my host machine

[benjackwhite]

Personally I see no issue with it, especially given the default docker compose should support the higher message size 🤔

[pauldambra]

@benjackwhite only thing I can think is that maybe it should be configurable... 2MB works for this user... 1MB has worked previously without reports... We really should have better visibility on how big these messages are and what users can do about it 🤔

[benjackwhite]

@benjackwhite only thing I can think is that maybe it should be configurable... 2MB works for this user... 1MB has worked previously without reports... We really should have better visibility on how big these messages are and what users can do about it 🤔

Given this is the hobby deploy and the kafka config only affects this local kafka, I think it's still pretty safe.
If they have externalised kafka then none of this really applies anyways and they can still override the env var so 🤷

[xvello]

I do not recommend changing the defaults for the hobby deploy:

• as Paul has mentioned, 2MiB would not fix it for everyone anyway, and increasing the Kafka message size too much comes with operational challenges on the Kafka side that I do not wish to investigate
• the analytics pipeline assumes a 1MiB limit and does not check sizes on the producer size. It probably should, but in the meantime increasing this default introduces the risk of a PR working on local devenv and CI (with 2MiB message limit), then failing on prod where we use 1MiB. I don't want to take that risk
• lastly, our compose stack does not use the bitnami image anymore, but https://github.com/PostHog/kafka-container, so this PR would not use the linked PR

I don't oppose making it configurable, but I strongly suggest to keep the default at 1MiB.

[posthog-bot]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

[posthog-bot]

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.