feat: rust funnel udfs #25326
Merged
feat: rust funnel udfs #25326
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aspicer
force-pushed
the
aspicer/rust_udf
branch
from
c226fe9 to
7bb5b66
Compare
aspicer
force-pushed
the
aspicer/funnel_actors
branch
from
3abc7a5 to
ff7d4e7
Compare
aspicer
force-pushed
the
aspicer/rust_udf
branch
from
7bb5b66 to
86e70e1
Compare
fuziontech
reviewed
Oct 2, 2024
| @@ -79,7 +79,7 @@ services: | |||
| # Note: please keep the default version in sync across | |||
| # `posthog` and the `charts-clickhouse` repos | |||
| # | |||
| image: ${CLICKHOUSE_SERVER_IMAGE:-clickhouse/clickhouse-server:23.12.6.19-alpine} | |||
| image: ${CLICKHOUSE_SERVER_IMAGE:-clickhouse/clickhouse-server:23.12.6.19} | |||
There was a problem hiding this comment.
This is probably a good idea moving off of alpine since this will reflect more what production is
There was a problem hiding this comment.
Yeah for sure - in this case, a glibc binary is more than 2x faster than a MUSL libc binary - apparently the musl libc binary has a slow memory allocator (going for simplicity and portability over speed), but that's huge lol
There was a problem hiding this comment.
I actually started wondering if clickhouse would be faster locally running on a normal container vs an alpine one
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Based off of #25013
For some reason, this makes CI not run backend tests. The full diff is here
Problem
Memory usage too high on python UDFs in Clickhouse.
Changes
Rewrite in rust for a 10x speed improvement. Set a lifetime value in clickhouse that reloads the UDFs every 10 minutes as a safety valve against memory leaking.
Does this work well for both Cloud and self-hosted?
Yes
How did you test this code?
Passed funnels tests.