-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
simd: add new SIMD support for JSON escaping #9500
Conversation
The following change in the utils write utility, improve the handling of characters that needs escaping by optimizing the character check with a lookup table. Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
Signed-off-by: Eduardo Silva <[email protected]>
@pwhelan @leonardo-albertovich @cosmo0920 @pwhelan @patrick-stephens I need your help on this for workflows and overall testing:
so:
comments are welcome |
The one of the candidates is RISC-V vector extension("RVV"): |
For safety, we might need to have two images for PC architecture that is:
PC architectures are fragmented. |
thanks for the feedback. Merging it for now since the feature needs to be enabled at build time |
For cases where JSON encoding is needed, if SIMD is available, this brings 30%-50% performance improvement.
PostgreSQL SIMD
The header file with SIMD functionality has been taken from the PostgreSQL project, stripped down, and adapted for our specific needs.
Notes on other improvements
the routine that is used to escape characters, now uses a lookup table which heavily improves performance when Fluent Bit is built on release mode (optimizations on). This brings performance improvements for all systems/architectures.
SIMD operations are available for architectures that implement SSE2 (Intel/AMD) and Neon (Arm) based instructions. Note that AVX2 is not implemented so there is still more room for improvement.
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.