Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor UTF-8 decoding to enable use of SIMD. #23

Closed
wants to merge 1 commit into from

Conversation

christianparpart
Copy link
Member

@christianparpart christianparpart commented Jul 12, 2021

Implements #20.

CHECKLIST

  • basic SSE2 accelerated decoder (128-bit version; based on BobSteagall's utf_utils)
  • AVX decoder (256-bit?)
  • AVX2 decoer (256-bit)
  • AVX512 decoder?
  • AVX512-BW accelerated decoder (512-bit version)
  • only compile SIMD variants if target arch does support it.
  • default to optimal implementation for target host
  • add accelerator::runtime_detect to chose best implementation at runtime (the result should be cached but also the cache should be invalidatable?)
  • improve API documentation on members

@github-actions github-actions bot added the test label Jul 12, 2021
@christianparpart christianparpart force-pushed the feature/fast-utf8-to-utf32 branch 4 times, most recently from e136fda to 912d3a6 Compare July 13, 2021 23:20
@christianparpart christianparpart added the enhancement New feature or request label Jul 13, 2021
@christianparpart christianparpart force-pushed the feature/fast-utf8-to-utf32 branch 7 times, most recently from 4ddc1e4 to 229e1d7 Compare July 14, 2021 00:14
@christianparpart christianparpart force-pushed the feature/fast-utf8-to-utf32 branch from 229e1d7 to 9750a3a Compare July 14, 2021 09:21
@data-man
Copy link
Contributor

data-man commented Jul 17, 2021

add accelerator::runtime_detect to chose best implementation at runtime

Can we use some third-party library for this (and in Contour too)?
E.g. https://github.com/google/cpu_features, https://github.com/anrieff/libcpuid or https://github.com/steinwurf/cpuid.

@christianparpart
Copy link
Member Author

add accelerator::runtime_detect to chose best implementation at runtime

Can we use some third-party library for this (and in Contour too)?

I think CPU feature testing is so simple and just a few lines of code with regards to what we need. There is no need to pull in another dependency. :)

p.s. i have the code already around.

@christianparpart
Copy link
Member Author

outdated, and most importantly, superseeded by our new scan API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants