Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Compare performance of decompression engine (HW) versus decompression kernels (SW) on Blackwell #17247

Open
GregoryKimball opened this issue Nov 5, 2024 · 0 comments
Labels
cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.

Comments

@GregoryKimball
Copy link
Contributor

GregoryKimball commented Nov 5, 2024

Is your feature request related to a problem? Please describe.
The NVIDIA Blackwell Architecture includes a "decompression engine". This hardware component complements the existing approach for decompression that relies on nvCOMP and specialized decompression kernels. Let's evaluate the decompression throughput for the HW and SW implementations across the cuIO benchmarks.

Blackwell’s Decompression Engine and ability to access massive amounts of memory in the NVIDIA Grace™ CPU over a high-speed link—900 gigabytes per second (GB/s) of bidirectional bandwidth—accelerate the full pipeline of database queries for the highest performance in data analytics and data science with support for the latest compression formats such as LZ4, Snappy, and Deflate.

The cuIO benchmarks for parquet files use the "cardinality" and "run_length" axes to control page encoding efficiency and compression ratio. It's likely that the page count, compression ratio and page size influence decompression throughput.

@GregoryKimball GregoryKimball added cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. labels Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Projects
Status: In Progress
Status: No status
Development

No branches or pull requests

1 participant