-
Notifications
You must be signed in to change notification settings - Fork 170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SNOW-1501662 Allow JDBC to handle ZSTD compressed http response streams #1802
Merged
sfc-gh-dbouassida
merged 4 commits into
master
from
dbouassida-SNOW-1501662-allow-jdbc-to-decompress-zstd-compressed-query-results
Sep 10, 2024
Merged
SNOW-1501662 Allow JDBC to handle ZSTD compressed http response streams #1802
sfc-gh-dbouassida
merged 4 commits into
master
from
dbouassida-SNOW-1501662-allow-jdbc-to-decompress-zstd-compressed-query-results
Sep 10, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
All contributors have signed the CLA ✍️ ✅ |
sfc-gh-dbouassida
changed the title
SNOW-1452649 update dependency to httpclient library to support zstd
SNOW-1452649 Allow JDBC to handle ZSTD compressed query results
Jun 24, 2024
sfc-gh-dbouassida
force-pushed
the
dbouassida-SNOW-1501662-allow-jdbc-to-decompress-zstd-compressed-query-results
branch
from
September 3, 2024 12:26
b2a52a0
to
b789b62
Compare
sfc-gh-dbouassida
requested review from
sfc-gh-pbulawa,
sfc-gh-akolodziejczyk and
sfc-gh-ioukid
September 3, 2024 15:30
sfc-gh-dbouassida
force-pushed
the
dbouassida-SNOW-1501662-allow-jdbc-to-decompress-zstd-compressed-query-results
branch
from
September 5, 2024 13:08
2d50eb2
to
0748b77
Compare
sfc-gh-dbouassida
force-pushed
the
dbouassida-SNOW-1501662-allow-jdbc-to-decompress-zstd-compressed-query-results
branch
from
September 5, 2024 15:14
0748b77
to
7ed1497
Compare
I have read the CLA Document and I hereby sign the CLA |
sfc-gh-pbulawa
approved these changes
Sep 6, 2024
sfc-gh-dbouassida
changed the title
SNOW-1452649 Allow JDBC to handle ZSTD compressed query results
SNOW-1501662 Allow JDBC to handle ZSTD compressed query results
Sep 6, 2024
sfc-gh-ioukid
approved these changes
Sep 9, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
sfc-gh-pmotacki
approved these changes
Sep 9, 2024
sfc-gh-dbouassida
changed the title
SNOW-1501662 Allow JDBC to handle ZSTD compressed query results
SNOW-1501662 Allow JDBC to handle ZSTD compressed http response streams
Sep 9, 2024
…mpress-zstd-compressed-query-results
sfc-gh-dbouassida
deleted the
dbouassida-SNOW-1501662-allow-jdbc-to-decompress-zstd-compressed-query-results
branch
September 10, 2024 08:14
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
SNOW-1501662
Summary
This PR introduces the integration of the
zstd-jni
library into Snowflake JDBC.Key Changes
Dependency Addition:
zstd-jni
version1.5.6-5
as a dependency to support Zstandard compression.pom.xml
,FIPS/pom.xml
,thin_public_pom.xml
,parent-pom.xml
) to include thezstd-jni
dependency.Shading and Relocation:
com.github.luben.zstd
into${shadeBase}.com.github.luben.zstd
.FIPS/pom.xml
andpom.xml
to include relocation rules for thezstd-jni
package.Code Integration:
DefaultResultStreamProvider.java
to handle streams encoded with Zstandard. The code now checks for thezstd
encoding and usesZstdInputStream
for decompression.Script Updates:
ci/scripts/check_content.sh
andFIPS/scripts/check_content.sh
to ignore native library files (.so
,.dylib
,.dll
) located in OS-specific directories (aix
,darwin
,freebsd
,linux
,win
). This prevents errors during the dependency shading check.Linkage Checker Updates:
com.github.luben.zstd
inlinkage-checker-exclusion-rules.xml
, as the dependencies are now properly shaded and relocated.Pre-review self checklist
master
branchmvn -P check-style validate
)mvn verify
and inspecttarget/japicmp/japicmp.html
)SNOW-XXXX:
External contributors - please answer these questions before submitting a pull request. Thanks!
What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes #NNNN
Fill out the following pre-review checklist:
@SnowflakeJdbcInternalApi
(note that public/protected methods/fields in classes marked with this annotation are already internal)Please describe how your code solves the related issue.
Please write a short description of how your code change solves the related issue.