Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1732907: Change custom cloud storage header metadata handling to be case-insensitive #1919

Merged
merged 13 commits into from
Oct 18, 2024
Merged

SNOW-1732907: Change custom cloud storage header metadata handling to be case-insensitive #1919

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
fbc5449
SNOW-1732907 Change custom Azure header metadata handling to be case-…
sfc-gh-dheyman Oct 15, 2024
e61605d
Merge branch 'master' into SNOW-1732907-custom-azure-headers
sfc-gh-dheyman Oct 15, 2024
e1dd5ac
Add GCS & Azure Blob Storage header handling
sfc-gh-dheyman Oct 16, 2024
124a3b3
Merge branch 'master' of github.com:snowflakedb/snowflake-jdbc into S…
sfc-gh-dheyman Oct 16, 2024
b22f43c
Merge branch 'SNOW-1732907-custom-azure-headers' of github.com:snowfl…
sfc-gh-dheyman Oct 16, 2024
f3bb222
Merge branch 'master' into SNOW-1732907-custom-azure-headers
sfc-gh-dheyman Oct 16, 2024
5678d58
Remove proxy
sfc-gh-dheyman Oct 17, 2024
f9db619
Merge branch 'SNOW-1732907-custom-azure-headers' of github.com:snowfl…
sfc-gh-dheyman Oct 17, 2024
95defa9
Remove test changes
sfc-gh-dheyman Oct 17, 2024
6ce45ae
Add null check
sfc-gh-dheyman Oct 17, 2024
145c998
Fix checkstyle
sfc-gh-dheyman Oct 17, 2024
af2293d
Fix test
sfc-gh-dheyman Oct 17, 2024
3cec597
Fix casting
sfc-gh-dheyman Oct 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 16 additions & 6 deletions src/main/java/net/snowflake/client/jdbc/cloud/storage/SnowflakeAzureClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;

import net.snowflake.client.core.HttpUtil;
import net.snowflake.client.core.ObjectMapperFactory;
import net.snowflake.client.core.SFBaseSession;
Expand Down Expand Up @@ -273,7 +275,7 @@ public StorageObjectMetadata getObjectMetadata(String remoteStorageLocation, Str
blob.downloadAttributes(null, null, opContext);

// Get the user-defined BLOB metadata
Map<String, String> userDefinedMetadata = blob.getMetadata();
Map<String, String> userDefinedMetadata = getUserDefinedMetadata(blob);

// Get the BLOB system properties we care about
BlobProperties properties = blob.getProperties();
Expand Down Expand Up @@ -348,7 +350,7 @@ public void download(
blob.downloadAttributes(null, transferOptions, opContext);

// Get the user-defined BLOB metadata
Map<String, String> userDefinedMetadata = blob.getMetadata();
Map<String, String> userDefinedMetadata = getUserDefinedMetadata(blob);
AbstractMap.SimpleEntry<String, String> encryptionData =
parseEncryptionData(userDefinedMetadata.get(AZ_ENCRYPTIONDATAPROP), queryId);

Expand Down Expand Up @@ -447,13 +449,10 @@ public InputStream downloadToStream(
InputStream stream = blob.openInputStream(null, null, opContext);
stopwatch.stop();
long downloadMillis = stopwatch.elapsedMillis();
Map<String, String> userDefinedMetadata = blob.getMetadata();

Map<String, String> userDefinedMetadata = getUserDefinedMetadata(blob);
AbstractMap.SimpleEntry<String, String> encryptionData =
parseEncryptionData(userDefinedMetadata.get(AZ_ENCRYPTIONDATAPROP), queryId);

String key = encryptionData.getKey();

String iv = encryptionData.getValue();

if (this.isEncrypting() && this.getEncryptionKeySize() <= 256) {
Expand Down Expand Up @@ -934,6 +933,17 @@ private static URI buildAzureStorageEndpointURI(String storageEndPoint, String s
return storageEndpoint;
}

/*
* getUserDefinedMetadata
* Method introduced to avoid inconsistencies in custom headers handling, since these are defined on drivers side
* e.g. some drivers might internally convert headers to canonical form.
*/
private Map<String, String> getUserDefinedMetadata(CloudBlob blob) {
sfc-gh-dheyman marked this conversation as resolved.
Show resolved Hide resolved
Map<String, String> userDefinedMetadata = new TreeMap<>(String.CASE_INSENSITIVE_ORDER);
userDefinedMetadata.putAll(blob.getMetadata());
return userDefinedMetadata;
}

/*
* buildEncryptionMetadataJSON
* Takes the base64-encoded iv and key and creates the JSON block to be
Expand Down
Loading