SNOW-999335: Spark snowflake read results in certificate issue #1591
Comments
github-actions
bot
changed the title
|
@dyang108 thanks for reporting this. Do you know if you have a proxy server in your environment and if you expect your S3 connection to go through it or it should bypass it? Can you open a support case and provide the Spark logs after adding the following JVM argument, please? You would need to add that to the Spark driver's extra Java options, for instance: |
|
@dyang108 do you still need help with this? |
|
I did a downgrade to 3.12.12 as a workaround. I think this remains an issue on 3.14.2. I'm not sure if we have a proxy configured - if we do, this would be the first i've heard of it. |
|
@dyang108 It's definitely interesting that your issue goes away after you downgrade to 3.12.12, but I'm not sure why that would be the case, and that's a pretty old version. Based on the error stack the problem was raised by the Apache HTTP client code when verifying the hostname, and that library version changed from v4.5.5 to v4.5.14 between those two JDBC driver versions. I don't see any difference in the implementation of that method in both of those branches: The challenge here is that this isn't something I'm able to reproduce, so it's hard to really say what might be going on here. While the JDBC driver version change has caused some sort of problem in your case, there's just no way for us to debug your problem without additional information. A good place to start is to review the output produced by the following JVM argument when reproducing the issue: Will you be able to open a support case to share that information with us? Otherwise, I'm not entirely sure how else we can look into this. |
sfc-gh-wfateem
added
the
status-information_needed
|
I'm going to close this issue for now. If you're able to provide additional information to help us debug the issue then please feel free to open this again. |
|
Reopening case since we received more information about the problem from a different user that experienced the same issue. |
|
I've encountered this issue today with the Kafka Connector 2.2.1 and 2.1.2. Maybe this can help you track down the issue |
|
@dyang108 @richard-axual we debugged this extensively with a user experiencing the same issue and based on our findings this had to do with the Apache HTTP client's usage of the public-suffix-list.txt file. PR #1690 addresses that issue and was included in version 3.15.1 which the user confirmed addressed their problem. |
|
@sfc-gh-wfateem Thanks for the update and explanation. |
|
@richard-axual was your issue consistent or was it a one-time problem you experienced with the Snowflake Kafka Connector? The issue we're discussing here is a consistent problem once the JDBC driver version is upgraded. |
|
@sfc-gh-wfateem The error with the updated version of the connector is consistent, but since it is on a different intermediate project I cannot guarantee it's the same problem |
|
Thanks @richard-axual |
|
I'm going to close this issue based on the assumption that #1690 addresses the issue, if not, please feel free to open this again. |
sfc-gh-wfateem
removed
the
status-information_needed
Please answer these questions before submitting your issue.
In order to accurately debug the issue this information is required. Thanks!
What version of JDBC driver are you using?
3.14.2
What operating system and processor architecture are you using?
amazon linux
What version of Java are you using?
11
What did you do?
Reading from snowflake with snowflake jdbc driver in Spark results in S3 ssl certificate issue. It seems like our Spark job is trying to access a snowflake customer staging bucket and is failing:
I reverted back to 3.12.12 and that helped fix the issue for me, but I wanted to flag this moving forward. LMK if this belongs better in the spark-snowflake project.
The text was updated successfully, but these errors were encountered: