SNOW-565611: Requests to Snowflake fail using https over HTTP proxy

[knueven]

Please answer these questions before submitting your issue. Thanks!

1. What version of NodeJS are you using (node --version and npm --version)?
   npm: '7.24.2',
   node: '16.8.0',

2. What operating system and processor architecture are you using?
   macOS 12.2.1 (21D62) x86_64 Darwin 21.3.0

3. What are the component versions in the environment (npm list)?
   snowflake-sdk 1.6.7

  '@azure/storage-blob': '^12.5.0',
  'agent-base': '^6.0.2',
  'asn1.js-rfc2560': '^5.0.0',
  'asn1.js-rfc5280': '^3.0.0',
  'aws-sdk': '^2.878.0',
  axios: '^0.21.4',
  'big-integer': '^1.6.43',
  'bignumber.js': '^2.4.0',
  binascii: '0.0.2',
  'browser-request': '^0.3.3',
  debug: '^3.2.6',
  'expand-tilde': '^2.0.2',
  extend: '^3.0.2',
  'generic-pool': '^3.8.2',
  glob: '^7.1.6',
  'http-signature': '^1.3.6',
  'https-proxy-agent': '^5.0.0',
  jsonwebtoken: '^8.5.1',
  'json-schema': '^0.4.0',
  jsprim: '^2.0.2',
  lodash: '^4.17.21',
  'mime-types': '^2.1.29',
  mkdirp: '^1.0.3',
  'mock-require': '^3.0.3',
  moment: '^2.23.0',
  'moment-timezone': '^0.5.15',
  ocsp: '^1.2.0',
  open: '^7.3.1',
  'python-struct': '^1.1.3',
  request: '^2.88.2',
  requestretry: '^7.0.1',
  'simple-lru-cache': '^0.0.2',
  'string-similarity': '^4.0.4',
  'test-console': '^2.0.0',
  tmp: '^0.2.1',
  uuid: '^3.3.2',
  winston: '^3.1.0'
}

5. What did you do?
   Attempt to connect to Snowflake using both pooling and one-use connection methods with proxy_host, proxy_port, proxy_protocol set

6. What did you expect to see?
   A connection to Snowflake completes sucessfully allowing requests to be made

7. What did you see instead?

{"level":"DEBUG","message":"[3:42:08.428 PM]: Contacting SF: /session/v1/login-request?requestId=cf5d151e-f3dd-466c-96af-350ac7c3d5eb, (5/5)"}
{"level":"TRACE","message":"[3:42:08.428 PM]: https://<ACCOUNT>.snowflakecomputing.com/session/v1/login-request?requestId=cf5d151e-f3dd-466c-96af-350ac7c3d5eb&clientStartTime=1648582870017&retryCount=4"}

repeated over and over again.

This is a known bug with Axios (axios/axios#3384) that has popped up for us repeatedly, the solution is to use a custom https.Agent() - a configuration variable allowing passing one here should do the trick.

I noticed there was a problem with proxies on v1.6.8 as reported in another issue, but downgrading to 1.6.7 also returned the same end result.

[monti-python]

Hi @knueven , I also bumped into this issue (#159). Made a PR a while ago (#160) with a suggested solution that's working for me, but it never got reviewed... I've just updated it

[sfc-gh-dszmolka]

hi folks first of all thank you for reporting this issue ! second, massive apologies @monti-python that your issue #159 was closed ; due to an error it happened to a ton of other issues well, but those were reopened. not sure why yours wasnt :(

anyways. tested with your code from issue #159 , using the latest v1.6.17 of the connector; it works flawlessly. i noticed there was a bump to axios 0.27.2 in v1.6.10 (earlier we depended on axios 0.21.4) so maybe they fixed this along the way.
the issue doesn't reproduce anymore with the later connector versions (it does reproduce with e.g. v1.6.9). please let us know if it's still an issue for you.

[monti-python]

Hi @sfc-gh-dszmolka,

Never mind, thanks for having a look at it!

At the moment I'm not actively using the nodejs connector so no longer facing that problem... however it seems this is still an issue with axios when using proxy tunneling (e.g. axios/axios#4531, axios/axios#3384).

In any case feel free to close #160 if this is no longer needed

[sfc-gh-wfateem]

@sfc-gh-dszmolka @sfc-gh-jfan I'm reopening this issue because it's still a problem. I discovered this when working with a customer and found that the connectAsync() call was behaving differently. I was using snowflake-sdk version 1.8.0.

In my case, I confirmed the behavior by using authenticator='EXTERNALBROWSER' and I noticed that the first request to Snowflake to get the SSO URL never went through the proxy server. I tried the inverse by pointing to a bogus proxy server so I would expect that all connections would fail. My connection to Snowflake still went through, and the browser opened up, but everything else after that point fails with a timeout because the proxy server wasn't listening to requests.

[sfc-gh-dszmolka]

i think this was fixed with #659, released in version 1.9.1
thank you everyone for bearing with us !