OAuth over proxy does not work

[alekdavis]

This is a CLI problem only. In the VSCode extension, everything works fine.

ENVIRONMENT

• HttpYac CLI v6.16.4
• NodeJS v23.3.0
• NPM v10.9.0

BACKGROUND

We have a cloud hosted REST endpoint which is exposed directly (for testing) and also via Apigee (for consumption by other apps). The endpoint uses OAuth 2.0 with the client_credentials flow. The OAuth token URL points to Apigee, but Apigee internally calls the token endpoint in the original cloud, so the bearer token can be obtained from either Apigee (when consumed by other apps) or the cloud endpoint (intended for testing only). The Apigee URLs look like they are company URLs (the host part ends with the company domain), but they resolve to external IPs and need to be accessed via a proxy.

PROXY

Here is the proxy requirements matrix:

ENDPOINT	PROXY REQUIRED
API in CLOUD	YES
API in APIGEE	YES
Token in CLOUD	NO
Token in APIGEE	YES

I can obtain a bearer token from the cloud without using a proxy, but for everything else, a proxy is required.

PROBLEM

When I use CLI to test an API in Apigee (and using OAuth 2.0 settings pointing to the token endpoint in Apigee), requests time out with error:

[-] connect ETIMEDOUT 35.227.173.176:443 (RequestError: connect ETIMEDOUT XX.XXX.XXX.XXX: 443 - ClientRequest.<anonymous> (C:\Users\user\AppData\Roaming\npm\node_modules\httpyac\node_modules\got\dist\source\core\index.js:970:111)
RequestError: connect ETIMEDOUT 35.227.173.176:443
    at ClientRequest.<anonymous> (C:\Users\user\AppData\Roaming\npm\node_modules\httpyac\node_modules\got\dist\source\core\index.js:970:111)
    at Object.onceWrapper (node:events:628:26)
    at ClientRequest.emit (node:events:525:35)
    at origin.emit (C:\Users\user\AppData\Roaming\npm\node_modules\httpyac\node_modules\@szmarczak\http-timer\dist\source\index.js:43:20)
    at emitErrorEvent (node:_http_client:104:11)
    at TLSSocket.socketErrorListener (node:_http_client:512:5)
    at TLSSocket.emit (node:events:513:28)
    at emitErrorNT (node:internal/streams/destroy:170:8)
    at emitErrorCloseNT (node:internal/streams/destroy:129:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1615:16)

The XX.XXX.XXX.XXX IP address points to the Apigee (Google) IP address (I assume it is the IP address of the Apigee token endpoint).

WORKAROUND

I figured out two workarounds:

1. When using Apigee API endpoint, use the token URL of the cloud endpoint, since it can be accessed without a proxy (for general use, this would not be allowed).
2. Instead of oauth2 authorization, use Bearer with token generated in a referenced test, so, all tests would have a dependency on a test that generates a token from the Apigee token endpoint (this would be allowed, but it would complicate the code because it would only be required for the CLI execution).

Neither workaround is a permanent solution, so hopefully this issue gets fixed.

IMPACT

Majority of our APIs (not just our team's APIs, the whole company) are either behind Apigee or a similar solution, so it affects pretty much every API.

SUMMARY

Since I can generate a bearer token from the Apigee token endpoint explicitly (using a proxy), the fact that the calls timeout, when a test uses the oauth2 authorization against the same Apigee token endpoint, suggests that the implicit oauth2 authorization call somehow forgets to use the specified proxy settings. Ideally, the OAuth call should allow a separate proxy configuration, but at the very least, it needs to recognize the primary (request) proxy configuration.

[AnWeber]

Currently, the request for the OAuth2 token does not use a proxy. I use neither the proxy of the actual request nor the one from the environment (HTTP_PROXY). The idea was to add the proxy or other values using the variable oauth2_interceptRequest. This method is called beforce executing the oauth2 token request.

{{
  exports.oauth2_interceptRequest = function(request, context) {
    console.info(request);
    request.proxy = ....;
  }
}}
GET https://httpbin.org/anything
Authorization: oauth2 password

Would a variable oauth2_proxy help, which configures the proxy? Always using the proxy of the actual request also feels wrong, doesn't it? I have too little experience of how proxies are used in company networks.

[alekdavis]

@AnWeber You do not want to know how some companies use proxies. :-) The things we run into are crazy. And they are very difficult to troubleshoot.

Anyway, you are right: authorization endpoint may have a totally different proxy requirement from the actual endpoint. At least, in theory. However, based on my experience, in practice, they are mostly the same. I can't think of any case, where they would be different. Also, generally, endpoints that do not need a proxy can be accessed via a proxy (this may involve a bit of overhead, so IT may not appreciate it as a policy, but generally, the calls to internal sites should work over proxy).

Here is the combination of proxy configurations that need to be allowed:

REQUEST	AUTH
no proxy	no proxy
proxy	proxy
no proxy	proxy
proxy	no proxy
proxy1	proxy2

In my mind (and I may be wrong), the preferred behavior for the oauth calls would be:

• Use the same proxy as main request (default).
• Use oauth2-specific proxy.
• Do not use proxy.

So, your suggestion to add the oauth2_proxy variable would probably be the simplest, although in most cases when the same proxy is needed it will need to be duplicated, but it should be fine.

Another thing to keep in mind is that for the bearer token authorization, it may be the same issue, if the token endpoint that generates the token uses different proxy settings, but I am not sure how to set this up, since these would be just calls like any other requests.

[alekdavis]

Thinking of the bearer token endpoint calls, maybe have some sort of metadata flag to indicate that the call should use the auth proxy, e.g.:

# @use_oauth_proxy

Or mark it as an auth token generation request:

# @oauth

Or something like this. I haven't given this a lot of thought, though, so there may be better ways.

[AnWeber]

I am not convinced by the special treatment for Bearer Request. It is also possible to set the proxy manually in this way at any time.

@request_proxy=oauth2_proxy

I would not implement it right now. Proxy support is implemented with next version