Helidon Webclient (4.0.10) is not routing the requests through proxy configured using Proxy Builder.

[shashikumarmh77]

Environment Details

• Helidon Version: 4.0.10
• Helidon SE or Helidon MP - Helidon SE
• JDK version: 21
• OS: Linux
• Docker version (if applicable):

---

Problem Description

I am trying to configure MockServer as a proxy to facilitate testing of REST requests.
In the following simplified test case, http://echo.jsontest.com/1/one is invoked. The actual endpoint returns {"1": "one"} but MockServer configured as proxy should return mocked response of {"1": "ONE"}

@Test
    public void helidonClientWithProxySetUsingSystem() throws ExecutionException, InterruptedException, IOException {
        final String URL = "http://echo.jsontest.com/1/one";
        var builder = WebClient.builder().followRedirects(true).relativeUris(true)
                // set proxy address.
                .proxy(Proxy.builder().type(Proxy.ProxyType.HTTP).host("localhost").port(1026))
                .baseUri(URL);
        var webClient = builder.build();
        var webClientRequestBuilder = webClient.method(Method.GET);
        System.out.println("Invoking " + URL);
        try (HttpClientResponse response = webClientRequestBuilder.request()) {
            System.out.println("Response code with WebClient:" + response.status().code());
        }
    }

However, the execution results in below exception.

o.helidon.common.buffers.DataReader$InsufficientDataAvailableException
	at io.helidon.common.buffers.DataReader.pullData(DataReader.java:83)
    ...
	at io.helidon.webclient.http1.Http1StatusParser.readStatus(Http1StatusParser.java:48)
	at io.helidon.webclient.http1.Http1CallChainBase.readResponse(Http1CallChainBase.java:218)
	at io.helidon.webclient.http1.Http1CallEntityChain.doProceed(Http1CallEntityChain.java:66)
	at io.helidon.webclient.http1.Http1CallChainBase.proceed(Http1CallChainBase.java:149)
	at io.helidon.webclient.api.ClientRequestBase.invokeServices(ClientRequestBase.java:395)
	at io.helidon.webclient.http1.Http1ClientRequestImpl.invokeWithServices(Http1ClientRequestImpl.java:205)
	at io.helidon.webclient.http1.Http1ClientRequestImpl.invokeRequestWithEntity(Http1ClientRequestImpl.java:195)
	at io.helidon.webclient.http1.RedirectionProcessor.invokeWithFollowRedirects(RedirectionProcessor.java:49)
	at io.helidon.webclient.http1.RedirectionProcessor.invokeWithFollowRedirects(RedirectionProcessor.java:40)
    ...
	at io.helidon.webclient.api.ClientRequestBase.validateAndSubmit(ClientRequestBase.java:452)
	at io.helidon.webclient.api.ClientRequestBase.submit(ClientRequestBase.java:266)
	at io.helidon.webclient.api.HttpClientRequest.doSubmit(HttpClientRequest.java:79)
	at io.helidon.webclient.api.ClientRequestBase.validateAndSubmit(ClientRequestBase.java:452)
	at io.helidon.webclient.api.ClientRequestBase.request(ClientRequestBase.java:257)

Steps to reproduce

1. Download the mockserver jar and start mockserver (test expects it at port 1026), see this page
2. Register an expectation
   curl -X PUT "http://localhost:1026/mockserver/expectation" -H 'Content-Type: application/x-www-form-urlencoded' -d '{ "httpRequest": { "method": "GET", "path": "/1/one" }, "httpResponse": {"body": {"1": "ONE"}}}'
3. Run above test which results in the above error.

Sample code:
ClientTest.txt

[klustria]

Some observations to this issue:

1. The behavior is the same for Helidon 3 Webclient.
2. Proxying with curl to MockServer works. User also claims that jdk's httpURLConnection() also works.
3. Proxying with Helidon Webclient works with mitmproxy but fails with MockServer. After inspecting the protocol exchange using Wireshark, it looks like MockServer does not respond to Connect which is what the Webclient uses to initiate proxy, while mitmproxy does.
4. Curl does not use CONNECT to proxy when using http, but looks like it does for https. It seems to use the same method as the request, for example, it just uses GET, when the request is a GET.

[spericas]

https://www.mock-server.com/proxy/configuring_sut.html

[jbescos]

It works if we avoid the HTTP CONNECT when it is HTTP (not HTTPs). However, I think this is an issue in MockServer, because CONNECT is also necessary with HTTP when you use authentication. They should always respond to CONNECT.

I am going to open them one issue.

My reproducer for test:

@ServerTest
class HttpMockProxyTest {

    private static final int PROXY_PORT = 1080;
    private final WebClient webClient;
    private final int serverPort;
    private ClientAndServer mockServer;

    HttpMockProxyTest(WebServer server) {
        this.serverPort = server.port();
        this.webClient = WebClient.builder()
                .baseUri("http://localhost:" + serverPort)
                .proxy(Proxy.builder().host("localhost").port(PROXY_PORT).build())
                .build();
    }

    @SetUpRoute
    static void routing(HttpRouting.Builder router) {
        router.route(GET, "/get", Routes::get);
    }

    @BeforeEach
    public void before() {
        mockServer = startClientAndServer(PROXY_PORT);
        mockServer.when(request()).forward(forward().withScheme(Scheme.HTTP).withHost("localhost").withPort(serverPort)); 
    }

    @AfterEach
    public void after() {
        mockServer.stop();
    }

    @Test
    public void issue9022() {
        try (HttpClientResponse response = webClient.get("/get").request()) {
            assertThat(response.status(), is(Status.OK_200));
            String entity = response.entity().as(String.class);
            assertThat(entity, is("Hello"));
        }
    }

    private static class Routes {
        private static String get() {
            return "Hello";
        }
    }
}

[jbescos]

I have created an issue there: mock-server/mockserver#1890

[shashikumarmh77]

@jbescos Thanks for filing a bug with mock server. Since curl and HttpURLConnection works is it possible to bypass the CONNECT used by helidon to connect to the proxy as done by curl and HttpUTLConnection (May be introducing a flag at builder level)

[jbescos]

I created a PR in the way @shashikumarmh77 suggested.

[klustria]

Closing this issue as @jbescos has merged the resolution to this issue via PR #9023.