Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-13142] SSE Streaming is broken (follow up to TT-12318) #6554

Open
alekc opened this issue Sep 19, 2024 · 1 comment
Open

[TT-13142] SSE Streaming is broken (follow up to TT-12318) #6554

alekc opened this issue Sep 19, 2024 · 1 comment

Comments

@alekc
Copy link

alekc commented Sep 19, 2024

Original bug (#6322) is marked as solved, however it's not the case.

If the response is coming from SSE streaming, AND record detailed statistics is enabled, then it would crash out with panic.

2024/09/19 17:32:03 http: panic serving [::1]:53263: runtime error: invalid memory address or nil pointer dereference
goroutine 265 [running]:
net/http.(*conn).serve.func1()
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:1903 +0x118
panic({0x1060041a0?, 0x1071d5260?})
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/runtime/panic.go:770 +0xf0
io.ReadAll({0x0, 0x0})
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/io/io.go:712 +0xb8
github.com/TykTechnologies/tyk/gateway.(*SuccessHandler).RecordHit(0x14000eb9c20, 0x14000614d80, {0xa850, 0x72}, 0xc8, 0x14000131b00)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/handler_success.go:223 +0x5f4
github.com/TykTechnologies/tyk/gateway.(*SuccessHandler).ServeHTTP(0x14000eb9c20, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/handler_success.go:380 +0x408
github.com/TykTechnologies/tyk/gateway.(*DummyProxyHandler).ServeHTTP(0x14000eb9c20, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/api_loader.go:652 +0xf08
github.com/TykTechnologies/tyk/gateway.(*Gateway).createMiddleware.func1.1({0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/middleware.go:201 +0xef0
net/http.HandlerFunc.ServeHTTP(0x14000ed8000, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2171 +0x40
github.com/TykTechnologies/tyk/gateway.(*Gateway).createMiddleware.func1.1({0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/middleware.go:201 +0xef0
net/http.HandlerFunc.ServeHTTP(0x14000ed8050, {0x10630e7b0, 0x140000f2000}, 0x14000614d80)
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2171 +0x40
github.com/TykTechnologies/tyk/trace.Handle.func1({0x10630e7b0, 0x140000f2000}, 0x14000614a20)
        /Users/alexander.chernov/dev/genai/go/tyk/trace/handler.go:11 +0x148
net/http.HandlerFunc.ServeHTTP(0x14000ed24e0, {0x10630e7b0, 0x140000f2000}, 0x14000614a20)
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2171 +0x40
github.com/gorilla/mux.(*Router).ServeHTTP(0x140003fd200, {0x10630e7b0, 0x140000f2000}, 0x14000614a20)
        /Users/alexander.chernov/go/pkg/mod/github.com/gorilla/[email protected]/mux.go:212 +0x254
github.com/TykTechnologies/tyk/gateway.(*handleWrapper).ServeHTTP(0x140002dcee0, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/proxy_muxer.go:104 +0x3a4
golang.org/x/net/http2/h2c.h2cHandler.ServeHTTP({{0x1062ffc28, 0x140002dcee0}, 0x14000644780}, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/go/pkg/mod/golang.org/x/[email protected]/http2/h2c/h2c.go:125 +0xb80
github.com/TykTechnologies/tyk/gateway.(*h2cWrapper).ServeHTTP(0x140004026d8, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/dev/genai/go/tyk/gateway/proxy_muxer.go:46 +0x40
net/http.serverHandler.ServeHTTP({0x14000472b40}, {0x10630e7b0, 0x140000f2000}, 0x140006147e0)
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3142 +0x2b0
net/http.(*conn).serve(0x140001315f0, {0x106311f20, 0x14000f22550})
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:2044 +0x15f8
created by net/http.(*Server).Serve in goroutine 80
        /Users/alexander.chernov/go/pkg/mod/golang.org/[email protected]/src/net/http/server.go:3290 +0x88c

As per previous report this is caused by the line responseContent, err := io.ReadAll(responseCopy.Body) where responseCopy.Body is nil.

Happening on the 5.3.4 / release-5.3 branch

@alekc alekc added the bug label Sep 19, 2024
@Yubee116 Yubee116 changed the title SSE Streaming is broken (follow up to TT-12318) [TT-13142] SSE Streaming is broken (follow up to TT-12318) Oct 8, 2024
@JanMa
Copy link

JanMa commented Oct 9, 2024

@alekc I am running in the same issue. It seems this is caused by enabling opentelemetry support in the Tyk gateway. If you disable it, SSE will work again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants