This repository has been archived by the owner on Apr 18, 2024. It is now read-only.
Response doesn't end cleanly. #27
Comments
|
Reproduction in golang package main
import (
"fmt"
"io"
"net/http"
"os"
_"time"
)
func main() {
CID := "bafybeifpz6onienrgwvb3mw5rg7piq5jh63ystjn7s5wk6ttezy2gy5xwu/Mexico.JPG"
urlStr := fmt.Sprintf("http://localhost/ipfs/%s?format=car&dag-scope=entity&bytes=0:1048575", CID)
fmt.Println(urlStr)
req, err := http.NewRequest("GET", urlStr, nil)
if err != nil {
panic(err)
}
req.Header.Set("Accept", "application/vnd.ipld.car")
client := http.Client{
//Timeout: time.Second * 5, // Assume file is cached.
}
res, err := client.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
fmt.Println(res.StatusCode)
file, err := os.Create("mexico.car")
if err != nil {
fmt.Println("Error:", err)
return
}
defer file.Close()
_, err = io.Copy(file, res.Body)
if err != nil {
fmt.Println("Error:", err)
return
}
}The script will timeout after 2 minutes with this output. I can still This explains the high volume of "Unexpected EOF" errors that caboose is reporting. Since the connection just hangs until a timeout, it may have other negative impacts across rhea. |
|
when i run the curl command above i just get a timeout |
|
Output from just now It downloads 1MB ( Should be reproducible on a local L1. |
|
I think something is wrong with my machine, I can't repro on another machine. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Run
$ curl "http://138.199.41.51/ipfs/bafybeifpz6onienrgwvb3mw5rg7piq5jh63ystjn7s5wk6ttezy2gy5xwu/Mexico.JPG?format=car&dag-scope=entity&bytes=0:1048575" -k -v -o mexico-1mb.car -H "Accept: application/vnd.ipld.car"138.199.41.51is a core node. You can also repro with a localhost L1.The request should timeout after 2 minutes with this error message:
transfer closed with outstanding read data remainingFull output
I believe this is related to chunked transfer encoding. curl is waiting for the "terminating chunk" (
0/r/n/r/n) so it knows the response is done.If you remove the
-o mexico-1mb.car, the request ends immediately and the error message changes to* Failed reading the chunked-encoded stream, which confirms it's related to chunked transfer encoding.I know that nginx automatically handles chunked transfer encoding. What I don't know is how that interacts with modules that write the response body. Does nginx add the encoding before this module, after this module, or not at all if a module "claims" the response.
The expected behavior is for the curl response to end cleanly with no error. If you remove
&bytes=0:1048575from the request, it works fine because the module isn't involved.Handling intentional missing terminating chunks.
If lassie is unable to download the full CAR, it will signal a streaming error by not sending the terminating chunk.
The L1 does the same.
Unfortunately, every http hop handler needs to explicitly forward this error signaling, I think the module may have to as well. But i'm not sure, this is something we need to investigate. What does the module see when the L1 closes uncleanly?
Action items
transfer closed with outstanding read data remainingorFailed reading the chunked-encoded stream.The text was updated successfully, but these errors were encountered: