[Streamer | Server] Link header + rel prefetch

[JayPanoz]

Abstract

Some clients e.g. Chrome (or Edge and Firefox, albeit in a different manner, and cURL, etc.) will abort if the response header to a request is too big.

Chrome, for instance, throws the following error: ERR_RESPONSE_HEADERS_TOO_BIG, while cURL will return curl: (27) Avoided giant realloc for header (max is 102400)!

This may impact manifest requests if the list of resources is large and all are rel=prefetch e.g. fixed-layout EPUB file with a lot of fonts.

Technical details

Let’s take a successful request first:

As you can see, the number of requests can escalate very quickly (1300+ of the total number of requests are font resources) so that may already slow down rendering, cf. the following waterfall.

When failing, you’ll get an error in Chrome, 0 bytes in Firefox, etc.

This is the corresponding waterfall

The limit I can find for Chrome is 256 K, cf. chromium source

If the body start is not found within this range of the response, the transaction will fail with ERR_RESPONSE_HEADERS_TOO_BIG.
Note: |kMaxHeaderBufSize| should be a multiple of |kHeaderBufInitialSize|.
static const int kMaxHeaderBufSize = kHeaderBufInitialSize * 64; // 256K

But I’m not sure about other clients since I focused on browsers.

There should probably be a limit/best practice so that prefetch (via Link header) doesn’t put a burden on the client.

cc @HadrienGardeur and @danielweck

[danielweck]

See also: #97