DO NOT MERGE YET - HTTP Caching API

[harmony7]

This PR tracks the implementation of the HTTP caching API.

Implements #991

[harmony7]

Starting by committing proposed TypeScript types for the API surface. The design and behavior are very similar to what's in the Rust SDK, described in the developer documentation: Customizing cache interaction with the backend

[harmony7]

Examples

Inject headers before sending

Sometimes it is useful to perform modifications to the incoming Request before invoking the origin through the readthrough cache, for example to add an authorization header. If this is expensive to generate, then it makes sense to add it only if the request would make it to the backend. Specify onBeforeSend on CacheOverride to define a before-send callback function, an operation to be performed just before the readthrough cache would invoke the backend.

// Example: inject headers before sending.
const request = event.request;
const response = await fetch(request, {
  backend: 'example',
  cacheOverride: new CacheOverride('override', {
    onBeforeSend(req) {
      // Assume buildAuthorizationHeader() may be expensive, so
      // we only want to call this when the request would actually reach the backend.
      req.headers.set('Authorization', buildAuthorizationHeader());
    },
  }),
});

Customize caching based on content type

Sometimes it is useful to modify caching policy based on the backend response. Specify onAfterSend on CacheOverride to define an after-send callback function, an operation that runs only when the readthrough cache has received a response from the backend, before it is (potentially) stored into the cache.

The CandidateResponse object passed to the callback represents the response from the backend and contains interfaces to read and manipulate headers and cache policy. It intentionally does not allow reading or writing directory the response body (more on that later).

This example shows usages that utilize these members of CandidateResponse.

• the ttl property, which is used to override the Time to Live (TTL) of the object in the cache
• the setUncacheable() property, which is used to specify that this object is not to be stored in the cache

// Example: customizing caching based on content type.
const request = event.request;
const response = await fetch(request, {
  backend: 'example',
  cacheOverride: new CacheOverride('override', {
    onAfterSend(resp) {
      const contentType = resp.headers.get('Content-Type') ?? '';
      switch(true) {
        case contentType.startsWith('image/'):
          resp.ttl = 67;
          break;
        case contentType === 'text/html':
          resp.ttl = 321;
          break;
        case contentType === 'application/json':
          // setUncacheable() with no param (default false) marks this object as uncacheable
          // without disabling request collapsing
          resp.setUncacheable();
          break;
        default:
          resp.ttl = 2;
      }
    },
  }),
});

Creating a hit-for-pass object

By specifying true when calling the setUncacheable() method of CandidateResponse, you mark the request as hit-for-pass, which is a marker to disable request collapsing until a cacheable response is returned.

// Example: creating a hit-for-pass object.
const request = event.request;
const response = await fetch(request, {
  backend: 'example',
  cacheOverride: new CacheOverride('override', {
    onAfterSend(resp) {
      if (resp.headers.has('my-private-header')) {
        // setUncacheable() with true param marks this object as uncacheable
        // and marks it as hit-for-pass, resulting in disabling request collapsing
        resp.setUncacheable(true);
      }
    },
  }),
});

Manipulating the response body that is stored to the cache

In an after-send callback, optionally set the bodyTransform property of the CandidateResponse object to an instance of TransformStream to define a body-transform to be applied to the backend response body before it is stored into the cache.

Employing TransformStream allows working with streamed chunks of the backend body, rather than necessarily reading it entirely into memory (though the code example below does not take advantage of this, as it attempts to parse JSON, which requires the entire body to be present).

The transformation is specified by setting a property (resp.bodyTransform =) rather than directly working with the body during the after-send callback function. This is because not every response contains a fresh body. Specifically, 304 Not Modified responses, which are used to revalidate a stale cached response, are valuable precisely because they do not retransmit the body; in this case, the backend and (if specified) your after-send callback function update the headers and cache policy of the existing response object "in-place", without applying the body-transform or changing the cached response body.

This design enables the readthrough cache to internally manage the complexities of revalidation, allowing the developer to provide a single code path without needing to think about revalidation at all.

// Example: expanding a template before caching.
const request = event.request;
const response = await fetch(request, {
  backend: 'example',
  cacheOverride: new CacheOverride('override', {
    onAfterSend(resp) {
      resp.headers.set('Content-Type', 'text/html');
      resp.bodyTransform = new TransformStream({
        bytes: null,
        start() {
          this.bytes = new Uint8Array(0);
        },
        transform(chunk) {
          // The ideal transform would transform bytes as it goes over the wire,
          // but for a transformation whose input is JSON, we need to buffer the
          // bytes to memory, because we need the whole body before we can
          // deserialize it.
          const newBytes = new Uint8Array(this.bytes.length + chunk.length);
          newBytes.set(this.bytes, 0);
          newBytes.set(chunk, this.bytes.length);
          this.bytes = newBytes;
        },
        flush(controller) {
          const str = new TextDecoder().decode(this.bytes);
          // jsonToHtml applies a template to generate HTML from JSON
          const html = jsonToHtml(str);
          controller.enqueue(new TextEncoder().encode(html));
        },
      });
    },
  }),
});

// The resulting cached object will have a body that is HTML
// despite that the backend returned a JSON response:
return response;

Notes

• The HTTP readthrough cache interface performs a number of transformations related to range collapsing, client revalidation, and backend revalidation automatically. To gain an understanding of why this is useful, and to make best use of the interface, it is worth reading about these automatic request transformations.

• The new onBeforeSend() and onAfterSend() callback functions either return nothing or return a Promise that resolves to nothing. This should be doable since fetch() is a function that returns a Promise. Additionally, if these callback functions throw an exception or the Promise returned from them rejects, then the exception, or the rejected value of the Promise, is used to reject the Promise returned from fetch(), and nothing is stored to the cache.

  • In a request-collapsing case, one request out of the pooled requests will be selected to perform the before-send callback function → backend fetch → after-send callback function sequence, while the other pooled requests will wait. If the sequence fails at any point, then any response received from the backend is discarded, the Promise returned by fetch() in just that instance will reject with the error value, and another request out of the pooled requests will be selected to attempt the before-send callback function → backend fetch → after-send callback function sequence again.

• When using the new before-send and after-send callback functions, as well as the body-transform, there are some additional considerations that help you understand that behavior.

• The Fetch API defines a cache property on RequestInit that specifies "cache modes". However, these cache modes call for behavior that is not possible under the existing set of host calls, so at this time let's not touch that property, and maybe revisit them if there is real demand or use-cases for them.

  cache value	check cache?	fresh	stale	uncached	store? (*1)	notes
  "default"	yes	return cached	revalidate (*2)	fetch	yes	this is the exiting behavior in Compute
  "no-store"	no				no	readthrough cache has no way to skip checking the cache
  "reload"	no				yes	readthrough cache has no way to skip checking the cache
  "no-cache"	yes	revalidate	revalidate	fetch	yes	readthrough cache has no way to force a revalidation for fresh objects
  "force-cache"	yes	return cached	return cached	fetch	yes	readthrough cache has no way to get the existing stale value
  "only-if-cache"	yes	return cached	return cached	error	no	readthrough cache has no way to get the existing stale value

  *1 - if allowed by object's cache policy specified by backend or set in after-send callback
  *2 - if the object is in the stale-while-revalidate window, the stale object is returned immediately, and "revalidate and store" are done async