Support for streaming body via UNSIGNED_PAYLOAD

[ghost]

Scenario:

Sending large post with byte array using ReadableStream for performance reasons.

Expected Behavior:

ReadableStream is sent to S3 via a streaming read, offering improved performance.

Actual Behavior:

Receive "body must be a string, ArrayBuffer or ArrayBufferView, unless you include the X-Amz-Content-Sha256 header"

Steps to reproduce:

  let { readable, writable } = new TransformStream();
  const arr = new Uint8Array([ 102,111,111,98,97,114 ])
  writeArrayToStream(arr, writable);
  const signed = await aws.sign(url, { method: 'PUT', headers, body: readable});
  const resp = await fetch(signed);
  return resp

Details:

Problem 1 - Cloning the request
In the signing flow, the request is cloned (https://github.com/mhart/aws4fetch/blob/master/src/main.js#L89) for signing, including the body. This is probably strike #1 for using ReadableStream, ideally there is no cloning of a large request.

Problem 2 -

In the Canonical String generation, there is a call to get the hexBodyHash (https://github.com/mhart/aws4fetch/blob/master/src/main.js#L310).
In the hexBodyHash method, it checks that the body is of type string or has a byteLength (like in an ArrayBuffer I think). A readable stream meets neither of those cases, so it throws the Error.

Proposed solution:

Add an option for unsigned payloads (see https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-header-based-auth.html) by replacing the hexBodyHash with the literal string UNSIGNED-PAYLOAD. I didn't dig too far into this, but it looks like it may be an option on the node version of this (https://github.com/mhart/aws4/blob/master/aws4.js#L244)?

[mhart]

You can specify your own X-Amz-Content-Sha256 header if you know the body hash already – otherwise you'll need to read the entire stream to calculate the hash

[ghost]

Yes, I saw that. I was hoping to use Content-Md5 instead (I happen to have those handy) for validating the payload is received but use an UNSIGNED-PAYLOAD.