BREAKING: Disabling automatic response decompression

[ianbotsf]

An upcoming release of the AWS SDK for Kotlin will disable the automatic decompression of response bodies when using the OkHttp engine.

Release date

This feature will ship with the v1.1.0 release planned for 3/19/2024.

What's changing

When using the default HTTP engine OkHttp, the underlying HTTP client will no longer automatically decompress response bodies when they are Gzip compressed. Previously OkHttp would transparently decompress response bodies when the Content-Encoding: gzip header was present. (See OkHttp's documentation on calls for more detail.) No other HTTP engine automatically decompresses response bodies.

This may manifest when retrieving an object from S3 that was uploaded with a Content-Encoding: gzip header. For instance, if compressed and uncompressed data exist like this:

val uncompressed = ByteArray(1024) { it.mod(Byte.MAX_VALUE) }
println("Uncompressed length: ${uncompressed.size}") // Uncompressed length: 1024

val compressed = ByteArrayOutputStream()
    .also { GZIPOutputStream(it, true).apply {
        write(uncompressed)
        flush()
        close()
    } }
    .toByteArray()
println("Compressed length: ${compressed.size}") // Compressed length: 164 (may vary depending on JVM version)

And the compressed data is uploaded to S3:

s3.putObject {
    bucket = "<some-bucket>"
    key = "myCompressedObj"
    contentEncoding = "gzip"
    body = ByteStream.fromBytes(compressed)
}

Then the object may be retrieved with a getObject call:

s3.getObject(GetObjectRequest {
    bucket = "<some-bucket>"
    key = "myCompressedObj"
}) { response ->
    println("Response length: ${response.body?.toByteArray()?.size}")
    println("Content-Encoding: ${response.contentEncoding}")
}

Before the change the getObject example prints:

Response length: 164
Content-Encoding: null

After the change the getObject example prints:

Response length: 1024
Content-Encoding: gzip

How to migrate

If you are using the default OkHttp engine and do not want to decompress response bodies, then no change is necessary.

If you do want to decompress response bodies, you will need to update your code. Response bodies may be decompressed using JVM's GZIPInputStream and the ByteStream.toInputStream() extension function:

val uncompressedBytes = s3.getObject(GetObjectRequest {
    bucket = "<some-bucket>"
    key = "myCompressedObj"
}) { response ->
    val body = response.body ?: error("no body received")
    when (response.contentEncoding) {
        "gzip" -> GZIPInputStream(body.toInputStream()).readBytes()
        else -> body.toByteArray()
    }
}
// ...make use of `uncompressedBytes`...

To apply automatic decompression to multiple response bodies you may consider writing an interceptor.

Feedback

If you have any questions concerning this change, please feel free to engage with us in this discussion. If you encounter a bug with these changes, please file an issue.