Incorrect header check with Content-Encoding deflate

[Jasper-M]

I am calling a REST API which I cannot disclose, but it returns a response with Content-Encoding: deflate.

scala> requests.get(url = redactedUrl, params = redactedParams)
java.util.zip.ZipException: incorrect header check
  at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
  at java.io.FilterInputStream.read(FilterInputStream.java:107)
  at geny.Internal$.transfer0(Internal.scala:40)
  at geny.Internal$.transfer(Internal.scala:61)
  at geny.Readable.$anonfun$writeBytesTo$2(Writable.scala:64)
  at geny.Readable.$anonfun$writeBytesTo$2$adapted(Writable.scala:64)
  at requests.Requester$$anon$1.processWrappedStream$1(Requester.scala:338)
  at requests.Requester$$anon$1.readBytesThrough(Requester.scala:345)
  at geny.Readable.writeBytesTo(Writable.scala:64)
  at geny.Readable.writeBytesTo$(Writable.scala:64)
  at requests.Requester$$anon$1.writeBytesTo(Requester.scala:165)
  at requests.Requester.apply(Requester.scala:114)
  ... 40 elided

The issue here essentially boils down to this stackoverflow post. The real bug is probably with the REST API that I'm calling, but it should be said that regular browsers, curl, wget and Python Requests can handle this discrepancy. The culprit is most likely some misnamed PHP function.

A workaround is to manually handle decompression. Something akin to this:

def inflateResponse(r: geny.Readable): String = {
  val inflater = new Inflater(true)
  val out = new ByteArrayOutputStream()
  val inflaterOut = new InflaterOutputStream(out, inflater)
  try {
    r.writeBytesTo(inflaterOut)
    out.toString("UTF-8")
  } 
  finally {
    inflaterOut.close()
  }
}

val resp = requests.get(url = redactedUrl, params = redactedParams, autoDecompress = false)
inflateResponse(resp)
// or preferably
val str = requests.get.stream(url = redactedUrl, params = redactedParams, autoDecompress = false)
inflateResponse(str)

Or alternatively set headers to Map("Accept-Encoding" -> "gzip") if the server supports it, and avoid the deflate nonsense.