Setting Cache-Control header with serve_static

[ross-byrne]

Problem

Serving static files using wisp.serve_static doesn't give the option of setting a cache-control header. This means static files served this way are not cached by the browser.

I noticed this while serving my static content using the following:

use <- wisp.serve_static(req, under: "/static", from: ctx.asset_path)

It seems this code is responsible:

case simplifile.is_file(path) {
  Ok(True) ->
    response.new(200)
    |> response.set_header("content-type", content_type)
    |> response.set_body(File(path))
  _ -> handler()
}

Possible Solution

I don't have any strong opinions on the best solution. I doubt just throwing in an extra header here is a good idea as it would change the behavior.

Maybe we could add a new function that can take some kind of config?

For context my use case is, I'm serving a Svelte SPA from the backend. It would be nice to have the js files cached without having to re-implement my own version of wisp.serve_static :)

Happy to help with implementation if this is something you'd like to add.

Thanks!

[lpil]

Thank you! I think this would be great to have, though I'm not entirely sure what the API and behaviour might be. How long would the cache be valid for? What about etags? Should the programmer be able to configure this? etc.

[ross-byrne]

I'm not entirely sure what the API and behaviour might be

Yeah same honestly. My guess is that there would be too much nuance for there to be one good default. I'd like it to cache but others might not depending on the situation. So I'd lean towards having some way to set it.

Maybe making it less about caching specifically and more about being able to set response headers, could be good? I'm not sure if that level of flexibility is useful for others?

[ross-byrne]

I get that this could also be a non issue if most people just use a CDN. So I'm happy to just implement my own version of this for now.

[lpil]

I'm sure there is a good default out there! Maybe we can look at what other frameworks do and copy them.

While using a CDN is common I think there's a lot of value in supporting non-cloud deployments too, so let's figure that out.

[ross-byrne]

OK leave it with me for now. I had a quick look but I think I need to do some more reading on what the different options are.

It seems the two Go based web frameworks I checked, Echo and Chi both leave it up to the developer to set the headers. By just wrapping the handler with another handler to set the response headers.

I looked at Ruby on Rails and they seem to set it for you, but they have a lot of options for caching. So I'd have to dig a bit more to find out what it's doing.

Either way, interesting!

[ross-byrne]

@lpil so I've done a bit more reading on the topic and I'm curious about what your take on this is.

The quick summary is: HTML files should either not be cached or have a private, very short lived cache eg. 5 or 10 minutes.

Cache-Control: max-age:300, private

JS and CSS should ideally have versioned names to aid cache invalidation or "cache busting". If versioned, you can set the max-age to a year (1 year is the max a cache can be set).

Cache-Control: max-age=31536000, immutable

Un-versioned files are trickier. Those would need ETags to avoid issues with cached resources going stale. ETags is how Rails handles caching assets with it's asset pipeline but I believe it's off by default. While ETags would be a good feature, the file hashing required would add overhead. So I think that would be a separate thing a user opts into.

My Thoughts

Caching seems to be very context specific but if we're going to set a default value, the cache should probably be private to avoid accidentally leaking sensitive information (see: MDN). We could go with something like: Cache-Control: private, max-age=604800 which is valid for 7 days.

My gut feeling is, I'd like to be able to set the response header and/or a config to control how this works. So maybe it's better to leave serve_static as it is for now and look at adding an additional function that can take a config? Or something that returns the response so the user can customise it as needed (similar to Go based frameworks). Curious what your thoughts are on this and your appetite for either changing the existing api or adding new functions.

I'll be looking at tackling caching in my own backend soon, maybe next week. So I can prototype some things and share them here if that would be useful? Maybe we can start narrowing down what the API could look like?

Links: 1 2

[lpil]

That sounds sensible, but it would be good to be able to support the common options people would want. Did you look at what other frameworks do?

[ross-byrne]

Yeah sorry, I should have compiled a bit of a list. So I looked through some popular web frameworks and this is how they handle caching:

• Express.js - uses etags but has default max-age set to 0. So files aren't cached. Takes options to make it easy to set. link
• Laravel - No caching by default
• Rails - Does asset fingerprinting by hashing the file contents - link
• Echo - No caching by default, needs to be configured
• Chi - No caching by default, needs to be configured
• Next.js - Doesn't cache static assets by default. Needs to be configured. link
• Hono.js - Doesn't cache by default. Has middleware with config. link
• Spring - Sets cache for 1 year by default. Allows for configuration

Unless I've missed an obvious one, it seems most don't actually do caching by default but have middleware for configuring it.

[ross-byrne]

Something like the express api would be nice

var options = {
  dotfiles: 'ignore',
  etag: false,
  extensions: ['htm', 'html'],
  index: false,
  maxAge: '1d',
  redirect: false,
  setHeaders: function (res, path, stat) {
    res.set('x-timestamp', Date.now())
  }
}

app.use(express.static('public', options))