Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add caching for images #281

Closed
wants to merge 0 commits into from

Conversation

kokoISnoTarget
Copy link
Contributor

This would close #234, but I'm not quite sure if my implementation is any good.
Further, how should the cache be cleared? Delete the directory? Add a Command line option?

@CosmicHorrorDev CosmicHorrorDev self-requested a review March 23, 2024 05:26
@CosmicHorrorDev
Copy link
Collaborator

CosmicHorrorDev commented Mar 23, 2024

Thanks for the pull request! I should have added more info to the original issue since I've slowly been thinking through things, but implementation-wise I was thinking of using an embedded database like redb to act as the cache itself (embedded database as in mostly just persistent btrees. Not full SQL like sqlite)

In regard to detecting stale entries:

  • for local files this should be as simple as storing and then comparing the modified times
  • for URLs this gets a lot more complicated when you start considering Cache-Control stuff

I also think it would be good to have some kind of garbage control over entries in the cache. Either a time to live or some overall size limit would be good (maybe both?)

And then it would likely be good to have some command line option to clear the cache too since we don't want users to have to mess with deleting files on their own


The initial implementation doesn't have to handle all of this of course. That's just my rough vision for the future

@CosmicHorrorDev
Copy link
Collaborator

CosmicHorrorDev commented Mar 23, 2024

Oh and before I forget we should probably also version the data format used for the image data before we want to change the format and run into issues in the future. We currently just LZ4 compress the raw image data, so storing an initial byte for the format before each entry should work. E.g.

#[repr(u8)]
enum Format {
    RawLz4Compressed = 0,
}

// Boilerplate `u8` <-> `Format` `From` impls

@kokoISnoTarget
Copy link
Contributor Author

And then it would likely be good to have some command line option to clear the cache too since we don't want users to have to mess with deleting files on their own

Maybe we should rework the CLI first, so something like this can be added easily later? I could open an issue for that.

@CosmicHorrorDev
Copy link
Collaborator

Sure! That would be a good start. There's all the stuff I mentioned in #131 (comment) that I think would be a good direction to move in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cache downloaded images
2 participants