Skip to content

Commit

Permalink
Merge pull request #29 from nflverse/caching-fix
Browse files Browse the repository at this point in the history
Resolve #28
  • Loading branch information
john-b-edwards authored Sep 25, 2024
2 parents 41bf4cc + 4fac1e9 commit cd3fbc9
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 9 deletions.
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "NFLData"
uuid = "38e18452-fdda-4cae-b91e-088906595f57"
authors = ["John Edwards"]
version = "1.1.3"
version = "1.1.4"

[deps]
Artifacts = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
Expand Down
22 changes: 14 additions & 8 deletions src/getdata.jl
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
module getdata
using Scratch: @get_scratch!, delete_scratch!
using Scratch
using Preferences
using DataFrames
using Parquet2
using HTTP
using Downloads
using CSV
using Dates
import Base: UUID

export cache_data_pref, clear_cache, from_url

PKG_UUID = UUID("38e18452-fdda-4cae-b91e-088906595f57")

"""
cache_data_pref(pref::Bool)
Expand All @@ -25,8 +28,10 @@ end
Clear the NFLData.jl scratch space.
"""
function clear_cache()
delete_scratch!("NFLData_cache")
global download_cache = @get_scratch!("NFLData_cache");
path = scratch_dir(string(PKG_UUID), "NFLData_cache/")
rm(path; force=true, recursive=true)
mkpath(path)
global download_cache = path;
end

const cache_data = @load_preference("cache", true)
Expand All @@ -39,18 +44,19 @@ function __init__()
printstyled("To disable this caching, run `cache_data_pref(false)` and restart Julia.\n", color = :blue)
printstyled("To clear the cache, run `clear_cache()`.\n", color = :blue) =#
# initialize cache
tmp_cache = @get_scratch!("NFLData_cache")
path = scratch_dir(string(PKG_UUID), "NFLData_cache/");
mkpath(path)
# check for what files are in the cache and how old the oldest one is
if length(readdir(tmp_cache)) > 0
oldest_file = unix2datetime(minimum([mtime(joinpath(tmp_cache, file)) for file in readdir(tmp_cache)]))
if length(readdir(path)) > 0
oldest_file = unix2datetime(minimum([mtime(joinpath(path, file)) for file in readdir(path)]))
# check how old the oldest file in the cache is
time_since_last_cache = round(now() - oldest_file, Hour(1))
# if it's been more than 24 hours, clear the cache
if time_since_last_cache >= Hour(24)
delete_scratch!("NFLData_cache")
rm(path; force=true, recursive=true)
end
end
global download_cache = @get_scratch!("NFLData_cache")
global download_cache = path;
end

"Helper function for reading a .parquet file to a DataFrame (while ensuring the connection closes after the file is read)."
Expand Down

2 comments on commit cd3fbc9

@john-b-edwards
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator register

Release notes:

Bug fixes

  • Resolved issue where the cache would not clear correctly, either w/ 24 hours or with manual clear_cache() call.

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/115989

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v1.1.4 -m "<description of version>" cd3fbc9f465cc1d3db18ea1432515444b80add56
git push origin v1.1.4

Please sign in to comment.