Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameterize key query fields in riem_measures() #48

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
ee51f9e
Parameterize 'data' field in riem_measures()
JElchison Dec 16, 2024
e95c3ec
Add comment linking field documentation
JElchison Dec 16, 2024
2f6a39c
Parameterize 'elev' field in riem_measures()
JElchison Dec 16, 2024
695875d
Parameterize 'latlon' field in riem_measures()
JElchison Dec 16, 2024
ca212f2
Parameterize 'report_type' field in riem_measures(); Skip HFMETAR by …
JElchison Dec 16, 2024
2a5f27e
Choose a safer default date_start
JElchison Dec 16, 2024
3c358b4
Speficy nometa = no, since parsing counts on column names
JElchison Dec 16, 2024
60d8c28
Update self-documentation for riem_measures()
JElchison Dec 16, 2024
d5a7f4c
Update test fixtures files since default endpoint query changed
JElchison Dec 16, 2024
8737611
Document
JElchison Dec 16, 2024
1c8a136
Preserve order of riem_measures() positional arguments; Remove defaul…
JElchison Dec 17, 2024
a75b5b9
Specify the now-required station srgument in test-measures.R
JElchison Dec 17, 2024
3b082f0
Correct copy/paste error
JElchison Dec 17, 2024
12aeea5
Specify the now-required date_start argument in test-measures.R
JElchison Dec 17, 2024
540fb46
Add call to rlang::check_dots_empty() in riem_measures()
JElchison Dec 17, 2024
fe048f4
Add better validation for new riem_measures() args
JElchison Dec 17, 2024
b14e790
Specify the now-required station srgument in test-measures.R
JElchison Dec 17, 2024
3b8bc96
Correct order of args in documentation
JElchison Dec 17, 2024
ae205a3
refactor: use `rlang::is_character()`
maelle Dec 19, 2024
2f0c7f9
refactor: make report_type's input more readable
maelle Dec 19, 2024
b3c5dd4
test: add test for new parameters
maelle Dec 19, 2024
115a373
Address lintr warnings
JElchison Dec 19, 2024
045b229
Add new tests; Address lintr warnings
JElchison Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,4 @@ Suggests:
xts
Config/testthat/edition: 3
Encoding: UTF-8
RoxygenNote: 7.3.1
RoxygenNote: 7.3.2
23 changes: 18 additions & 5 deletions R/measures.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,12 @@
#'
#'
#' @param station station ID, see riem_stations()
#' @param date_start date of start of the desired data, e.g. "2000-01-01"
#' @param data The data columns to return. The available options are: all, tmpf, dwpf, relh, drct, sknt, p01i, alti, mslp, vsby, gust, skyc1, skyc2, skyc3, skyc4, skyl1, skyl2, skyl3, skyl4, wxcodes, ice_accretion_1hr, ice_accretion_3hr, ice_accretion_6hr, peak_wind_gust, peak_wind_drct, peak_wind_time, feel, metar, snowdepth # nolint: line_length_linter
#' @param elev If set to 'yes', the elevation (m) of the station will be included in the output. # nolint: line_length_linter
maelle marked this conversation as resolved.
Show resolved Hide resolved
#' @param latlon If set to 'yes', the latitude and longitude of the station will be included in the output. # nolint: line_length_linter
maelle marked this conversation as resolved.
Show resolved Hide resolved
#' @param date_start date of start of the desired data, e.g. "2016-01-01"
#' @param date_end date of end of the desired data, e.g. "2016-04-22"
#' @param report_type The report type to query. The available options are: 1 (HFMETAR), 3 (Routine), 4 (Specials). # nolint: line_length_linter
#'
#' @return a data.frame (tibble tibble) with measures,
#' the number of columns can vary from station to station,
Expand Down Expand Up @@ -64,8 +68,13 @@
#' }
riem_measures <- function(
station = "VOHY",
date_start = "2014-01-01",
maelle marked this conversation as resolved.
Show resolved Hide resolved
date_end = as.character(Sys.Date())) {
data = "all",
maelle marked this conversation as resolved.
Show resolved Hide resolved
elev = "no",
latlon = "yes",
JElchison marked this conversation as resolved.
Show resolved Hide resolved
date_start = "2024-01-01",
date_end = as.character(Sys.Date()),
# skip HFMETAR by default
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why, by the way?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First, the IEM download page defaults to routine and specials, but not HFMETAR. They describe their rationale:

When in doubt, pick both routine and specials.

It seems to me that IEM is specifying their desired default behavior. Tangential, I've also found this IEM news article.

Second, there is a bandwidth/size penalty (9.1x) and time penalty (6.4x) when selecting HFMETAR. It seems like an unsafe default.

I compared two single data points:

  1. Link, 2.1 MB in 597ms
  2. Link, 19.1MB in 3820ms

I'm sure our friends at IEM would appreciate the bandwidth savings from users who don't really need HFMETAR.

Third, as a token consumer of this data, my specific application is hampered by the extra time required to download HFMETAR data ... but I'd be happy to set a parameter if I actually needed it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, it makes sense!

report_type = "3,4") {
date_start <- format_and_check_date(date_start, "date_start")
date_end <- format_and_check_date(date_end, "date_end")
if (date_end < date_start) {
Expand All @@ -74,17 +83,21 @@ riem_measures <- function(

resp <- perform_riem_request(
path = "cgi-bin/request/asos.py/", # nolint: nonportable_path_linter
# query fields per https://mesonet.agron.iastate.edu/cgi-bin/request/asos.py?help # nolint: line_length_linter
query = list(
station = station,
data = "all",
data = data,
elev = elev,
latlon = latlon,
year1 = lubridate::year(date_start),
month1 = lubridate::month(date_start),
day1 = lubridate::day(date_start),
year2 = lubridate::year(date_end),
month2 = lubridate::month(date_end),
day2 = lubridate::day(date_end),
report_type = report_type,
format = "tdf",
latlon = "yes",
nometa = "no",
tz = "UTC"
)
)
Expand Down
18 changes: 15 additions & 3 deletions man/riem_measures.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading