Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to apply retro to user specified window of years #12

Open
brianlangseth-NOAA opened this issue May 20, 2022 · 4 comments
Open
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@brianlangseth-NOAA
Copy link

To fully diagnose potential sources of retrospective patterns it would be add functionality to specify a set window of years to do a retrospective analysis on. Similar to this would be to automate the exclusion of entire datasets as well, which could serve both as a sensitivity, but also to explore fixes of possible retrospective patterns.

@brianlangseth-NOAA brianlangseth-NOAA added the enhancement New feature or request label May 20, 2022
@brianlangseth-NOAA brianlangseth-NOAA self-assigned this May 20, 2022
@kellijohnson-NOAA
Copy link
Contributor

@brianlangseth-NOAA you can do this by using the model_settings argument of retro_wrapper().
https://github.com/nwfsc-assess/nwfscDiag/blob/55f49fb16b540ed2752482dd46774d79791d9d83/R/retro_wrapper.R#L68
I am not sure if you can run it multiple times in the same directory because the naming of the directory is done automatically which would put the results in the same folder.
Regarding automating exclusion of entire datasets, I have moved this to a new issue. Keeping issues focused on a single topic helps future users search them and helps keep individual threads to a manageable size.

@kellijohnson-NOAA kellijohnson-NOAA added documentation Improvements or additions to documentation and removed enhancement New feature or request labels May 23, 2022
@kellijohnson-NOAA
Copy link
Contributor

Perhaps @brianlangseth-NOAA you could suggest changes to the documentation, maybe more links to r4ss::SS_doretro() to make it clear what options users might want to change?

@brianlangseth-NOAA
Copy link
Author

brianlangseth-NOAA commented May 24, 2022

@kellijohnson-NOAA You are right in that years can be specified within the function, upon which all years after that point are removed. Thank you for reminding me of that functionality.

There remains a lack of capability to automatically remove only a window of years, and not all years after that window, as well as to remove years at the back end of the time series (i.e. a reverse retrospective). Neither of these are a true retrospective run but rather diagnostics for exploring effects of data on our models.

The reverse retrospective can be done by iteratively changing

  1. the #_StartYr in the .dat file, and
  2. updating the first year of main recr_devs and the recdev_early_start in the .ctl file, if needed.

Utilizing the set up from retro_wrapper could make this automated and easy to explore a range of starting years in one fell swoop.

The window approach would be hard because r4ss::Do_retro() changes the starter file retro year, which is coded within SS3 itself
https://github.com/r4ss/r4ss/blob/82abd14df22bc6ad39fb9b59e371d2e16944a805/R/SS_doRetro.R#L140
I dont see a way to adjust with the current retrospective setting. A hacky way would be to add a negative value for any data in the window of years, and then advance that, which may be tricky to automate. Certainly not as easy as adjusting just a few lines.

@iantaylor-NOAA
Copy link
Contributor

I think that if we choose to get better at modifying our data files through r4ss::SS_readdat() and r4ss::SS_writedat() where removing data for any subset of years/fleets/data-types could be pretty easy to script, e.g. https://github.com/nwfsc-assess/lingcod/blob/main/models/remove_ages_south.R#L40-L42.

If excluding a window of years is a common task, we could write a function to loop across all data types to exclude a window of years to make it all the easier within the data file workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

3 participants