Explore and process PacFIN bds

[chantelwetzel-noaa]

Data questions that should be explored for PacFIN bds data:

Pre-data cleaning:

• Examine the ageing methods in the data (AGE_METHOD1, AGE_METHOD2, ...) to determine what ageing methods should be removed from the data and retained.
• Examine the prevalence of ages where the age method is NA or blank. Early data looks flagged that a number of ages from Oregon were not present in the early years of the data. Compare the age-at-length of fish without an ageing method to those with to determine if these ages look fundamentally different.
• Are there a number of Oregon samples with SAMPLE_TYPE == "S"? These samples should be kept, if possible, and not removed when applying PacFIN.Utilities::cleanPacFIN().

Post-data cleaning:

• Double-check the removed samples to ensure that all the requested ageing methods and sample types were retained.
• Look at the distribution of length and age samples by gear type before grouping. The model will likely use either a trawl, hook-and-line, pot or trawl and fixed gear fleet structure.
• Examine the distribution of length and age samples by port to identify areas that may have unrepresentative sampling. An example from the 2021 Dover sole assessment: length samples and age samples by port.

Data processing:

• Start from the vignette provided in PacFIN.Utilities that is based on sablefish.

[Ovec8hkin]

@chantelwetzel-noaa regarding the ageing methods cleaning, what should I be looking for to determine where data is suitable for removal?

[chantelwetzel-noaa]

I suggest looking at the codify_age_method function because it provides a useful key to the age method types and how they are coded. Generally, we always keep break and burn ages. Surface and scale ages can be good for some species, but can result in underaged fish. For a species like sablefish, I think we would probably not want to keep surface or scale ages, but they should be looked at. I noticed a number of records without an age method (NA) that we will want to keep if we can, so these should be looked at closely to make sure they are reasonable (e.g., do the age and length combinations make sense by sex). Additionally, we should look at any other age method in the data and decide whether there are other records that should be retained.

I have put together a starting script for process bds data that needs to be improved. I am having some repository issue but hope to get this pushed tomorrow so you can work from this script and modify as appropriate.