MRFSS lengths #5
Replies: 10 comments 5 replies
-
|
Testing this out. I received the message and have pinged on John B for a response. I will update with his response once received. |
Beta Was this translation helpful? Give feedback.
-
|
I have had this account since CA scorpionfish. Below is the request provided to Jason Edwards in explaining what is needed in the wholesale revisions to RecFIN. Currently the CRFS collects fork lengths, thus LNGTH is preferable. There are conversions from total length in the LNGTH column that are decimals. In some instances there are decimals for both LNGTH and T_LEN or omitted conversions, so some follow up was requested from Jason.
Chantel also made the following request. Remaining requests are here. https://docs.google.com/spreadsheets/d/1VEdWcL6tC_V54q4bmn1l6HD7fuemEUcv-UMCWDNzEbc/edit#gid=0 |
Beta Was this translation helpful? Give feedback.
-
|
@JohnBudrick It is great that this issues is included on the list of needed revisions for RecFIN. Assuming that this will not be able to be completed in time for next year's assessments, which column within the MRFSS data do you recommend people using? Did MRFSS consistently record fork length similar to CRFSS and hence the LNGTH column is the actual measurement? As we noted in last week's meeting the fact that both the T_LEN and LNGTH can have multiple decimal points it becomes challenging to understand which column should be used depending upon the species and the period of time being examined. If a species does not have a forked tail (e.g., copper rockfish) does CRFSS measure the fish length from nose to the center of the fanned tail and then record that as a fork length (LNGTH)? MRFSS? |
Beta Was this translation helpful? Give feedback.
-
|
If you want to get really into the weeds on all the weird tail types there are diagrams on how CRFS measures them starting on page 4-16 of the sampler manual. This is what the manual says "Measuring Various Types of Fish Most species are measured from the most anterior tip of the longest jaw (mouth closed) or end of snout, whichever is terminal, to the posterior tip of the tail at its center line. This procedure is the same whether the tail forks in (e.g., mackerels) or protrudes out (e.g., flounders)" |
Beta Was this translation helpful? Give feedback.
-
|
Hi Chantel,
Below is an email from Jason Edwards of PSMFC regarding the flags for lengths. Total length and fork length appear to have been recorded at different times throughout MRFSS, while it was consistently fork length for CRFS. The LNGTH column in MRFSS is the fork length consistent with CRFS era. Where there are decimals for LNGTH there has been a conversion from total length presumably using data from 1993 where both appear to be collected allowing conversions. Given the error in length measurements due to rigamortis, jaw issue etc, for species with a flat edge to their caudal fin rather than a fork or heterocercal tail may not show much of a difference in any case.
Thanks,
John
Thank you again for bringing this to my attention. I have also CCed Mike on this email, as we discussed this during our RecFIN call yesterday. While I don't have a complete explanation for you, I may be able to provide a little more information regarding the lengths. As I mentioned, the MRFSS data that is currently available for download in the zip format came from a bulk upload from the old system. However, the public query tool I used to extract the data did not include some columns that may help explain the length values with decimals. I have attached the Type 3 data from 1980-2003 and associated variable definitions. This data was previously password protected in the Survey Operations section of the old RecFIN site.
The data file contains additional columns, including a LEN_FLAG column with values of: null, "t", "0", "c", "?", and "f." Unfortunately, the only documentation I have found for this column reads "c=calc. from wgt t=total lngth calc." so I can only make assumptions regarding the other codes at this time.
c - It does appear from the description and from looking at the data that when LEN_FLAG = c, then LNGTH is calculated (contains decimals).
null - When LEN_FLAG is null there is a curious pattern, as the data appear to contain no decimals from 1980 - 1996. In 1997-1998, decimals are present, and then disappear again in 1999 - 2003
t - When LEN_FLAG = t, decimals do not appear in either the LNGTH or T_LEN columns. The LEN_FLAG description is vague, so I can't confirm if LNGTH is being calculated from T_LEN or vice versa. However, T_LEN is defined as 'Measured Total Length' so I would assume FL is being calculated in this case. In many records, the LNGTH = T_LEN. I haven't investigated fully, but these may be instances for fish without forks? The 't' value occurs from 1987-1993 only.
0 - When LEN_FLAG = 0, the data is a mix of decimals and integer values. These values are only present from 1980-1989.
f - When LEN_FLAG = f, there are no decimal values in the data, and this only occurs in 1993. Again, the description is vague so I am not sure of the direction of a potential calculation. However, when the flag is equal to 'f', both LNGTH and T_LEN are integer values.
? - I wouldn't be too concerned with these records as there are only 4 of them in the time series and the LNGTH values are null. However, the T_LEN values are populated for these records.
Again, I apologize that I can't be more useful here. Like you, Mike and I tended to agree that decimal values are definitely calculated and should be excluded if you do not want to include calculated values. If you need more information or explanation, hopefully between you, Mike, and myself, we can come to a determination. I will continue to keep looking for documentation on this stuff. I will also add this to the growing list of items that need to be resolved with the MRFSS data cleansing tasks. Also, I will work on getting the full data sets with these extra columns available on the RecFIN site for download.
From: Chantel Wetzel ***@***.***>
Sent: Wednesday, December 7, 2022 8:39 AM
To: pfmc-assessments/california-data ***@***.***>
Cc: Budrick, ***@***.*** ***@***.***>; Mention ***@***.***>
Subject: Re: [pfmc-assessments/california-data] MRFSS lengths (Discussion #5)
You don't often get email from ***@***.******@***.***>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
WARNING: This message is from an external source. Verify the sender and exercise caution when clicking links or opening attachments.
@JohnBudrick<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJohnBudrick&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C9d8c2a2e9c7248d4d09008dad87189e2%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638060279405482675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=tzncjWidqHDzhiaADSCPjtADTH7zRnXupxgTSReeS2I%3D&reserved=0> It is great that this issues is included on the list of needed revisions for RecFIN. Assuming that this will not be able to be completed in time for next year's assessments, which column within the MRFSS data do you recommend people using? Did MRFSS consistently record fork length similar to CRFSS and hence the LNGTH column is the actual measurement? As we noted in last week's meeting the fact that both the T_LEN and LNGTH can have multiple decimal points it becomes challenging to understand which column should be used depending upon the species and the period of time being examined.
If a species does not have a forked tail (e.g., copper rockfish) does CRFSS measure the fish length from nose to the center of the fanned tail and then record that as a fork length (LNGTH)? MRFSS?
-
Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpfmc-assessments%2Fcalifornia-data%2Fdiscussions%2F5%23discussioncomment-4334970&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C9d8c2a2e9c7248d4d09008dad87189e2%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638060279405482675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kjhniC4M6hbNGpjd0qV4QPR6WaFKoVX8P1qZWdbPhSU%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFN7JX5EEXUI5WDDTLVM2XLWMC4SDANCNFSM6AAAAAASUKQQWQ&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C9d8c2a2e9c7248d4d09008dad87189e2%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638060279405482675%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ZQouLh7VQ8k1iwfJdoqV5KgQlV6qX2v%2BqCZaTilWIK8%3D&reserved=0>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the response. Looking at the MRFSS data I have for copper rockfish most years have a length reading that does not contain decimal points. However, this is not the case for data in 1980 where both T_LEN and LNGTH contain decimal length recordings and the LEN_FLAG column is 0. I don't know if this is an issue in 1980 data for other species (e.g., canary rockfish and black rockfish) or if this is just an issue for copper rockfish. |
Beta Was this translation helpful? Give feedback.
-
|
@JohnBudrick, @JuliaCoates and @chantelwetzel-noaa To follow up on this issue for copper rockfish, I applied Echeverria and Lenarz's Fork Length to Total length equation to the MRFSS column LNGTH, (0.629+(1.01*LNGTH)). This gets you back to what is pretty much a whole number, i.e., LNGTH = 230.07029703 resulting in Total_length = 233.0000000003. This particular record has a matching record for the same day, and port in Deb's database with a copper rockfish with total length = 233; and all of the others from July 1997 seems to match up as well. I believe this conversion is off because Echeverria and Lenarz used PINCHED tail total length in all of the conversions, which if you were going from a fork length to total length, you would be overestimating the total length, and with an unpinched total length to fork length you'd underestimate the fork length. One proposed resolution is to replace the MRFSS lengths in the copper assessment with Deb's lengths, without a conversion from total to fork length given that coppers don't have a forked tail. |
Beta Was this translation helpful? Give feedback.
-
|
I looked into this for canary and think it is happening there too. LNGTH has decimals in 1997-1998 (and before 1993). When I transform LNGTH to a total length using the Echeverria and Lenarz I get a lot of nearly near whole numbers too. When comparing those to the Deb TL for those years, there are many that are the same. I cant confirm that the day and months are the same because my MRFSS data's DATE1 field doesn't align with the YEAR column, and so I think it was converted oddly and so dont trust the month and day. However, there are enough lines of evidence to suggest some Deb data in 1997-1998 are somehow occuring in the MRFSS data for canary as well. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Melissa,
I believe in 1993 MRFSS collected both fork and total length to facilitate a conversion used in bridging from total length collected previously and fork length collected subsequently. They tend to have a pretty flat caudal fin unless they are pinched etc, which is not typical of total length measurements by MRFSS that I understand to just be the fish layed flat on the board and caudal fin spread. The difference might be within the error for sampling in any case with rigor mortis and jaw issues or inter observer error at just 3 mm, so they may not be pinched and the conversion may be from their own collections. With that degree of error, I would even consider just using them as they are. Looks like a 3 mm change rather than cm, which seems reasonable for fork vs. total length. Not sure that degree of difference would result from a pinched tail. If it were cm, then maybe.
I don't think you need to throw out the MRFSS lengths, but rather could use the 1993 data with both integers rather than decimals to create a conversion as I believe they did. If you believe them to be duplicates somehow between the data bases (which I don't believe to be the case having never heard of Debs data being in MRFSS) as seems implied, that should be looked into further and addressed.
Let me know if I missed something.
Thanks,
John
From: Melissa Monk ***@***.***>
Sent: Thursday, March 9, 2023 4:56 PM
To: pfmc-assessments/california-data ***@***.***>
Cc: Budrick, ***@***.*** ***@***.***>; Mention ***@***.***>
Subject: Re: [pfmc-assessments/california-data] MRFSS lengths (Discussion #5)
WARNING: This message is from an external source. Verify the sender and exercise caution when clicking links or opening attachments.
@JohnBudrick<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJohnBudrick&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C72d81fbb5cf440e8865608db21023f85%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638140065773506262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=imPhW2zKOGKAJXboaW5TEvLw8%2BRkBpEEGCdyW8eGR8Y%3D&reserved=0> and @chantelwetzel-noaa<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fchantelwetzel-noaa&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C72d81fbb5cf440e8865608db21023f85%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638140065773506262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yw2kTVtvG3lUbVcHlxUZYRhcJPp8kbRkEtNG8L2J5bs%3D&reserved=0> To follow up on this issue for copper rockfish, I applied Echeverria and Lenarz's Fork Length to Total length equation to the MRFSS column LNGTH, (0.629+(1.01*LNGTH)). This gets you back to what is pretty much a whole number, i.e., LNGTH = 230.07029703 resulting in Total_length = 233.0000000003. This particular record has a matching record for the same day, and port in Deb's database with a copper rockfish with total length = 233; and all of the others from July 1997 seems to match up as well. I have a hard time envisioning a copper rockfish with a 3 cm fork in their tail. I believe this conversion is off because Echeverria and Lenarz used PINCHED tail total length in all of the conversions, which if you were going from a fork length to total length, you would be overestimating the total length, and with an unpinched total length to fork length you'd underestimate the fork length. Our proposed resolution is to replace the MRFSS lengths in the copper assessment with Deb's lengths, without a conversion from total to fork length given that coppers don't have a forked tail. Let us know if that sounds reasonable to CDFW staff.
-
Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpfmc-assessments%2Fcalifornia-data%2Fdiscussions%2F5%23discussioncomment-5261453&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C72d81fbb5cf440e8865608db21023f85%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638140065773506262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uQ0UY921d4clGf3QU7RqsB8AfgTTgSkwKTy4eZb5NtE%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFN7JX3MBYUZ6F26PMMZOGTW3J323ANCNFSM6AAAAAASUKQQWQ&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C72d81fbb5cf440e8865608db21023f85%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638140065773506262%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hwisQjVGtt%2FtDlxDgh3HM0qOW5rs%2Feg%2FaAzsLBFbn3Q%3D&reserved=0>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
|
Hi John, After some additional querying, I believe the decimal lengths from 1997 and 1997 in MRFSS are in fact from Deb's onboard trips. I just uploaded an Excel file with commit cee7ca7 that shows 806 lengths from MRFSS and Deb's data from 1997 and 1998 where the date and port match, as well as the length if the MRFSS Length's are back-calculated using the conversion in the last comment. There are no matches in Deb's data when there is a non-decimal value in MRFSS. Melissa |
Beta Was this translation helpful? Give feedback.
-
|
Seems something to discuss with PSMFC as the commingling of data is not ideal. The RecTech might need to discuss its inclusion/exclusion in the efforts by Jason Edwards. |
Beta Was this translation helpful? Give feedback.
-
|
For canary rockfish, the differences between LNGTH and T_LEN are more pronounced, ranging from 15-30 mm within the MRFSS data. LNGTHS have decimals before 1993 whereas T_LEN mostly have decimals after 1993, with 1993 having overlap. Oddly, when I look at Echeverria and Lenarz's relationship: tlen = (-4.107 +(1.07*LNGTH)) the converted tlen matches T_LEN for all years (except 1993) regardless of whether T_LEN has decimals or not. Converting the other way does not match. This tells me that decimals do not indicate whether LNGTH is measured. The only entry for LEN_FLAG is 0, mostly when T_LEN does not have decimals. The short of this is that it is confusing. @JohnBudrick I plan to move forward with LNGTH as if it is measured. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Brian,
I matched date and port after pulling the MRFSS date information from the
ID numbers. I don't think we need to do it for every species since the
matches were nearly exact for coppers.
Melissa
…On Mon, Apr 3, 2023 at 10:08 AM Brian Langseth ***@***.***> wrote:
I looked into this for canary and think it is happening there too. LNGTH
has decimals in 1997-1998 (and before 1993). When I transform LNGTH to a
total length using the Echeverria and Lenarz I get a lot of nearly near
whole numbers too. When comparing those to the Deb TL for those years,
there are many that are the same. I cant confirm that the day and months
are the same because my MRFSS data's DATE1 field doesn't align with the
YEAR column, and so I think it was converted oddly and so dont trust the
month and day. However, there are enough lines of evidence to suggest some
Deb data in 1997-1998 are somehow occuring in the MRFSS data for canary as
well.
—
Reply to this email directly, view it on GitHub
<#5 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAWRXM2RC42YEZOFKO3D4U3W7L7YRANCNFSM6AAAAAASUKQQWQ>
.
You are receiving this because you authored the thread.Message ID:
<pfmc-assessments/california-data/repo-discussions/5/comments/5512462@
github.com>
--
*Melissa Monk, Ph.D. (she/her)*
Fisheries Ecology Division
National Marine Fisheries Service
National Oceanic and Atmospheric Administration
110 McAllister Way Santa Cruz, CA 95060
Phone: 831-420-3950
|
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Hi Brian,
The length conversions etc. is one of the items on Jason Edwards list to address as the patterns observed in the data were not always consistent. Below are a few potential considerations that might resolve some of your concerns. We can talk more about it at 11.
Thanks,
John
From: Brian Langseth ***@***.***>
Sent: Thursday, March 30, 2023 3:43 PM
To: pfmc-assessments/california-data ***@***.***>
Cc: Budrick, ***@***.*** ***@***.***>; Mention ***@***.***>
Subject: Re: [pfmc-assessments/california-data] MRFSS lengths (Discussion #5)
WARNING: This message is from an external source. Verify the sender and exercise caution when clicking links or opening attachments.
For canary rockfish, the differences between LNGTH and T_LEN are more pronounced, ranging from 15-30 mm within the MRFSS data. LNGTHS have decimals before 1993 whereas T_LEN mostly have decimals after 1993, with 1993 having overlap.
This is as expected since total lengths were collected up until 1989 to my understanding, and they collected both in 1993 to create a conversion.
Oddly, when I look at Echeverria and Lenarz's relationship: tlen = (-4.107 +(1.07*LNGTH)) the converted tlen matches T_LEN for all years (except 1993) regardless of whether T_LEN has decimals or not. Converting the other way does not match. This tells me that decimals do not indicate whether LNGTH is measured.
After 1993 when they had lengths from both to create conversions, perhaps they used their data rather than Lenars's after 1993 when fork lengths were collected.
The only entry for LEN_FLAG is 0, mostly when T_LEN does not have decimals. The short of this is that it is confusing.
Were these for a particular time frame?
@JohnBudrick<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FJohnBudrick&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C88724439fa9140efe0b908db31702177%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638158129912833131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=X38xjyaGFdXOF1ltIu%2FaNd%2BxgHngeHFnuxdlYRkFR8g%3D&reserved=0> I plan to move forward with LNGTH as if it is measured.
-
Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpfmc-assessments%2Fcalifornia-data%2Fdiscussions%2F5%23discussioncomment-5482819&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C88724439fa9140efe0b908db31702177%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638158129912833131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Ku50BEIhyLWaFmkx%2BfUxNAm5269MDG8dVYfnEFfBBjU%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFN7JX66H26FPJRF6P2NVKDW6YD7VANCNFSM6AAAAAASUKQQWQ&data=05%7C01%7CJohn.Budrick%40wildlife.ca.gov%7C88724439fa9140efe0b908db31702177%7C4b633c25efbf40069f1507442ba7aa0b%7C0%7C0%7C638158129912833131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=IbQN3ZrMzCmHgBnCxRxVyVJf8FEh2swhoafWi%2BPImuI%3D&reserved=0>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
Q (CRW-NWFSC) @JPhillipsCDFW There are two columns in the MRFSS data where length is recorded, LNGTH and T_LEN, is the T_LEN column total length while LNGTH the fork length (noting that these values can be different for fish without a fork in their tail)? Is one column “better” to use than the other? In each column there are lengths to multiple decimal places implying that they may be imputed values, how should these records be treated?
#2
James - tagged you since you have an account as a test (John B does not yet).
Beta Was this translation helpful? Give feedback.
All reactions