Feature branch: Rename core + output assets to match new naming protocols

[e-belfer]

See #2812 for issue description and progress tracking. This will collect naming PRs, which will all need to be merged in together to avoid major headaches for downstream users.

The PR includes renamed assets for:

• core EIA 923, EIA 860 and harvested EIA tables
• core FERC Form 1 tables
• core EIA 861 tables
• core FERC Form 714 tables
• coding tables
• glue/association tables
• EPA CEMS parquet files and core assets

It also does a few other things:

• fixes mixed usage of annual and yearly in core assets
• reorganizes modules to match _core and core structure.

Not converted:

• entity_types_eia , which is currently not loaded into the PUDL DB.
• assets from the XBRL explosion (currently just clean_xbrl_metadata_json)

Steps for merging

1. Rename all of the tables and merge docs PR into this branch
2. Run a full build with validation for this branch
3. Once the build passes and this branch is ready to be merged into dev, create a PR for dev -> main.
4. Create a tag and push a tag called pre-name-change. Question: Should we do a final full software release or just let people install from git tab?
5. Squash merge this branch into dev
6. Once the schedule dev nightly build passes, merge dev -> main and tag a release using our desired release naming convention: vYYYY.MM.DD
7. Help known users migrate!

PR Checklist

• Merge the most recent version of the branch you are merging into (probably dev).
• All CI checks are passing. Run tests locally to debug failures
• Make sure you've included good docstrings.
• For major data coverage & analysis changes, run data validation tests
• Update the release notes and reference reference the PR and related issues.
• Do your own explanatory review of the PR to help the reviewer understand what's going on and identify issues preemptively.

[bendnorman]

Thanks @e-belfer! I feel pretty good about all of the proposed asset names in the naming convention sheet except for the assets currently in the glue group:

• We decided to use assn for the asset type for all glue/association tables.
• I'm not sure what the datasource portion of the asset name should be. EIA EPA crosswalk assets come from EPA. Most of the other current glue assets are hand-crafted excel sheets we create so maybe pudl is a reasonable datasource for these assets? Would it be awkward to have a module named pudl.core.pudl to contain these assets?
• plants_pudl and utilities_pudl feel more like entity tables assets because they only contain pudl ids and names. If we follow the naming convention the new names would be core_pudl__entity_plants_pudl and core_pudl__entity_utilities_pudl. These names are pretty similar to core_eia__entity_utilities and core_eia__entity_plants. Do you think this will confuse users? Is there another way we can name these?

[e-belfer]

* I'm not sure what the `datasource` portion of the asset name should be. EIA EPA crosswalk assets come from EPA. Most of the other current glue assets are hand-crafted excel sheets we create so maybe `pudl` is a reasonable datasource for these assets? Would it be awkward to have a module named `pudl.core.pudl` to contain these assets?

I think the alternative to pudl.core.pudl would be using a name like "manual" or "hand" but it wouldn't well characterize all the machine learning outputs that will likely come to live here. Alternately we could use "cat" to point to Catalyst as the origin of this data, in the event that we start integrating glue tables from other non-agency sources.

* `plants_pudl` and `utilities_pudl` feel more like entity tables assets because they only contain pudl ids and names. If we follow the naming convention the new names would be `core_pudl__entity_plants_pudl` and `core_pudl__entity_utilities_pudl`. These names are pretty similar to `core_eia__entity_utilities` and `core_eia__entity_plants`. Do you think this will confuse users? Is there another way we can name these?

Is there a reason to keep the _pudl other than legacy compatibility? I think they'd be clear mirrors of the EIA tables but containing PUDL-only info in this case.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[e-belfer]

Updated decisions: keep "pudl" as datasource, and leave duplicated PUDL as is.

[codecov]

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (98492e6) 92.7% compared to head (9f3d293) 92.6%.
Report is 12 commits behind head on dev.

Files	Patch %	Lines
src/pudl/analysis/eia_ferc1_train.py	84.2%	3 Missing ⚠️
src/pudl/etl/cli.py	33.3%	2 Missing ⚠️
src/pudl/metadata/classes.py	90.0%	1 Missing ⚠️
src/pudl/output/pudltabl.py	80.0%	1 Missing ⚠️
src/pudl/transform/eia860.py	66.7%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             dev   #2818     +/-   ##
=======================================
- Coverage   92.7%   92.6%   -0.0%     
=======================================
  Files        134     134             
  Lines      12597   12611     +14     
=======================================
+ Hits       11672   11682     +10     
- Misses       925     929      +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bendnorman]

Looks good to me! Just a couple of assets in notebooks that might need to be renamed.

I'm thinking we should use this as the main feature branch for the core/output rename. What do you think @e-belfer?

[bendnorman]

Should utilities_entity_eia be core_eia__entity_utilities and utilities_eia be core_pudl__assn_utilities_eia?

[bendnorman]

Hmm I'll take a look and update this branch with the changes from dev again.

[bendnorman]

It looks like retained_earnings_appropriations_ferc1 is a "disabled" FERC table. Should we rename it or just remove it from the metadata?

[zaneselvans]

This file has been deleted from dev. Do we know why it's showing up here now?

[bendnorman]

Yeah that is strange. I'll look into it.

[zaneselvans]

This file has been deleted on dev

[zaneselvans]

This file has been deleted on dev

[zaneselvans]

If nothing inside this module is going to be called MCOE maybe we should change the name of the module?

[bendnorman]

I agree! I decided to punt updating references to MCOE everwhere because it's a large change. #2910

[bendnorman]

The CI and full build passed... should we merge this thing in?

[zaneselvans]

I think merging it in and then using it ourselves for a while to iron out the kinks before we cut a new release with the new names is probably a good idea. I'm sure there will be issues and inconsistencies that we only discover when using it day-to-day.

[bendnorman]

Should we point datasette at v2023.12.01 data so datasette continues to have the old names until we do a full release with the new names?

[zaneselvans]

You mean just stop updating it?

[zaneselvans]

I dunno. The docs say that Datasette reflects the nightly builds, so we would need to change that. And the latest tag on Read The Docs points at dev so by default the documentation that anyone sees after the merge will reflect the new names.

If we can get on an actual monthly release schedule, then having the public Kaggle and Datasette outputs track the stable release rather than nightly doesn't sound like a bad idea.

[bendnorman]

Having datasette and docs point at stable seems like the ideal setup. I guess we should leave it on dev for now.

[zaneselvans]

I'm sure we will find things to fix and improve as we use it, but I think that's only going to happen through actual usage.