Merge dev into main for 2023-09-08

[zaneselvans]

• Replace no longer available SQLAlchemy VisitableType with plain type
• Require SQLAlchemy>2 to force compatibility testing.
• Update for compatibility with pandas 2.0
• Temporarily depend on ferc_xbrl_extractor pandas-2.0 branch
• Merge non-dependency changes from sqlalchemy-2.0 branch
• Require pandas 2.0 RC for testing.
• Specify resolution of datetime64 types.
• Add missed changes from merge to setup.py
• Use actual pandas 2.0.0 now that it's out.
• Update to ferc-xbrl-extractor 0.8.2
• Use pandas extras to declare some dependencies
• Update recordlinkage to version compatible with pandas 2.0
• the road to hell is paved with dtypes
• Use tuple instead of list of SQL params.
• Set SQLAlchemy view w/ metadata error to XFAIL.
• Ignore DeprecationWarning coming up from imported packages.
• Update fix_eia_na() regex to avoid pandas df.replace() vectorization bug.
• Remove deprecated infer_date_format flag.
• Allow a mix of Zenodo sandbox & production DOIs
• Integrate some Pydantic validation into ZenodoFetcher
• Update allowed tox versions.
• [pre-commit.ci] pre-commit autoupdate
• Update docs to reflect availability of 2022 EPA CEMS data.
• Bump max allowed Tox version.
• Stop importing urllib3 Retry from deprecated location
• Create a ZenodoDoiSettings Pydantic BaseSettings class.
• Update Zenodo DOI test to work better with new ZenodoDoiSettings
• Fix some pandas 2 incompatibilities and highlight others.
• Fix more minor pandas 2 compatibility issues.
• Fix another datetime dtype merge mismatch.
• More permissive type check in convert_col_to_datetime
• Apply types before setting non-unique index.
• Update pyarrow requirement from <12.1,>=5 to >=5,<13.1
• Update sphinxcontrib-bibtex requirement from <2.6,>=2.4 to >=2.4,<2.7
• Use numeric_only=True in rolling average mean()
• Replace complex convert_cols_dtypes with simple apply_pudl_dtypes
• Fix ambiguous index/column name overlap in plants_small_ferc1.
• Fix small pandas-2.0 incompatibilities in glue tests.
• Update docs/dev/datastore.rst
• Remove deprecated pudl_datastore --pudl_in option and unused get_resource_key() method
• Add references to CEMS and Dagster in the annual updates docs
• Update description for small plants table
• Add HI to CEMS processing
• Update release notes
• Update annual updates doc based on Ella's feedback in the PR
• Add more tweaks to annual updates table
• Add github action usage notebook and use large runner for tox-pytest workflow
• Remove github token from notebook
• Run ci-integration on large runner
• Simplify code linting & formating; use ruff
• Use ruff to ensure use of pathlib
• Apply python syntax simplification autofixes.
• Add NumPy specific static analysis checks from ruff.
• Use ruff to simplify & check for bad return values
• Add several additional ruff linter checks.
• Add pandas-specific ruff checks.
• Update pre-commit requirement from <3.4,>=2.9 to >=2.9,<3.5
• Update gcsfs requirement from <2023.6.1,>=2021.7 to >=2021.7,<2023.9.1
• Update dask requirement from <2023.8.2,>=2021.8 to >=2021.8,<2023.9.1
• Update build requirement from <0.11,>=0.10 to >=0.10,<1.1
• Update tox requirement from <4.11,>=4 to >=4,<4.12
• Update tox requirement from <4.11,>=4 to >=4,<4.12
• Bump tox version
• [pre-commit.ci] pre-commit autoupdate
• Remove py const reference from table description because it doesn't render in Datasette
• Fix typos and clarify _core vs. core assets in Dagster
• reword description a little
• Slight rewording in order to get the tests to run
• Add generator to the retirement_year / _month columns for the monthly data so it matches the annual data and doesn't get nulled out
• Add release notes
• Update mambaforge image & google cloud sdk, use python 3.11
• Ensure required versions of urllib3, pandas, numpy are installed by conda
• Pin to pyarrow<13
• Add a docker build test on pull requests

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov]

Codecov Report

Patch coverage: 91.3% and no project coverage change.

Comparison is base (3b983b9) 88.5% compared to head (da51425) 88.5%.
Report is 5 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #2840   +/-   ##
=====================================
  Coverage   88.5%   88.5%           
=====================================
  Files         90      90           
  Lines      10126   10134    +8     
=====================================
+ Hits        8964    8978   +14     
+ Misses      1162    1156    -6     

Files Changed	Coverage Δ
src/pudl/cli/etl.py	56.5% <ø> (-1.0%)	⬇️
src/pudl/convert/epacems_to_parquet.py	64.7% <ø> (ø)
src/pudl/metadata/codes.py	100.0% <ø> (ø)
src/pudl/metadata/dfs.py	100.0% <ø> (ø)
src/pudl/metadata/fields.py	100.0% <ø> (ø)
src/pudl/metadata/resources/ferc1.py	100.0% <ø> (ø)
src/pudl/output/eia.py	59.0% <ø> (ø)
src/pudl/resources.py	100.0% <ø> (ø)
src/pudl/transform/eia861.py	97.4% <ø> (ø)
src/pudl/transform/epacems.py	91.1% <ø> (ø)
... and 42 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.