Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in reduced schedule CSV file #259

Open
dt-woods opened this issue Oct 9, 2024 · 2 comments
Open

Error in reduced schedule CSV file #259

dt-woods opened this issue Oct 9, 2024 · 2 comments
Assignees
Labels
in v2 Issues marked for inclusion in version 2

Comments

@dt-woods
Copy link
Collaborator

dt-woods commented Oct 9, 2024

In the read_eia923_fuel_receipts method in coal_upstream.py, Page 5 of the EIA923 Excel workbook is saved to CSV. The header columns include a new line character "\n" for the following coal min columns:

  • Coalmine\nType
  • Coalmine\nState
  • Coalmine\nCounty
  • Coalmine\nMsha Id

When written to CSV, these headers are all truncated to "Coalmine," dropping the context after the newline. This results in a CSV file with four columns all of the same name and causes errors with merging.

eia_fuel_receipts_df.to_csv(csv_path, index=False)

To fix, consider running the data frame through the _clean_columns method before writing to CSV.

A symptom of this is a 'KeyError' on key 'fuel_group', accessed in generate_upstream_coal_map from the data frame returned by read_eia_fuel_receipts.

@dt-woods dt-woods self-assigned this Oct 9, 2024
@dt-woods
Copy link
Collaborator Author

dt-woods commented Oct 9, 2024

The worksheet:
Screenshot 2024-10-09 at 12 56 49

And the reduced CSV:
Screenshot 2024-10-09 at 12 57 03

dt-woods added a commit to KeyLogicLCA/ElectricityLCI that referenced this issue Oct 9, 2024
dt-woods added a commit to KeyLogicLCA/ElectricityLCI that referenced this issue Oct 9, 2024
@dt-woods
Copy link
Collaborator Author

Note: to implement this fix on your machine, you need to delete any CSV files in the f923_YEAR folders in your data directory.

@dt-woods dt-woods added the in v2 Issues marked for inclusion in version 2 label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in v2 Issues marked for inclusion in version 2
Projects
None yet
Development

No branches or pull requests

1 participant