Potential uses for complete EIA 860m data

[arengel]

This is the discussion that @zaneselvans suggested I create here, now exactly one month ago. I delayed doing this both because I was busy and I didn't feel like the case I had for this data was particularly compelling. I'm re-raising the idea because I now see a couple potential uses for this data.

The basic idea

Take every instance of EIA 860m (from July 2015 through the present) and combine them together to create a dataset of existing, planned, canceled, and retired generator data at a monthly time resolution.

Use cases

I freely admit that it is not immediately obvious why you should want to do this—at least it wasn't to me. Other than being voluminous and repetitive, not much seems notable about the resulting data. Having thought about it more though, I think there are a few useful things we could derive from these data.

1. Date of retirement announcement

As I understand the annual 860 data, it records when an existing generator plans to retire, but it does not record when that planned retirement was announced, or at least, when that planned retirement was reported to EIA. By finding the first month in which a given generator provides a planned retirement date, we can see precisely when that retirement was announced. Of course, the meaningfulness of this date depends on if reporting a new planned retirement to EIA is generally coincident with announcing the retirement, or some other consistent stage of the retirement planning process.

2. Date of proposal announcement

Conceptually similar to the previous idea but here we would look for when a planned generator first shows up on the list of planned generators. Similar caveats about meaningfulness apply here as well. I'd also include here the same idea but for when a generator changes from planned to canceled/postponed.

3. Mid-year fuel changes

Without actually looking at the data it's impossible to know if this is actually visible in 860m data but we might be able to see generators switching primary fuels mid-year which could help in better interpreting and allocating generation data from 923 to generator and CEMS to fuel.

4. Other stuff with dates

Looking at a complete series of 860m data we'd be able to see how planned retirement and operating dates changed over time. We'd also be able to see if generators consistently began operation or retired when they reported they planned to. I put this down the list because I'm not sure what we'd do with this information.

A variation

The first two use cases are, to me at least, the most interesting and compelling. They also don't actually require that PUDL produce a monthly generator data output. The monthly 860m data could be analyzed to determine the dates that generators were announced to be built or retired, and those dates could be inserted into the annual generator data.

[zaneselvans]

My impression from conversations with Brendan Pierpont (recently departed from Sierra Club) is that it's common for there to be a significant lag between when a retirement is announced / decided on / mandated and when the retirement date is reported to EIA. IIRC Sierra Club maintained it's own database of planned retirements, compiled from news stories and regulatory filings, because it was more up to date than what was available from EIA. But maybe there's some official regulatory threshold beyond which they need to get before reporting the date to EIA? I don't know why this lag would exist, or if there's even any rhyme and reason to it. But yes, I guess this would be an interesting timeline to be able to construct, maybe for the purpose of inferring the impacts of various campaigns or legislative changes or economic events?

Do we have a sense of how many mid-year primary fuel changes there are? I suspect this would be a pretty marginal improvement in accuracy overall unless they're mostly generator conversions from one fuel to another, rather than a generator that has two or more fuels that are used a similar amount, and so which one is "primary" changes from month to month or year to year based on availability, price, etc.

If we've already got all the metadata mapped for the many, many months of released data (which I think y'all did) then I don't think it would be much work to add the concatenated monthly table containing all of that information, which could then feed into various derived outputs, one of which would be our current use case for the eia860m (to update the generators table) and another of which would be to compile the history of reporting changes. Do you have code that already does that compilation?