Estimating suppressed EIA MECS data

[matthewlchambers]

It appears that we are attempting to estimate values for cells in the EIA MECS FBA with 0 value, as we do for the QCEW. However, in the original EIA MECS table, there is a specific symbol for suppression, and quite a few cells with an actual 0 value. So, I think we need to revise how the FBA is created so that cells that are actually suppressed are estimated appropriately but cells that are actually 0 are left as 0. I see 3 possible approaches:

1. Estimate suppressed industries during FBA creation. Not ideal, since as I understand it the premise of FBAs is that they reformat but don't modify the data.
2. Delete suppressed industries so that they can be disaggregated over using the QCEW (or other secondary source) just like industries that are left out of the MECS. This could work, if the method we would use for estimating suppressed industries is already similar to what we do for industries that are left out.
3. Decide on a non-0 value to identify suppressed industries. This allows for more sophisticated estimation techniques to be applied by the user if desired.

Thoughts?

[matthewlchambers]

To give a sense of the counts involved: for Coal in 2018 at the national level there are roughly 40 actual 0 values, 20 '*' values (representing <0.5), and 2 'D' values (actually suppressed). So treating 0 as if it is suppressed introduces significant error. Also, we should have a different solution for '*' values than for 'D' values.

[WesIngwersen]

Thanks @matthewlchambers for investigating. This is a generally very important topic that is very common in automating all read of statistical data and we need to make sure that flowsa is handling this in the best possible way.
Regarding this particular source we have a working relationship with some MECS experts and we need to consult them on any approach(es) that we propose

[matthewlchambers]

Makes sense. I'll start thinking about how to handle it code wise, allowing for flexibility in the actual estimation approach. Do you think it would it be possible to talk with the MECS experts sometime in the next month? I'd love to be involved in that conversation if possible.

[WesIngwersen]

Yes, sure

[matthewlchambers]

As a philosophical question, where do we think estimation of suppressed cells should be occurring? Should it be part of the FBA creation, or part of the use of the FBA in creating an FBS?

As mentioned above, It seems that the goal with FBAs is that they reformat but don't modify the data. On the other hand, we can't ensure that the FlowAmount column is numeric until after estimating the suppressed cells, and FBAs being formatted correctly for use in creating FBSs is also a goal.

[catherinebirney]

@matthewlchambers Thanks for catching this issue - we definitely need to address incorrectly updating true 0 values with estimates. We should not estimate the suppressed data as part of FBA creation due to reasons you've identified.

Could we use np.nan to identify suppressed data? Those work in numeric columns

[matthewlchambers]

The problem with using np.nan to identify suppressed cells is that there may be multiple types of suppression requiring different estimation techniques. For instance, the EIA MECS data contains * symbols for cells with values < 0.5 and D for cells suppressed to avoid disclosing data on individual firms. These should obviously be treated differently.

[matthewlchambers]

After some research, I think the best solution is going to be adding a text column that gives the string value for any FlowAmount cell that is non-numeric. Then, as part of one of the existing function sockets (such as clean_fba) or in a new one, we have a function that uses the information from that column to estimate values for suppressed cells while leaving actual 0s alone, then drops the extra column. Thoughts?

[catherinebirney]

Yes that can work - we can make the new column an optional requirement for FBAs

[hpliddell]

Hi folks! @WesIngwersen asked me to pop into this discussion with comments on how our team at Energetics handles data suppression in MECS - for example, in the Manufacturing Energy and Carbon Footprints (https://www.energy.gov/eere/amo/manufacturing-energy-and-carbon-footprints-2018-mecs). We generally do not treat the suppressed values as zero; as you've noted, the suppressed values can be large and treating as zero can introduce some significant error. Luckily, the MECS datasets are generally manageable in size and most of these values can be reasonably estimated by deduction. This is how we have specifically handled this issue in our current and past work:

1. “-“ and “* values (estimate is less than 0.5 TBtu): These data values are set to 0 (in most cases). It is understood that the small fraction of energy use omitted by setting these values to 0 contributes to the identified rounding error. In certain cases, we adjust certain values by a value less than 0.5 to balance out the sums in the All Manufacturing row.
2. “W” or “D” values (withheld to avoid disclosing data for individual establishments) and “Q” values (withheld due to a Relative Standard Error>50%): These data values are calculated by subtraction (looking across a row and calculating the value by simple subtraction if it is the only unknown), estimated based on the proportional usage from 2014 or prior MECS data, or estimated based upon the difference between the sums of the three digit NAICS sector codes and all manufacturing (NAICS 31-33) total across a column. In certain cases, estimates can be made in the regional tables to ascertain an unknown in the national table. Rarely, these are actually assumed to be zero (usually a case of looking at previous MECS data).

Hope this helps. Please feel free to flag us with questions or if we can further assist.
Heather