-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathscrapsheet-2WFE.qmd
268 lines (239 loc) · 22.3 KB
/
scrapsheet-2WFE.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
# 2WFE Scrapsheet
- **todo**
- see [{etwfe}]{style="color: #990000"} vignette for woolridge method
- Re-read printed results and compare with corollary descriptions. Try to figure out what the fuck he's saying
- I think the two results might conflict in the union/lwage example. What do you do when that happens
- Finish paper examples and see what his conclusions are
- Run on data of first example (newpapers) and see if it's also close
- add results of pkg to each example
- they don't match the paper, but they're close. Could be just rounding errors
- Organize this first section as notes on the Heiss/Jakiela article
- Choose what's going into note in Papers notebook and what stays here in the TWFE section
- Maybe email the bastard that made that pkg
- [THREAD](https://twitter.com/paulgp/status/1534169803388293120) +paper, biased estimates for multi-treatment with saturated group adjustment variables
- Two-way fixed effects estimators with heterogeneous treatment effects
- author: Chaisemartin
- paper: [arxiv](https://arxiv.org/pdf/1803.08807.pdf)
- data: [website](https://www.openicpsr.org/openicpsr/project/118363/version/V2/view?path=/openicpsr/118363/fcr:versions/V2&type=project)
- conditional on all treatments, the absolute value of the expectation of βfe divided by the standard deviation of the weights is equal to the minimal value of the standard deviation of the ATEs across the treated (g, t) cells under which the average treatment effect on the treated (ATT) may actually have the opposite sign than that coefficient. One can estimate that ratio to assess the robustness of the two-way FE coefficient. If that ratio is close to 0, that coefficient and the ATT can be of opposite signs even under a small and plausible amount of treatment effect heterogeneity. In that case, treatment effect heterogeneity would be a serious concern for the validity of that coefficient. On the contrary, if that ratio is very large, that coefficient and the ATT can only be of opposite signs under a very large and implausible amount of treatment effect heterogeneity
- first difference regression
- βfd is ATE from period i-1 to i
- The outcome variable Y has been first differenced
- With group and period control variables
- assumptions
- assume that Ng,t/Ng,t−1 does not vary across g
- all groups experience the same growth of their number of observations from t − 1 to t, a requirement that is for instance satisfied when the data is a balanced panel
- Notation
- For every (g, t) ∈ {1, ..., G} × {1, ..., T}, let
- **Ng,t** denote the number of observations in **group g** at **period t**
- For every y (i, g, t) ∈ {1, ..., Ng,t}×{1, ..., G}×{1, ..., T}, let
- **Di,g,t** denote treatment status of **observation/unit i** in group g at period t
- **(Yi,g,t(0), Yi,g,t(1))** denote the potential outcomes without and with treatment of observation i in group g at period t
- The outcome of observation i in group g and period t is **Yi,g,t = Yi,g,t(Di,g,t**).
- **Dg,t** denotes the average treatment in group g at period t
- **Yg,t(0)** denotes average potential outcomes without treatment in group g at period t.
- **Yg,t(1)** denotes average potential outcomes with treatment in group g at period t.
- **Yg,t**denotes the average observed outcome in group g at period t.
- **Ng,.** = Σ(t = 1 to T) Ng,t denote the total number of observations in group g
- **N.,t** = Σ(g =1 to G) Ng,t denote the total number of observations in period t
- **N1** is the number of treated units
- **Xg,.** = Σ(t = 1 to T) (Ng,t/Ng,.)Xg,t denote the average value of Xg,t in group g
- Similar for **X.,t** and **Xg,t**
- **∆TR** average treatment effect (**ATE**) across all treated units
- **δTR** = E\[∆TR\]denotes the expectation of that parameter, referred to as the average treatment on the treated (**ATT)**
- **δTR** and **βfe** are both equal to the expectation of a weighted average of the treated cells' ∆g,ts
- **∆g,t** the **ATE** in cell (g, t).
- **wg,t** weight formula on pg 7
- **Pk** = Σ(i≥k) N(i)/N1
- **N(i)** is the number of observations of treated cell i
- k ∈ {1, ..., n} where n is the number of treated cells
- **Sk** = Σ (i≥k) (N(i)/N1)w(i)
- **Tk** = Σ (i≥k) (N(i)/N1)w2(i)
- Robustness Test
- [{TwoWayFEWeights}]{style="color: #990000"}
- see page 3, penultimate paragraph of Frenchy paper
- \|βfe\| divided by the standard deviation of the weights
- βfe denotes the coefficient of Dg,t
- σfe described on pg 11 see 1.
- Corollary 1a: σfe = \|βfe\| / σ(**w**)
- If close to zero, treatment effect heterogeneity would be a serious concern for the validity of βfe
- says, βfe and true ATE can be of opposite signs even under a small and plausible amount of treatment effect heterogeneity
- Re: formula
- σ(**w**) = sum over treated cells( \[(Ng,t / N1)\*(w(g,t) - 1)2\]1/2 )
- w(g,t) = εg,t / sum over treated cells((Ng,t / N1)\* εg,t)
- Ng,t / N1 would be the proportion of the proportion of total units of all groups of a cell to treated units of a cell
- So this should be \> 1
- Guess you'd look at the residuals of the fe fit for values of εg,t
- This may be extractable from fixest pkg
- Corollary 1b: σfe = \|βfe\| / \[Ts + Ss2/(1 − Ps)\]1/2
- only defined if at least one of the weights is strictly negative
- same interpretation as corollary 1a, if close to 0
- Example: Newspapers and Politics
- regress the on
- state-year fixed effects and on the
- first difference of the number of newspapers available in that county
- first-difference of the turnout rate in county g between election years t − 1 and t
- Steps (example from paper)
- calc βfe
- −0.0011 (s.e.= 0.0011)
- find number of negative and positive weights attached to βfe
- 6,212 are strictly positive, 4,161 are strictly negative
- Find sd of negative weights
- negative weights sum to -0.53.
- σfe = 3 × 10−4
- hmm 0.0011/0.53 doesn't equal 0.0003 so that not it.
- Example: wage and unions
- variables
- lwage nr year union
- test_random_weights(educ)
- Stata
- `D.` / `d.`
- first difference
- `2D.` is second difference
- `drop`
- remove variables or observations from the dataset in memory
- `egen` / `generate`
- dplyr::mutate
- "generate" has abbreviations `gen` / `g`
- "egen" works across groups of variables
- `encode` / `decode`
- converts string to numeric and numeric to string respectively
- `forvalues` i=1/5
- for-loop for i = 1 to i = 5
- `quietly` / `qui`
- execute the code without printing the output
- `return` / `r`
- returns the result of a calculation
- `summarize` / `sum`
- summary statistics
- `tabulate` / `tab`
- frequency table
- `xtset` group_var, time_var
- Tells stata that to treat these as panel variables
1. comparing contributions (w/o standardizations)
2. correction factor
3. Wonder about his cv with time series split.+
Notes from Kubinec
- potential outcomes framework
- We use a control group to approximate Yit(0) and a treatment group for Yit(1) and we randomize treatment assignment to get an **average** treatment effect.
- sequential ignorability - sequence of data doesn't matter (i.e. time doesn't matter (so no autocorrelation within units?))
- we need this assumption because we can't randomize treatment/control assignments simultaneously across time points and must randomize sequentially
- Variation - all panel data estimands are combinations of these two basic dimensions of variation (ATEi and ATEt)
- over-time regression coefficient ( ATEt ) - ATE for each case (or unit) across all time points
- Preferred by researchers but can never obtain randomization of the same nature over time as we can with a cross-section
- would require that treatment assignment is independent of the potential outcomes
- can't do that kind of random assignment as it involves moving across time, not just space
- randomization can happen sequentially but is supposed to be simulaneous
- not the same thing as a cross-section randomization because randomization won't happen simultaneously to all observations, introducing potential correlation with time as a confounder if the unit changes over time
- often considered to be better because it is thought that there is less heterogeneity in the same unit observed over time but not necessarily true
- over-time inference could be more faulty than a cross-section, especially if the time periods were quite long, such as a panel with hundreds of years. A cross-section would have less heterogeneity in a given year than the same country compared over 300 years apart.
- over-time inference should be most credible when T is of the shortest duration.
- If we can observed repeated observations of the same unit in and out of treatment status, and those time periods are very close together, then we would have much more reason to believe that the units are comparable and treatment status is ignorable.
- Makes the most sense to calculate an ATEt for each unit, i, and then average those ATEts for all units, i, to get a more precise estimate of ATEt
- i.e. `group_by(unit) %>% summarize(unit_avg = mean(outcome)) %>% ungroup() %>% summarize(ate_t = mean(unit_avg))`
- This calculation requires the assumption of sequential ignorability since we're averaging across time points in order to get an unbiased estimate.
- cross-sectional (between-case) regression coefficient ( ATEi ) - ATE for a cross-section of different units at a given point in time
- treatment randomization is the typical randomization in non-panel data analysis
- i.e. `group_by(time_point) %>% summarize(cross_sec_avg = mean(outcome)) %>% ungroup() %>% summarize(ate_i = mean(cross_sec_avg))`
- what are the criteria for deciding what is a variable we can manipulate as opposed to just a coefficient from a model without any substantive meaning?
- He claims, "it has to have some kind of independent existence in the 'real world.'"
- He's talking about the intercept, α. It's represents a unit doesn't it? If so, it represents a real thing.
From Lecture [slides](https://yiqingxu.org/public/panel/lec1_handout.pdf)
- The 2WFE estimator under staggered adoption is a weighted average of all possible 2x2 DiD estimators that compare timing groups to each other (timing groups are probably the groups that receive treatment at different times)
- The weights on the 2x2 DiDs are proportional to timing group sizes and the variance of the treatment dummy in each pair, which is highest for units treated in the middle of the panel.
- I don't understand what "variance of the treatment dummy in each pair" means
- Assumptions
- variance weighted common trends, VWCT = 0
- generalizes the common trend assumption of DiD to a setting with timing variation
- the average of the difference in counterfactural trends between pairs of groups and different time periods using the weights from the previous decomposition, and captures how differential trends map to bias in the \^β estimate
- captures the fact that different groups might not have the same underlying trend in outcome dynamics, which biases the (really any) DiD estimate.
- heterogeneity - change in treatment effects over time, ∆ATT = 0
- weighted sum of the *change* in treatment effects within each unit's post-period with respect to another unit's treatment timing
- 2WFE estimates can be biased due to:
- presence of time-varying confounders (well-known)
- feedback from past outcome (known, but often ignored)
- heterogeneous treatment effects (often completely ignored)
- "2x2" difference-in-difference design, meaning there are two groups, and treatment occurs at a single point in time. Many difference-in-difference applications instead use many groups, and treatments that are implemented at different times (a "rollout" design). Traditionally these models have been estimated using fixed effects for group and time period, i.e. "two-way" fixed effects. However, this approach with difference-in-difference can heavily bias results if treatment effects differ across groups, and alternate estimators are preferred. See [Goodman-Bacon 2018](https://www.nber.org/papers/w25018) and [Callaway and Sant'anna 2019](https://papers.ssrn.com/sol3/Papers.cfm?abstract_id=3148250).
- In the first period, neither of these groups receive a treatment. So for all intents or purposes, you can assume that they are identical in every single say. In the second period, however, one of the groups receive a treatment (a training program, medicine, or other type of treatment), whereas the other is left "untreated".\
![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/image001.png)
- y10 means average outcome for treatment group (1) at time period 0 (should be rows then cols but whatevs)
- 2 equivalent methods to calculate the DiD Treatment Effect (TE)
- Estimate the treatment effect by comparing the treated-untreated outcomes difference in the post period to the pre-period outcome difference.
```
TT1= (y11-y01) Post-period Treated vs untreated
-(y10-y00) pre -period Treated vs untreated
```
- Uses the "bias-stability" assumption. That whatever factors explain the difference between treated and untreated outcomes after the treatment, are the same before the treatment. (so they can be eliminated)
- Estimate the TE by comparing the outcome change for the treated group across time to the outcome change experienced by the not-treated group.
```
TT2= (y11-y10) Treated Post vs pre
-(y01-y00) Untreated Post vs pre
```
- Uses "parallel-trends" assumption. That if the treated outcome would have experienced a similar and parallel change in outcome as the untreated units experience, then a double difference would also eliminate them.
- Using a linear model
``` r
lm(y ~ Dtreated_unit + Dpost_treatment + Dtreated_unit*Dpost_treatment, data = somedata)
```
- "Dtreated_unit" - indicator for treatment (TR = 1 for treatment group)
- "Dpost_treatment" - indicator for time (T = 1 for when treatment is occurring)
- Coefficient on the "Dtreated_unitTRUE:Dpost_treatmentTRUE" interaction term represents the treatment effect.
- Using two-way fixed effects (TWFE) which is an equivalent formulation to the linear model
``` r
lm(y ~ Dtreat + factor(id) + factor(period), data = somedata)
# or
library(fixest)
feols(y ~ Dtreat | id + period, data = somedata)
```
- The core idea of TWFE is that we can subsume the interaction term of the linear model by adding unit and time fixed effects. A single treatment dummy, "Dtreat," can then be used to capture the effect of treatment directly.
- Where before both indicators ("Dtreated_unit", "Dpost_treatment") held some information about treatment, now only one variable ("Dtreat") does.
- The TWFE shortcut is especially nice for more complicated panel data settings with multiple units and multiple times periods.
- Coefficient on the "Dtreat" represents the treatment effect.
- traditional TWFE model obtains a parameter for TE that is the average of all possible 2x2 designs that could be constructed from the above matrix
- Example Stepped Design ([article](https://friosavila.github.io/playingwithstata/main_didmany.html))\
![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/Screenshot%20(775).png)
- 1 control group: Never-Treat (NT)
- 3 treatment groups (G): 1, 2, 3
- 3 periods: 0, 1, 2, 3
- Groups receive treatment in the blue squares
- Treatment Effect (TE) calculations for Group 1 at Period 1, 2, 3![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/Screenshot%20(770).png)
- Calculation occurs where there's only 1 treated cell in 2x2 square ("good design")
- "Bad design" (i.e more that 1 cell that's treated) may identify the treatment effect if the treatment effect is homogenous across groups
- If a never treated group (NT) were not available, it is also possible, and valid given limited information, to use units from other "treated" groups as controls, as long as they have not yet been treated![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/image008.png)
- TE for Groups 2 (left, center) and 3 (right)![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/Screenshot%20(771).png)
- Test the parallel trends assumption for Group 2![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/Screenshot%20(774).png)
- Use a square of cells that don't contain a treatment square
- Calculate TE for this square. Should be (significantly close to ) 0.
- Using a cell above a treatment cell tests for "treatment anticipation" which may be the cause if the TE != 0
- Doubly-Robust DiD (DRDiD) - uses robust method for calculating Average Treatment Effect on Treated (ATT)
- Calloway and Santa Anna DiD (CSDiD) - TE calculations only occur on "good design" (see above) squares and it applies the DRDiD calculation method on those squares
- [{did}]{style="color: #990000"}
Notes ([article](https://andrewcbaker.netlify.app/2019/09/25/difference-in-differences-methodology/))
- Equivalent Treatment Effect (TE) Calculations for Traditional 2x2 DiD
- Manual - treated unit in period 2 (YT2) and the treated unit in period 1 (YT1), less the same difference in the control unit (YC2− YC1).
- i.e. TE = (YT2 - YT1) - (YC2− YC1)
- If units are clusters of individuals, then Y is a mean of those individuals during those periods (I think)
- Regression\
![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/Screenshot%20(782).png)
- Where POSTt is an indicator for being in the second period and TREATi is an indicator for the treated unit
- TE = δ the coefficient of the interaction
- 2wfe\
![](./qmd/_resources/Econometrics,_Fixed_Effects.resources/Screenshot%20(794).png)
-
- When the treatment effects do not change over time (homogeneous), βDD is the variance-weighted average of cross-group treatment effects, and all of the weights are positive.
- When the treatment effect does vary across time (heterogeneous), some of these 2x2 estimates enter the average with negative weights.
- This is because already-treated units act as controls, and changes in a portion of their treatment effects over time are subtracted from the DiD estimate.
- Question
- Statement: \^β2x2,k only uses group l's pre-period, so it's sample share is (nk+nl)(1−Dl), while \^β2x2,l, uses group k's post-period, so it's share is (nk+nl)Dk.
- If there are 3 periods (pre, mid, post), it's saying the effect calculated from the 2x2 in
- chart C only uses 33% of the "late" group (green) observations
- chart D uses 66% of the "early" group (red) observartions
-
## It must have something to do with the model isn't "smart" and something with indicator variable showing treatment in 3 out of 4 of the 2x2 cells in D and 1 out of 4 of the 2x2 cells in C but I don't understand how that translates to the *usage* of observations in the statement
| | | | | | | |
|-----------|-----------|-----------|-----------|-----------|-----------|-----------|
| model | estimate | se | pval | groups | description | |
| happiness \~ age | 0.1151 | 0.0121 | 2.96e-09 \*\*\* | | pooled estimator | |
| happiness \~ age + cohort | 0.03687 | 0.01512 | 0.0237 \* | 2 groups | | |
| happiness \~ age + idname | -0.14824 | 0.01742 | 1.56e-07 \*\*\* | 7 groups | within estimator: compares different periods within the same person and discards the between-person variance | plm: effect = "individual", model = "within"<br>or<br>lm(happiness \~ age + idname, data = df) |
| wage \~ marriage | 2066.7 | 446.4 | 0.00013 \*\*\* | | true effect = 500 | |
| wage \~ marriage + id | 850.000 | 84.552 | 4.834e-09 \*\*\* | | | plm: effect = "individual", model = "within" |