Feature request: Effective sample size for tbl_svysummary()

[malcolmbarrett]

We use tbl_svysummary() for propensity score-weighted analyses. tbl_svysummary() reports the sum of the weights as the sample size, which is certainly true from one perspective.

Another perspective is the effective sample size, the sample size from a simple random sampling that would give the same precision as the weighted analysis. It's often a good indicator of problems and intuitive for readers to understand the variance better. The calculation is very simple. In halfmoon, it's defined as:

ess <- function(wts) {
  sum(wts)^2 / sum(wts^2)
}

It would be great if tbl_svysummary() could have an option to include ESS. I think what I would really love is to be able to show the unweighted sample size, the sum of the weights, and the ESS all together, because each perspective is useful.

If you're interested, I'd be happy to make a PR if you could point me to the right spot for where these calculations happen

[ddsjoberg]

It seems reasonable to me! @larmarange any thoughts?

@malcolmbarrett would this calculation appear in the header of tbl_svysummary() only? Anywhere else these calculations are needed? Would these estimates be needed for every categorical tabulation and used as the denominator?

[larmarange]

Hi

They are also mentioned here: https://rdrr.io/github/mainwaringb/rakehelper/man/eff_n.html

This implementation is also adapted for survey objects.

These metrics should be used carefully as they do not take into account stratifications or clustering.

Also, I do not know if they are valid when the weights represent total population size (which could also happen), i.e. when the sum of weights is equal to the total population in the survey area and not to the total number of individuals surveyed.

We may consider making these indicators available to be used in modify_header() but we should be careful about the doc. What do you think?

[malcolmbarrett]

Would these estimates be needed for every categorical tabulation and used as the denominator?

You could definitely use them this way, but I think it's diminishing returns in terms of gaining insight into the analysis. As a first pass, I'd be happy with being able to do it for the overall sample and by the column groups.

As @larmarange, they are imperfect and don't represent the variance well in some cases. In IPW, they are often a good diagnostic. I can't speak as much to surveys