Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

calc_wn is biased #995

Open
ykyohei opened this issue Oct 10, 2024 · 0 comments
Open

calc_wn is biased #995

ykyohei opened this issue Oct 10, 2024 · 0 comments

Comments

@ykyohei
Copy link
Contributor

ykyohei commented Oct 10, 2024

fft_ops.calc_wn takes median of psd as a estimator of white noise level (wn), and this is biased from the average of psd.

The power spectrum density is usually calculated by the fft_ops.calc_psd using scipy.welch, and the amount of bias can be different between different observations because we allow different nperseg for different observations.

There are at least two ways to obtain the unbiased estimate of wn.

  1. Take an average of the psd in the frequency range without peaks.
  2. Take a median of the psd to be robust against peaks and then debias. For example. we set noverlap = 0 in scipy welch and debias depending on the number of segments. The average PSD of non-overlapping n segments (scipy.welch psd with noverlap = 0) follows chi square distribution with 2*n degrees of freedom, thus correction factor will be 2*n/scipy.stats.chi2.ppf(0.5, 2*n). The maximum correction factor for psd is ~1.44 when n=1. The current fft_ops.calc_psd allows overlap between segments and it's more complicated to obtain the correction factor.

Controlling the bias of wn is important for correct understanding of the detector noise performance, and for correct inverse variance weighting for mapmaking.

Figure 25 (3)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant