Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support multiple spws in UVData.frequency_average #1307

Merged
merged 4 commits into from
Jul 6, 2023
Merged

Conversation

bhazelton
Copy link
Member

@bhazelton bhazelton commented Jun 15, 2023

Description

This improves the frequency_average method in the following ways:

  • Add support for multiple spectral windows.
  • New respect_spws parameter to control whether averaging can cross spectral window boundaries or not (default is True to disallow averaging over boundaries).
  • Add support for keeping the ragged frequencies if the number of frequencies in each spectral window does not divide evenly by the number of channels being averaged.

Motivation and Context

closes #913

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation change (documentation changes only)
  • Version change
  • Build or continuous integration change

Checklist:

New feature checklist:

  • I have added or updated the docstrings associated with my feature using the numpy docstring format.
  • I have updated the tutorial to highlight my new feature (if appropriate).
  • I have added tests to cover my new feature.
  • All new and existing tests pass.
  • I have updated the CHANGELOG.

@bhazelton bhazelton changed the title start work on supporting frequency averaging with multiple spws Improve the UVData.frequency_average method to support multiple spws and ragged averaging Jun 15, 2023
@bhazelton bhazelton changed the title Improve the UVData.frequency_average method to support multiple spws and ragged averaging Support multiple spws in UVData.frequency_average Jun 15, 2023
@codecov
Copy link

codecov bot commented Jun 16, 2023

Codecov Report

Merging #1307 (2bcf934) into main (611dce2) will increase coverage by 0.00%.
The diff coverage is 100.00%.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1307   +/-   ##
=======================================
  Coverage   99.92%   99.92%           
=======================================
  Files          36       36           
  Lines       20142    20211   +69     
=======================================
+ Hits        20126    20195   +69     
  Misses         16       16           
Impacted Files Coverage Δ
pyuvdata/utils.py 100.00% <ø> (ø)
pyuvdata/uvdata/uvdata.py 100.00% <100.00%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 611dce2...2bcf934. Read the comment docs.

@bhazelton bhazelton force-pushed the freq_avg_flex_spw branch from b0627bf to 4a52dda Compare June 16, 2023 21:06
@bhazelton bhazelton marked this pull request as ready for review June 16, 2023 22:08
@bhazelton bhazelton requested a review from kartographer June 16, 2023 22:08
@bhazelton bhazelton force-pushed the freq_avg_flex_spw branch 3 times, most recently from db8d06e to 90b4a32 Compare June 22, 2023 19:51
@bhazelton
Copy link
Member Author

The test errors are due to deprecation warnings issued by numpy 1.25. Fixed in #1308

Copy link
Contributor

@kartographer kartographer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @bhazelton -- overall I think this is looking pretty good! One general comment, I think the frequency_average method could benefit from some additional comments within the code itself (there was a fair bit of jumping back and forth to try to understand the underlying logic of what was going on), but other than that mot much more than few small optimizations.

pyuvdata/uvdata/uvdata.py Outdated Show resolved Hide resolved
pyuvdata/uvdata/uvdata.py Outdated Show resolved Hide resolved
pyuvdata/uvdata/uvdata.py Outdated Show resolved Hide resolved
this_final_reg_inds = final_spw_chans[spw]
if nfreq_mod_navg != 0:
# not an even number of final channels
regular_inds = these_inds[0 : n_final_chan_reg * n_chan_to_avg]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If may be nice to have some user-specified control on the channel at which you start averaging, particularly in the "ragged" case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it would be nice to allow the user to have more detailed control. We thought about something like you're suggesting, but as we thought about it seemed more complicated than just specifying where you start, e.g. what do you do with all the frequencies before the starting place? It seems to me that the most flexible thing would be to allow users to pass in an integer array that specifies which final channel to put each input channel into (and allowing the user to drop channels by specifying -1 in this array). But we decided to just do the obvious incremental improvement for now and wait to see what users request.

Copy link
Member Author

@bhazelton bhazelton Jun 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We talked more about this on the telecon and decided that this kind of input array would make it fully flexible for the power user. I made an issue to this effect (#1311) so we don't lose track of the idea.

@bhazelton bhazelton force-pushed the freq_avg_flex_spw branch from 90b4a32 to 18fc925 Compare June 26, 2023 23:16
@bhazelton bhazelton force-pushed the freq_avg_flex_spw branch from 18fc925 to 2bcf934 Compare June 30, 2023 20:36
@bhazelton
Copy link
Member Author

One general comment, I think the frequency_average method could benefit from some additional comments within the code itself (there was a fair bit of jumping back and forth to try to understand the underlying logic of what was going on)

I added a bunch of comments, let me know if they meet your needs.

Copy link
Contributor

@steven-murray steven-murray left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bhazelton, I think this looks good -- at least I found no glaring issues!

@bhazelton bhazelton merged commit e2b48a9 into main Jul 6, 2023
@bhazelton bhazelton deleted the freq_avg_flex_spw branch July 6, 2023 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Adding support for flexible spectral windows w/ frequency_average method of UVData
3 participants