Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stale acu position data in hwp supervisor during check_acu_ok_for_spinup() #799

Open
BrianJKoopman opened this issue Dec 2, 2024 · 1 comment
Assignees
Labels
agent: hwp supervisor bug Something isn't working needs triage Cause of bug still unknown, needs investigation.

Comments

@BrianJKoopman
Copy link
Member

Related to this issue, satp3 had similar problem on Nov 29th 2024, by following schedule https://site.simonsobs.org/satp3/nextline/db/runs/2202

run.wait_until('2024-11-29T18:44:23.980000+00:00')
run.acu.move_to(az=180.0, el=40.0)
run.acu.move_to(az=180.0, el=48.0)
######## HWP spinning up ##########
run.hwp.set_freq(freq=2) # start HWP rotation

In this case, I think the safety was triggered due to the time lag of acu monitoring.
hwp spin up was called right after the telescope has moved from 40 to 48, and the hwp_supervisor checked the acu position that was updated slightly before, and it was lower than 48.
One of the solutions is to just add a time buffer in the schedule.

Originally posted by @ykyohei in #788 (comment)

As discussed in that original thread, it seems most likely the HWP supervisor has an outdated cache of the ACU elevation. Can we force a hwp_state.acu.update() at the start of check_acu_ok_for_spinup()?

@jlashner
Copy link
Collaborator

jlashner commented Dec 2, 2024

Yes, I think forcing a state update before a check like this is reasonable. I can implement that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent: hwp supervisor bug Something isn't working needs triage Cause of bug still unknown, needs investigation.
Projects
None yet
Development

No branches or pull requests

2 participants