Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

Slurm postprocess #502

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open

Slurm postprocess #502

wants to merge 42 commits into from

Conversation

saraloo
Copy link
Collaborator

@saraloo saraloo commented Feb 28, 2023

Added files for automatic postprocessing on MARCC. Postprocessing.sh script will run these automatically after a run and flepibot will send the results.

@shauntruelove
Copy link
Member

I'm approving this, but i have one question: are we going to have issues where we need to constantly change the slack channel id? I see a lot of commits changing that.

shauntruelove
shauntruelove previously approved these changes Mar 1, 2023
jcblemai
jcblemai previously approved these changes Mar 2, 2023
Copy link
Collaborator

@jcblemai jcblemai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, thanks !
some thought

  • need to make the channel-id an argument at submission. I'll work on that.
  • is there some changes in the postprocessing/*_SLURM.R that could be backproppagated into the template files ? Do we still need the template files if these are the default one ?

@jcblemai
Copy link
Collaborator

jcblemai commented Mar 2, 2023

Need to think on how to merge that as we're moving repository

@saraloo
Copy link
Collaborator Author

saraloo commented Mar 2, 2023

most of the slack channel commits were because it wasn't working. but making it an argument makes sense.

Could possibly backpropagate so it runs automatically on AWS as well? but not sure exactly on the compatibility? Also is set to be specifically for just processing one run, whereas the template allows for multiple buckets for SMH so there's some finicky things that need manual inputs there...

@jcblemai
Copy link
Collaborator

jcblemai commented Mar 6, 2023

  • yes, I think getting that working in aws is a low priority todo. It just need some file updatesand a test
  • We can make multi scenario post-processing work as well :) I'll make a script to just launch the post-prcessing job.

@jcblemai jcblemai dismissed stale reviews from shauntruelove and themself via ee53755 March 13, 2023 13:43
jcblemai
jcblemai previously approved these changes Mar 27, 2023
jcblemai
jcblemai previously approved these changes Mar 28, 2023
@saraloo saraloo requested a review from shauntruelove March 28, 2023 20:26
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants