HPC integration option 2 - Open OnDemand app

[rlskoeser]

The second option we want to test for HPC integration is a custom OnDemand interactive application that can be triggered by url from eScriptorium.

Note that CAS integration (included in #10) is a prerequisite for this. Once a user is logged in with CAS to eScriptorium their session should transfer to OnDemand.

implementation details

• dev access to OnDemand on della
• write a minimal python app (Flask?) that uses the eScriptorium REST API to pull necessary training data and model and start a slurm job as the logged in user
• figure out a way to trigger this task from eScriptorium interface (maybe a button? should it bypass celery?)
• implement method to check status of remote slurm job (maybe shared/overlaps with status check in HPC integration option 1 - ssh  #15 ? can the OnDemand app do this?)
  • figure out a way to report on status within eScriptorium ?
• Implement post-processing to run when the job completes: update the refined model in escriptorium and report on status

[linear]

RSE-101 HPC integration option 2 - Open OnDemand app

The second option we want to test for HPC integration is a custom OnDemand interactive application that can be triggered by url from eScriptorium.

Note that CAS integration (included in #10) is a prerequisite for this. Once a user is logged in with CAS to eScriptorium their session should transfer to OnDemand.

implementation details

• dev access to OnDemand on della
• write a minimal python app (Flask?) that uses the eScriptorium REST API to pull necessary training data and model and start a slurm job as the logged in user
• figure out a way to trigger this task from eScriptorium interface (maybe a button? should it bypass celery?)
• implement method to check status of remote slurm job (maybe shared/overlaps with status check in HPC integration option 1 - ssh  #15 ? can the OnDemand app do this?)
  • figure out a way to report on status within eScriptorium ?
• Implement post-processing to run when the job completes: update the refined model in escriptorium and report on status