Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add customizations for use in HEB 115 #3

Merged
merged 8 commits into from
Sep 23, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions local/heb115.yml.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Batch Connect app configuration file
#
# @note Used to define the submitted cluster, title, description, and
# hard-coded/user-defined attributes that make up this Batch Connect app.
<%-
userGroups = OodSupport::User.new.groups.sort_by(&:id).map(&:name).flat_map{ |str| str.scan(/^canvas(\d+)-\d+/) }.flatten
Copy link
Member

@arthurian arthurian Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great use of OodSupport to get the user groups (I wasn't previously aware of this helper library). One thing that jumps out at me as being slightly unclear, and this may just be my lack of familiarity with Ruby, is the use of flat_map. I wonder if that could be replaced with a standard map and then just flatten once at the end. To me that reads a little clearer and maybe less redundant, although the result will be exactly the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no strong opinions about Ruby syntax, so I'm happy to change it if it makes things clearer. Any Ruby code I'm writing, I'm doing with significant help from Stack Overflow and GPT.

enabledGroups = [
"135510", # HUIT Open OnDemand Testing
"139860" # HEB 115: Investigating the Human Genome
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and clear pattern for adding sub apps to other courses!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed! This is a nice pattern that we can use for other sub-apps as well. Do we think that it's clear from the context that enabledGroups and userGroups refer to lists of Canvas Course IDs, not POSIX group names? I'm mainly asking because as we propagate these changes to other apps, it may be helpful to quickly locate places that are operating specifically on Canvas Course IDs (like here), compared to places that are operating on POSIX group names (or IDs).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a note is warranted, since there's some nuance to it. It's the Canvas Course IDs, which are used because they're present in the grouper group names which are translated into POSIX group names.

]

def arrays_have_common_element(array1, array2)
# Use the `&` operator to get the intersection of the two arrays
# If the intersection is not empty, return true, otherwise false
!(array1 & array2).empty?
end

# Check if the groups that the user is in match any of the courses that should
# have access to this app.

if arrays_have_common_element(userGroups, enabledGroups)
cluster="*"
else
cluster="disable_this_app"
end
-%>
---

# Set cluster from group. "*" runs on whatever cluster is available, and
# "disable_this_app" hides the app from view.
cluster: "<%= cluster %>"

title: "Jupyter Lab - HEB 115"

# Define attribute values that aren't meant to be modified by the user within
# the Dashboard form
attributes:
# Set the corresponding modules that need to be loaded for Jupyter to run
#
# @note It is called within the batch job as `module load <modules>` if
# defined
# @example Do not load any modules
# modules: ""
# @example Using default python module
# modules: "python"
# @example Using specific python module
# modules: "python/3.5"
# @example Using combination of modules
# modules: "python/3.5 cuda/8.0.44"
environment: heb115

spack: "/shared/spack"

custom_num_cores:
widget: "number_field"
label: "Number of CPUs"
value: 1
min: 1
max: 4
step: 1

# Any extra command line arguments to feed to the `jupyter notebook ...`
# command that launches the Jupyter notebook within the batch job
extra_jupyter_args: ""

# All of the attributes that make up the Dashboard form (in respective order),
# and made available to the submit configuration file and the template ERB
# files
#
# @note You typically do not need to modify this unless you want to add a new
# configurable value
# @note If an attribute listed below is hard-coded above in the `attributes`
# option, then it will not appear in the form page that the user sees in the
# Dashboard
form:
- environment
- spack
- extra_jupyter_args
- bc_num_hours
- custom_num_cores
9 changes: 7 additions & 2 deletions manifest.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
---
name: Jupyter Notebook (spack)
name: Jupyter Lab
category: Interactive Apps
subcategory: Servers
role: batch_connect
description: |
This app will launch a Jupyter Notebook server on one or more nodes. This configuration uses spack to load Jupyter Lab.
This app will launch a Jupyter Lab server. This configuration uses
[Spack](https://spack.io/) to load Jupyter Lab. The app launches outside of a
container, so it has access to slurm commands, but since the app is running as
a slurm job, the `srun` command will use the resources of the current job,
rather than starting a new job. That is, unless you first run an `salloc`
command to allocate a new interactive session first.
19 changes: 19 additions & 0 deletions spack-environment/heb115/spack.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs:
- admixtools
- bcftools
- plink
- samtools
- py-jupyterlab
- py-numpy
- py-pandas
- py-matplotlib
- py-scipy
view: true
concretizer:
unify: true