Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Neur1630 #90

Merged
merged 8 commits into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/workflows/neur1630-fall-tag.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: tag-image-neur1630
on: [workflow_dispatch]

env:
CLASS: neur1630
TARGET: base
SQLITE: false
PYTHON_VERSION: "3.9.17"

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't really matter for this time around, but I think we can probably just use checkout@v3 and the system will pick the latest minor version? Saves effort for some future updates :)


- name: Set Up GCloud
uses: google-github-actions/[email protected]
with:
project_id: ${{ secrets.GCP_PROJECT_ID_JH_DOCKER }}
service_account_key: ${{ secrets.GCP_SA_KEY_JH_DOCKER }}
export_default_credentials: true

- name: Create Julia Environment Files
if: env.TARGET == 'r_julia'
run: |
sudo --preserve-env=CLASS docker-compose up julia_build

- name: Upload Julia Env Files to Google Storage Bucket
if: env.TARGET == 'r_julia'
run: |
gsutil cp -r requirements/classes/neur1630/julia_env gs://jupyterhub-environment-files/fall-2023/neur1630/past/${GITHUB_SHA}/julia
gsutil cp -r requirements/classes/neur1630/julia_env gs://jupyterhub-environment-files/fall-2023/neur1630/latest/julia

- name: Configure Docker
run: gcloud auth configure-docker

- name: Build and Push JH Image
run: |
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose up jh_image
docker tag jupyterhub-docker-images_jh_image:latest gcr.io/jupyterhub-docker-images/neur1630:${GITHUB_REF##*/}
docker tag jupyterhub-docker-images_jh_image:latest gcr.io/jupyterhub-docker-images/neur1630:${GITHUB_SHA}
docker tag jupyterhub-docker-images_jh_image:latest gcr.io/jupyterhub-docker-images/neur1630:latest
docker tag jupyterhub-docker-images_jh_image:latest gcr.io/jupyterhub-docker-images/neur1630:fall-2023
docker push gcr.io/jupyterhub-docker-images/neur1630:latest
docker push gcr.io/jupyterhub-docker-images/neur1630:fall-2023
docker push gcr.io/jupyterhub-docker-images/neur1630:${GITHUB_REF##*/}
docker push gcr.io/jupyterhub-docker-images/neur1630:${GITHUB_SHA}

- name: Upload Conda Env Files to Google Storage Bucket
run: |
gsutil cp -r requirements/out gs://jupyterhub-environment-files/fall-2023/neur1630/past/${GITHUB_SHA}/conda
gsutil cp -r requirements/out gs://jupyterhub-environment-files/fall-2023/neur1630/latest/conda
47 changes: 47 additions & 0 deletions .github/workflows/neur1630.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: build-image-neur1630
on:
push:
paths:
- 'requirements/classes/neur1630/**'
- 'Dockerfile'
- 'docker-compose.yml'
- 'scripts/**'
- '.github/workflows/neur1630.yml'
workflow_dispatch:

env:
CLASS: neur1630
TARGET: base
SQLITE: false
PYTHON_VERSION: "3.9.17"

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]

- name: Set Up GCloud
uses: google-github-actions/[email protected]
with:
project_id: ${{ secrets.GCP_PROJECT_ID_JH_DOCKER }}
service_account_key: ${{ secrets.GCP_SA_KEY_JH_DOCKER }}
export_default_credentials: true

- name: Create Julia Environment Files
if: env.TARGET == 'r_julia'
run: |
sudo --preserve-env=CLASS docker-compose up julia_build

- name: Configure Docker
run: gcloud auth configure-docker
- uses: satackey/[email protected]
# Ignore the failure of a step and avoid terminating the job.
continue-on-error: true
- name: Build and Push JH Image
run: |
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose up jh_image
docker tag jupyterhub-docker-images_jh_image:latest gcr.io/jupyterhub-docker-images/neur1630:${GITHUB_REF##*/}
docker tag jupyterhub-docker-images_jh_image:latest gcr.io/jupyterhub-docker-images/neur1630:latest
docker push gcr.io/jupyterhub-docker-images/neur1630:latest
docker push gcr.io/jupyterhub-docker-images/neur1630:${GITHUB_REF##*/}
15 changes: 15 additions & 0 deletions requirements/classes/neur1630/condarc
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# channel locations. These override conda defaults, i.e., conda will
# search *only* the channels listed here, in the order given.
# Use "defaults" to automatically include all default channels.
# Non-url channels will be interpreted as Anaconda.org usernames
# (this can be changed by modifying the channel_alias key; see below).
# The default is just 'defaults'.
channels:
- conda-forge

# Show channel URLs when displaying what is going to be downloaded
# and in 'conda list'. The default is False.
show_channel_urls: True

# For more information about this file see:
# https://conda.io/docs/user-guide/configuration/use-condarc.html
12 changes: 12 additions & 0 deletions requirements/classes/neur1630/packages.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Use this script to install packages via CRAN, for example:

# NOTE: Code below shows how you install R packages from CRAN and Bioconductor. For CRAN packages, you can use
# the standard install.packages() function; for Bioconductor packages, however, you
# must first install BiocManager and then use that for installs.

# install.packages("BiocManager", dependencies=TRUE, repos='http://cran.rstudio.com/')
# install.packages(c("ggplot2", "pheatmap", "RColorBrewer", "PoiClaClu",
# "patchwork", "tidyr", "GGally"), dependencies=TRUE,
# repos='http://cran.rstudio.com/')
# BiocManager::install(c("airway", "DESeq2", "vsn", "biomaRt",
# "AnnotationHub", "SummarizedExperiment"))
5 changes: 5 additions & 0 deletions requirements/classes/neur1630/requirements.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
const julia_packages = [
"IJulia", # DO NOT REMOVE
# add class specific classes here:

]
2 changes: 2 additions & 0 deletions requirements/classes/neur1630/requirements.pip.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# add class-specific classes here:

52 changes: 52 additions & 0 deletions requirements/classes/neur1630/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# add class-specific packages here:

# these are the scipy base packages ( YOU PROBABLY DON'T WANT TO REMOVE THESE )
dask
pandas
numexpr
matplotlib
scipy
seaborn
scikit-learn
scikit-image
sympy
cython
patsy
statsmodels
cloudpickle
dill
numba
bokeh
sqlalchemy
hdf5
vincent
beautifulsoup4
protobuf
xlrd
bottleneck
pytables

# these are the base r packages in the r-notebook image ( YOU PROBABLY WANT THESE IF YOUR TARGET IN DOCKER IS r_lang OR r_julia )
r-base=4.2.*
r-caret=6.*
r-crayon=1.*
r-devtools=2.*
r-forecast=8.*
r-hexbin=1.*
r-htmltools=0.*
r-htmlwidgets=1.*
r-irkernel=1.*
r-nycflights13=1.*
r-randomforest=4.*
r-rcurl=1.98*
r-rmarkdown=2.*
r-rodbc=1.*
r-rsqlite=2.*
r-shiny=1.*
r-tidyverse
unixodbc=2.*
r-tidymodels=1.*

# required for environment and plugins ( DO NOT REMOVE THESE!!! )
ipython
ipykernel