Executive summary of 22.01 Release Cycle Backend Working Group Goals

[jmchilton]

• Bring job setup decomposition from prototype to merged and deployable.
  • xref https://github.com/galaxyproject/galaxy/compare/dev...mvdbeek:wip_remote_command_line?expand=1
  • Containerized, Database-Free Job Creation #10873
• Batch-mode Pulsar pulsar#250
• Implement secrets management prototype based on planning document from 21.09 cycle. https://docs.google.com/document/d/1d-oWB8cdrIPjk60WHG4bx_S7yqA_kfzPr1Aknl_6PEs/edit?usp=sharing
• Drop uWSGI, modern server default.
  • Roadmap to FastAPI #12257
  • Provide example circus config for deploying nodejs proxy for ITs. #11823
  • Migrate ansible roles to support gunicorn, etc.. during 21.09 deployment cycle
  • Organize a documentation collaboration event with admin group on migrating all uWSGI documentation on docs and training material to newer stack
• Finish objectstore abstractions started in Q1 to enable scratch storage in Galaxy @jmchilton
  • Implementing quotas per objectstore [WIP] Implement quota tracking options per ObjectStore. #10977
  • Implemented private objectstores to disable sharing, etc.. [WIP] Implement abstractions to annotate non-sharable datasets & objectstores. #10840
• Assist debugging tool panel view issues in production setups.
• More async Celery tasks.
  • Prototype New Tool Submission/Job Creation Endpoint Based on Celery #11820
• Implement model store plugin for RO crate support.
  • Export Histories into Research Objects #3088
  • xref [WIP] Model Store Workflow Invocation Import/Export #9024
• Adopt Alembic for database migrations Move from sqlalchemy-migrate to alembic #10369
• Prepare code base for migration to SQLAlchemy 2.0
  • Complete migration to declarative mapping Declarative mappings + tests for all models #12064
  • Resolve declarative models follow-up items Declarative Models Follow Up #12355
  • Enable SQLALCHEMY_WARN_20 (exposes legacy patterns) (ref)
  • Resolve all RemovedIn20Warnings (ref)
  • Enable future flag for Engine (ref)
  • Enable future flag for Session (ref)
  • Resolve any leftover SAWarnings (except those we can safely ignore)