Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Workflow API routes - Part 1 #17463

Merged
merged 57 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
222f364
Start refactoring invoke operation
heisner-tillman Feb 13, 2024
409ee08
Rework pydantic models for invoke operation from workflows API
heisner-tillman Feb 14, 2024
52958a3
Remove mapping to legacy route
heisner-tillman Feb 14, 2024
90d833a
Refactor invoke operation
heisner-tillman Feb 14, 2024
6ec9013
Create service method for invoke operation
heisner-tillman Feb 14, 2024
1876361
Add json keyword in tests calling invoke endpoint
heisner-tillman Feb 14, 2024
3beb2f9
Regenerate client schema
heisner-tillman Feb 14, 2024
06505d0
Add json keyword in tests calling invoke endpoint
heisner-tillman Feb 15, 2024
9afa0a2
Include more fields in the field validatior to transform them from s…
heisner-tillman Feb 15, 2024
87b6bed
Re-add mapping to some of the legacy routes of the Workflows API
heisner-tillman Feb 16, 2024
3f9d8f6
Remove legacy mapping to the invocation routes
heisner-tillman Feb 16, 2024
28aaac6
Refine pydantic model of the show operation
heisner-tillman Feb 16, 2024
22a260b
Make use of the pydantic model of the show operation in the service l…
heisner-tillman Feb 16, 2024
f9c847f
Remove InvocationService from the WorkflowsAPIController as it is not…
heisner-tillman Feb 16, 2024
d6bff71
Regenerate the client schema
heisner-tillman Feb 16, 2024
dcd2167
Add mising mappings to endpoints of the WorkflowAPI
heisner-tillman Feb 16, 2024
8bbfc77
Add json keyword to tests, when posting to invoke operation of Workfl…
heisner-tillman Feb 16, 2024
a240e38
Access error message of API response correctly
heisner-tillman Feb 16, 2024
364ec7a
Rgenerate the client schema
heisner-tillman Feb 16, 2024
f319eb1
Add description to fields of pydantic model
heisner-tillman Feb 17, 2024
4aa2f74
Type workflow id path parameter of invoke operation
heisner-tillman Feb 17, 2024
a8b35eb
Regenerate the client schema
heisner-tillman Feb 18, 2024
3a31dcd
Add description to fields of pydantic model
heisner-tillman Feb 18, 2024
6e71cee
Refactor the refactor operation of the WorkflowAPI
heisner-tillman Feb 18, 2024
c4d6031
Remove the mapping to the legacy route
heisner-tillman Feb 18, 2024
36751dc
Remove comments
heisner-tillman Feb 18, 2024
2a39866
Regenerate the client schema
heisner-tillman Feb 18, 2024
4e585aa
Create service method for get_tool_predictions operation from Workflo…
heisner-tillman Feb 19, 2024
bd7dca7
Add payload model for get_tool_predictions operation from WorkflowAPI
heisner-tillman Feb 19, 2024
cab6227
Refactor get_tool_predictions operation
heisner-tillman Feb 19, 2024
5ae614a
Remove mapping to the legacy route
heisner-tillman Feb 19, 2024
1897dc2
Regenerate the client schema
heisner-tillman Feb 19, 2024
4e1fda9
Create service method for show_workflow operation from the Workflows API
heisner-tillman Feb 20, 2024
92855f6
Refactor show operation
heisner-tillman Feb 20, 2024
a9276a1
Remove mapping to the legacy route
heisner-tillman Feb 20, 2024
141e5f5
Regenerate the client schema
heisner-tillman Feb 20, 2024
f3ffc33
Rename path parameter of show_worfklow operation
heisner-tillman Feb 21, 2024
b2fd59d
Move pydantic model StoredWorkflowDetailed to schema file of workflows
heisner-tillman Feb 21, 2024
66e0b6c
Add missing fields to pydantic model inorder to use it in show_workfl…
heisner-tillman Feb 21, 2024
a86661d
Make fields of Workflows model optional inorder to use it in show_wor…
heisner-tillman Feb 21, 2024
f91dd97
Add pydantic model to return of show_workflows operation
heisner-tillman Feb 21, 2024
c18366a
Regenerate the client schema
heisner-tillman Feb 21, 2024
a8ee9c5
Create pydantic model for get_tool_predictions return
heisner-tillman Feb 22, 2024
c01ff2c
Make field optional in payload model for get_tool_predicitions operation
heisner-tillman Feb 22, 2024
e9d28d6
Add pydantic model to return of get_tool_predicitions operation
heisner-tillman Feb 22, 2024
6ff8f45
Regenerate the client schema
heisner-tillman Feb 22, 2024
65da13b
Move get_tool_predictions operation to legacy route
heisner-tillman Feb 23, 2024
78725fb
Remove models for get_tool_predictions
heisner-tillman Feb 23, 2024
80110bd
Remove comments
heisner-tillman Feb 23, 2024
362d245
Regenerate the client schema
heisner-tillman Feb 23, 2024
b82ed24
Move manager logic of refactor operation to service layer
heisner-tillman Feb 23, 2024
393ffc6
Refine desctiptions
heisner-tillman Feb 23, 2024
9345085
Regenerate the client schema
heisner-tillman Feb 23, 2024
5400f74
Use galaxy exceptions instead of FastAPI HTTPError
heisner-tillman Feb 23, 2024
6c1f30e
Access the error message of invoke endpoint correctly
heisner-tillman Feb 24, 2024
bc705a2
Change type of metadata source field in pydantic model for show_workf…
heisner-tillman Feb 26, 2024
1aa27cb
Regenerate the client schema
heisner-tillman Feb 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,309 changes: 1,289 additions & 20 deletions client/src/api/schema/schema.ts

Large diffs are not rendered by default.

47 changes: 10 additions & 37 deletions lib/galaxy/schema/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -2112,10 +2112,12 @@ class StoredWorkflowSummary(Model, WithModelClass):
title="Published",
description="Whether this workflow is currently publicly available to all users.",
)
annotations: List[str] = Field( # Inconsistency? Why workflows summaries use a list instead of an optional string?
...,
title="Annotations",
description="An list of annotations to provide details or to help understand the purpose and usage of this workflow.",
annotations: Optional[List[str]] = (
Field( # Inconsistency? Why workflows summaries use a list instead of an optional string?
None,
title="Annotations",
description="An list of annotations to provide details or to help understand the purpose and usage of this workflow.",
)
)
tags: TagCollection
deleted: bool = Field(
Expand All @@ -2138,13 +2140,13 @@ class StoredWorkflowSummary(Model, WithModelClass):
title="Latest workflow UUID",
description="TODO",
)
number_of_steps: int = Field(
...,
number_of_steps: Optional[int] = Field(
None,
title="Number of Steps",
description="The number of steps that make up this workflow.",
)
show_in_tool_panel: bool = Field(
...,
show_in_tool_panel: Optional[bool] = Field(
None,
title="Show in Tool Panel",
description="Whether to display this workflow in the Tools Panel.",
)
Expand Down Expand Up @@ -2344,35 +2346,6 @@ class Person(Creator):
)


class StoredWorkflowDetailed(StoredWorkflowSummary):
annotation: Optional[str] = AnnotationField # Inconsistency? See comment on StoredWorkflowSummary.annotations
license: Optional[str] = Field(
None, title="License", description="SPDX Identifier of the license associated with this workflow."
)
version: int = Field(
..., title="Version", description="The version of the workflow represented by an incremental number."
)
inputs: Dict[int, WorkflowInput] = Field(
{}, title="Inputs", description="A dictionary containing information about all the inputs of the workflow."
)
creator: Optional[List[Union[Person, Organization]]] = Field(
None,
title="Creator",
description=("Additional information about the creator (or multiple creators) of this workflow."),
)
steps: Dict[
int,
Union[
InputDataStep,
InputDataCollectionStep,
InputParameterStep,
PauseStep,
ToolStep,
SubworkflowStep,
],
] = Field({}, title="Steps", description="A dictionary with information about all the steps of the workflow.")


class Input(Model):
name: str = Field(..., title="Name", description="The name of the input.")
description: str = Field(..., title="Description", description="The annotation or description of the input.")
Expand Down
220 changes: 220 additions & 0 deletions lib/galaxy/schema/workflows.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
import json
from typing import (
Any,
Dict,
List,
Optional,
Union,
)

from pydantic import (
Field,
field_validator,
)

from galaxy.schema.schema import (
AnnotationField,
InputDataCollectionStep,
InputDataStep,
InputParameterStep,
Model,
Organization,
PauseStep,
Person,
PreferredObjectStoreIdField,
StoredWorkflowSummary,
SubworkflowStep,
ToolStep,
WorkflowInput,
)


class GetTargetHistoryPayload(Model):
# TODO - Are the descriptions correct?
history: Optional[str] = Field(
None,
title="History",
# description="The encoded history id - passed exactly like this 'hist_id=...' - to import the workflow into. Or the name of the new history to import the workflow into.",
description="The encoded history id - passed exactly like this 'hist_id=...' - into which to import. Or the name of the new history into which to import.",
)
history_id: Optional[str] = Field(
None,
title="History ID",
# description="The history to import the workflow into.",
description="The encoded history id into which to import.",
)
new_history_name: Optional[str] = Field(
None,
title="New History Name",
# description="The name of the new history to import the workflow into.",
description="The name of the new history into which to import.",
)


class InvokeWorkflowPayload(GetTargetHistoryPayload):
# TODO - Are the descriptions correct?
instance: Optional[bool] = Field(
False,
title="Is instance",
description="True when fetching by Workflow ID, False when fetching by StoredWorkflow ID",
)
scheduler: Optional[str] = Field(
None,
title="Scheduler",
description="Scheduler to use for workflow invocation.",
)
batch: Optional[bool] = Field(
False,
title="Batch",
description="Indicates if the workflow is invoked as a batch.",
)
require_exact_tool_versions: Optional[bool] = Field(
True,
title="Require Exact Tool Versions",
description="If true, exact tool versions are required for workflow invocation.",
# description="TODO",
)
allow_tool_state_corrections: Optional[bool] = Field(
False,
title="Allow tool state corrections",
description="Indicates if tool state corrections are allowed for workflow invocation.",
)
use_cached_job: Optional[bool] = Field(
False,
title="Use cached job",
description="Indicated whether to use a cached job for workflow invocation.",
)
parameters_normalized: Optional[bool] = Field(
False,
title="Parameters Normalized",
description="Indicates if parameters are already normalized for workflow invocation.",
)

@field_validator(
"parameters",
"inputs",
"ds_map",
"resource_params",
"replacement_params",
"step_parameters",
mode="before",
check_fields=False,
)
@classmethod
def inputs_string_to_json(cls, v):
if isinstance(v, str):
return json.loads(v)
return v

parameters: Optional[Dict[str, Any]] = Field(
{},
title="Parameters",
description="The raw parameters for the workflow invocation.",
)
inputs: Optional[Dict[str, Any]] = Field(
None,
title="Inputs",
description="TODO",
)
ds_map: Optional[Dict[str, Dict[str, Any]]] = Field(
{},
title="Dataset Map",
description="TODO",
)
resource_params: Optional[Dict[str, Any]] = Field(
{},
title="Resource Parameters",
description="TODO",
)
replacement_params: Optional[Dict[str, Any]] = Field(
{},
title="Replacement Parameters",
description="TODO",
)
step_parameters: Optional[Dict[str, Any]] = Field(
None,
title="Step Parameters",
description="TODO",
)
no_add_to_history: Optional[bool] = Field(
False,
title="No Add to History",
description="Indicates if the workflow invocation should not be added to the history.",
)
legacy: Optional[bool] = Field(
False,
title="Legacy",
description="Indicating if to use legacy workflow invocation.",
)
inputs_by: Optional[str] = Field(
None,
title="Inputs By",
# lib/galaxy/workflow/run_request.py - see line 60
description="How inputs maps to inputs (datasets/collections) to workflows steps.",
)
effective_outputs: Optional[Any] = Field(
None,
title="Effective Outputs",
# lib/galaxy/workflow/run_request.py - see line 455
description="TODO",
)
preferred_intermediate_object_store_id: Optional[str] = Field(
None,
title="Preferred Intermediate Object Store ID",
description="The ID of the ? object store that should be used to store ? datasets in this history.",
)
preferred_outputs_object_store_id: Optional[str] = Field(
None,
title="Preferred Outputs Object Store ID",
description="The ID of the object store that should be used to store ? datasets in this history.",
)
preferred_object_store_id: Optional[str] = PreferredObjectStoreIdField


class StoredWorkflowDetailed(StoredWorkflowSummary):
annotation: Optional[str] = AnnotationField # Inconsistency? See comment on StoredWorkflowSummary.annotations
license: Optional[str] = Field(
None, title="License", description="SPDX Identifier of the license associated with this workflow."
)
version: int = Field(
..., title="Version", description="The version of the workflow represented by an incremental number."
)
inputs: Dict[int, WorkflowInput] = Field(
{}, title="Inputs", description="A dictionary containing information about all the inputs of the workflow."
)
creator: Optional[List[Union[Person, Organization]]] = Field(
None,
title="Creator",
description=("Additional information about the creator (or multiple creators) of this workflow."),
)
steps: Dict[
int,
Union[
InputDataStep,
InputDataCollectionStep,
InputParameterStep,
PauseStep,
ToolStep,
SubworkflowStep,
],
] = Field({}, title="Steps", description="A dictionary with information about all the steps of the workflow.")
importable: Optional[bool] = Field(
...,
title="Importable",
description="Indicates if the workflow is importable by the current user.",
)
email_hash: Optional[str] = Field(
...,
title="Email Hash",
description="The hash of the email of the creator of this workflow",
)
slug: Optional[str] = Field(
...,
title="Slug",
description="The slug of the workflow.",
)
source_metadata: Optional[Dict[str, Any]] = Field(
...,
title="Source Metadata",
description="The source metadata of the workflow.",
)
Loading
Loading