Skip to content

Commit

Permalink
Doc script settings and triggers (#739)
Browse files Browse the repository at this point in the history
  • Loading branch information
hcourdent authored Oct 17, 2024
1 parent db49bc9 commit acc355b
Show file tree
Hide file tree
Showing 55 changed files with 339 additions and 210 deletions.
4 changes: 2 additions & 2 deletions blog/2023-11-21-dedicated-workers/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ Flows steps that use scripts with a dedicated worker already benefit from the sp
href="/docs/misc/worker_group_management_ui"
/>
<DocCard
title="Init Scripts"
description="Init Scripts are executed at the beginning when the worker starts."
title="Init scripts"
description="Init scripts are executed at the beginning when the worker starts."
href="/docs/advanced/preinstall_binaries#init-scripts"
/>
</div>
12 changes: 6 additions & 6 deletions blog/2023-11-21-workers-group/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -117,17 +117,17 @@ This approach offers added convenience to pre-install binaries or set initial co
<img
src={require('../../docs/advanced/8_preinstall_binaries/init_scripts.png').default}
alt="Init Script"
title="Init Script"
alt="Init script"
title="Init script"
width="75%"
/>
<br />
<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Init Scripts"
description="Init Scripts are executed at the beginning when the worker starts."
title="Init scripts"
description="Init scripts are executed at the beginning when the worker starts."
href="/docs/advanced/preinstall_binaries#init-scripts"
/>
</div>
Expand Down Expand Up @@ -174,8 +174,8 @@ You can then configure it directly from the UI and in particular set tags to lis
href="/docs/misc/worker_group_management_ui"
/>
<DocCard
title="Init Scripts"
description="Init Scripts are executed at the beginning when the worker starts."
title="Init scripts"
description="Init scripts are executed at the beginning when the worker starts."
href="/docs/advanced/preinstall_binaries#init-scripts"
/>
</div>
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
slug: concurrency-limit-observability
version: v1.331.0
title: Concurrency Limit Observability
title: Concurrency limit observability
tags: ['Concurrency limits', 'Script editor', 'Flow editor']
image: ./concurrent_runs.png
description: Obversability around concurrent jobs has been improved from the Runs menu.
Expand Down
3 changes: 1 addition & 2 deletions docs/advanced/10_browser_automation/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ Running chromium without the sandbox is a security risk. Make sure you trust the
:::

To run jobs on a chromium-equipped worker, you have to select the `chromium` tag in the settings of the script or flow step.
[Learn how here](/docs/script_editor/worker_group_tag).

[Learn how here](../../core_concepts/9_worker_groups/index.mdx).

## Examples

Expand Down
12 changes: 6 additions & 6 deletions docs/advanced/8_preinstall_binaries/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import DocCard from '@site/src/components/DocCard';

:::tip Init scripts

For an efficient way to preinstall binaries without the need to modify the base image, see [Init Scripts](#init-scripts).
For an efficient way to preinstall binaries without the need to modify the base image, see [Init scripts](#init-scripts).

:::

Expand Down Expand Up @@ -85,18 +85,18 @@ FROM ${WM_IMAGE}
[...]
```

## Init Scripts
## Init scripts

Init scripts provide a method to pre-install binaries or set initial configurations without the need to modify the base image. This approach offers added convenience.
Init Scripts are executed at the beginning when the worker starts, ensuring that any necessary binaries or configurations are set up before the worker undertakes any other job.
Init scripts are executed at the beginning when the worker starts, ensuring that any necessary binaries or configurations are set up before the worker undertakes any other job.

Init scripts are available only on [self-hosting and enterprise edition](/pricing) on private clusters.

<div style={{ display: 'flex', justifyContent: 'center', alignItems: 'center', height: '100%' }}>
<img
src={require('./init_scripts.png.webp').default}
alt="Init Scripts Infographics"
title="Init Scripts Infographics"
alt="Init scripts Infographics"
title="Init scripts Infographics"
width="70%"
/>
</div>
Expand All @@ -107,7 +107,7 @@ Any scripts are set from the [Worker Management UI](../../misc/11_worker_group_m

When adjustments are made in the [Worker Management UI](../../misc/11_worker_group_management_ui/index.mdx), the workers will shut down and are expected to be restarted by their supervisor (Docker or k8s).

The execution of Init Scripts is inspectable in the superadmin workspace, with `Kind = All` filter. The path of those executions are `init_script_{worker_name}`.
The execution of Init scripts is inspectable in the superadmin workspace, with `Kind = All` filter. The path of those executions are `init_script_{worker_name}`.

![Worker Group edit config](../../core_concepts/9_worker_groups/worker_group_ui.png.webp)

Expand Down
2 changes: 1 addition & 1 deletion docs/code_editor/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ The code editor is the main component for building scripts in Windmill's script

Code editor in [Scripts](../script_editor/index.mdx):

![Script languages](../../static/images/script_languages.png.webp 'Script languages')
![Script languages](../../static/images/script_languages.png 'Script languages')

Code editor in [Flows](../flows/1_flow_editor.mdx):

Expand Down
2 changes: 1 addition & 1 deletion docs/compared_to/prefect.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -432,7 +432,7 @@ This document has focused on the Flow editor since it is the product closest to

The [script editor](../script_editor/index.mdx) allows users to build long-running heavy background jobs, script with complex dependencies, endpoints with high rpm or simple one-off tasks without any overhead. They can be triggered them from a webhook, auto-generated UIs, flows, apps etc.

![Script languages](../../static/images/script_languages.png.webp 'Script languages')
![Script languages](../../static/images/script_languages.png 'Script languages')

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
Expand Down
4 changes: 2 additions & 2 deletions docs/core_concepts/20_jobs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ output and metadata in the dedicated details page of the job.
href="/docs/misc/worker_group_management_ui"
/>
<DocCard
title="Init Scripts"
description="Init Scripts are executed at the beginning when the worker starts."
title="Init scripts"
description="Init scripts are executed at the beginning when the worker starts."
href="/docs/advanced/preinstall_binaries#init-scripts"
/>
</div>
Expand Down
8 changes: 4 additions & 4 deletions docs/core_concepts/21_concurrency_limits/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Concurrency limit is a [Cloud plans and Pro Enterprise Self-Hosted](/pricing) on

Concurrency limit can be set from the Settings menu. When jobs reach the concurrency limit, they are automatically queued for execution at the next available optimal slot given the time window.

The Concurrency Limit operates globally and across flow runs. It involves three key parameters:
The Concurrency limit operates globally and across flow runs. It involves three key parameters:

## Max number of executions within the time window

Expand All @@ -28,7 +28,7 @@ Jobs can be filtered from the [Runs menu](../5_monitor_past_and_future_runs/inde

[Concurrency limit of a script](../../script_editor/concurrency_limit.mdx) can be set from the Settings menu. Pick "Concurrency" and define a time window, max number of executions of the flow within that time window and optionally a custom concurrency key.

![Concurrency Limit](../../assets/code_editor/concurrency_limit.png.webp)
![Concurrency limit](../../assets/code_editor/concurrency_limit.png.webp)

### Concurrency limit of flow

Expand All @@ -38,8 +38,8 @@ From the Flow Settings menu, pick "Concurrency" and define a time window, a max

### Concurrency limit of scripts within flow

The Concurrency Limit operates globally and across flow runs. It involves two key parameters: "Maximum number of runs" and the "Per time window (seconds)."
The Concurrency limit operates globally and across flow runs. It involves two key parameters: "Maximum number of runs" and the "Per time window (seconds)."

Concurrency limit can be set for each step of a flow in the `Advanced` menu, on tab "Concurrency".

![Concurrency Limit Scripts within Flow](../../assets/code_editor/concurrency_limit_flow.png.webp "Concurrency Limit Scripts within Flow")
![Concurrency limit Scripts within Flow](../../assets/code_editor/concurrency_limit_flow.png.webp "Concurrency limit Scripts within Flow")
14 changes: 14 additions & 0 deletions docs/core_concepts/4_webhooks/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,20 @@ Windmill does not support batching cloud events yet. This means that requests wi

Learn more about what Cloud Events are [here](https://cloudevents.io/).

### Delete after use

For a script, delete [logs](../14_audit_logs/index.mdx), arguments and results after use.

:::warning

This settings ONLY applies to [synchronous](#synchronous) webhooks. If used individually, this script must be triggered using a synchronous endpoint to have the desired effect.
<br/>
The logs, arguments and results of the job will be completely deleted from Windmill once it is complete and the result has been returned.
<br/>
The deletion is irreversible.

:::

## Workspace Webhook

Connect your Windmill workspace to an external service to sync or get notified about any change.
Expand Down
14 changes: 7 additions & 7 deletions docs/core_concepts/9_worker_groups/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In the [Community Edition](/pricing), worker management is done using tags that

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Set Tags to Assign Specific Queues"
title="Set tags to assign specific queues"
description="You can assign groups to flows and flow steps to be executed on specific queues."
href="#set-tags-to-assign-specific-queues"
/>
Expand All @@ -34,7 +34,7 @@ In the [Cloud plans & Self-Hosted Enterprise Edition](/pricing), workers can be

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Create Worker Group config"
title="Create worker group config"
description="workers can be commonly managed based on the group they are in, from the UI."
href="#create-worker-group-config"
/>
Expand Down Expand Up @@ -257,15 +257,15 @@ Examples of configurations include:
## Init scripts
[Init scripts](../../advanced/8_preinstall_binaries/index.mdx#init-scripts) provide a method to pre-install binaries or set initial configurations without the need to modify the base image. This approach offers added convenience. Init Scripts are executed at the beginning when the worker starts, ensuring that any necessary binaries or configurations are set up before the worker undertakes any other job.
[Init scripts](../../advanced/8_preinstall_binaries/index.mdx#init-scripts) provide a method to pre-install binaries or set initial configurations without the need to modify the base image. This approach offers added convenience. Init scripts are executed at the beginning when the worker starts, ensuring that any necessary binaries or configurations are set up before the worker undertakes any other job.
Under the [Cloud plans & Self-Hosted Enterprise Edition](/pricing), they can be set from Windmill UI.
<div style={{ display: 'flex', justifyContent: 'center', alignItems: 'center', height: '100%' }}>
<img
src={require('../../advanced/8_preinstall_binaries/init_scripts.png').default}
alt="Init Scripts Infographics"
title="Init Scripts Infographics"
alt="Init scripts Infographics"
title="Init scripts Infographics"
width="70%"
/>
</div>
Expand All @@ -274,8 +274,8 @@ Under the [Cloud plans & Self-Hosted Enterprise Edition](/pricing), they can be
<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Init Scripts"
description="Init Scripts are executed at the beginning when the worker starts."
title="Init scripts"
description="Init scripts are executed at the beginning when the worker starts."
href="/docs/advanced/preinstall_binaries#init-scripts"
/>
</div>
Expand Down
30 changes: 20 additions & 10 deletions docs/core_concepts/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ All details on Scripts can be found in the [Script section](../script_editor/ind

<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="Script kinds"
title="Script kind"
description="You can attach additional functionalities to Scripts by specializing them into specific Script kinds."
href="/docs/script_editor/script_kinds"
/>
Expand All @@ -290,13 +290,8 @@ All details on Scripts can be found in the [Script section](../script_editor/ind
href="/docs/core_concepts/versioning#script-versioning"
/>
<DocCard
title="Worker group tag"
description="Scripts can be assigned custom worker groups for efficient execution on different machines with varying specifications."
href="/docs/script_editor/worker_group_tag"
/>
<DocCard
title="Concurrency Limit"
description="The Concurrency Limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
title="Concurrency limit"
description="The Concurrency limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
href="/docs/script_editor/concurrency_limit"
/>
<DocCard
Expand Down Expand Up @@ -378,7 +373,7 @@ All details on Flows can be found in the [Flows section](../flows/1_flow_editor.
<DocCard
color="teal"
title="Concurrency limits"
description="The Concurrency Limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
description="The Concurrency limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
href="/docs/flows/concurrency_limit"
/>
<DocCard
Expand All @@ -401,7 +396,7 @@ All details on Flows can be found in the [Flows section](../flows/1_flow_editor.
/>
<DocCard
color="teal"
title="Cache for Steps"
title="Cache for steps"
description="Re-use a step's previous results."
href="/docs/flows/cache"
/>
Expand Down Expand Up @@ -599,4 +594,19 @@ All details & features on [Pricing page](/pricing).
description="The Windmill React SDK provides a suite of tools and components to integrate Windmill applications into React-based projects."
href="/docs/misc/react_sdk"
/>
<DocCard
title="Windows workers"
description="Windows Native Workers are a Self-Hosted Enterprise Feature. You can use the Windows worker natively if you do not want to run Docker or WSL on Windows (e.g., for policy or security reasons). This feature supports the **Python**, **Bun**, and **PowerShell** executors."
href="/docs/misc/windows_workers"
/>
<DocCard
title="Private Hub"
description="Host your own Hub of scripts, flows, apps and resource types for your team."
href="/docs/core_concepts/private_hub"
/>
<DocCard
title="White labeling Windmill"
description="Windmill offers white labeling capabilities, allowing you to embed and customize the Windmill platform to align with your brand."
href="/docs/misc/white_labelling"
/>
</div>
2 changes: 1 addition & 1 deletion docs/flows/11_flow_approval.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ The approval steps are received in the form of an HTTP request. For each event,
<div className="grid grid-cols-2 gap-6 mb-4">
<DocCard
title="WindmillHub | Approval"
description="Find a library of Approval Scripts on WindmillHub."
description="Find a library of Approval scripts on WindmillHub."
href="https://hub.windmill.dev/approvals"
color="teal"
/>
Expand Down
4 changes: 2 additions & 2 deletions docs/flows/1_flow_editor.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ The Flow editor has the following features which are the subject of specific pag
<DocCard
color="teal"
title="Concurrency limits"
description="The Concurrency Limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
description="The Concurrency limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
href="/docs/flows/concurrency_limit"
/>
<DocCard
Expand All @@ -149,7 +149,7 @@ The Flow editor has the following features which are the subject of specific pag
/>
<DocCard
color="teal"
title="Cache for Steps"
title="Cache for steps"
description="Re-use a step's previous results."
href="/docs/flows/cache"
/>
Expand Down
4 changes: 2 additions & 2 deletions docs/flows/3_editor_components.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -466,14 +466,14 @@ The advanced section allows to configure the following:
/>
<DocCard
color="teal"
title="Cache for Steps"
title="Cache for steps"
description="Re-use a step's previous results."
href="/docs/flows/cache"
/>
<DocCard
color="teal"
title="Concurrency limits"
description="The Concurrency Limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
description="The Concurrency limit feature allows you to define concurrency limits for scripts and inline scripts within flows."
href="/docs/flows/concurrency_limit"
/>
<DocCard
Expand Down
6 changes: 3 additions & 3 deletions docs/flows/6_concurrency_limit.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Concurrency limit is a [Cloud plans and Pro Enterprise Self-Hosted](/pricing) on

Concurrency limit can be set from the Settings menu. When jobs reach the concurrency limit, they are automatically queued for execution at the next available optimal slot given the time window.

The Concurrency Limit operates globally and across flow runs. It involves three key parameters:
The Concurrency limit operates globally and across flow runs. It involves three key parameters:
- [Max number of executions within the time window](../core_concepts/21_concurrency_limits/index.md#max-number-of-executions-within-the-time-window)
- [Time window in seconds](../core_concepts/21_concurrency_limits/index.md#time-window-in-seconds)
- [Custom concurrency key](../core_concepts/21_concurrency_limits/index.md#custom-concurrency-key)
Expand All @@ -33,8 +33,8 @@ From the Flow Settings menu, pick "Concurrency" and define a time window and max

## Concurrency limit of scripts within flow

The Concurrency Limit operates globally and across flow runs. It involves two key parameters: "Maximum number of runs" and the "Per time window (seconds)."
The Concurrency limit operates globally and across flow runs. It involves two key parameters: "Maximum number of runs" and the "Per time window (seconds)."

Concurrency limit can be set for each step of a flow in the `Advanced` menu, on tab "Concurrency".

![Concurrency Limit Scripts within Flow](../assets/code_editor/concurrency_limit_flow.png.webp 'Concurrency Limit Scripts within Flow')
![Concurrency limit Scripts within Flow](../assets/code_editor/concurrency_limit_flow.png.webp 'Concurrency limit Scripts within Flow')
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit acc355b

Please sign in to comment.