Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc object storage and git sync #734

Merged
merged 1 commit into from
Oct 11, 2024
Merged

Doc object storage and git sync #734

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/advanced/11_git_sync/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import DocCard from '@site/src/components/DocCard';

From the workspace settings, you can set a [git_repository](../../integrations/git_repository.mdx) resource on which the workspace will automatically commit and push scripts, flows and apps to the repository on each [deploy](../../core_concepts/0_draft_and_deploy/index.mdx).

You can use this feature to [Deploy to prod using a git workflow](#git-sync---item-mode-deploy-to-prod-using-a-git-workflow).

:::tip Version control

For all details on Version control in Windmill, see [Version control](../../advanced/13_version_control/index.mdx).
Expand Down Expand Up @@ -93,6 +95,8 @@ https://AZURE_DEVOPS_TOKEN(<path_to_the_azure_resource>)@dev.azure.com/<organiza
This feature can be used alongside GiHub Actions to adopt a robust development process for your Windmill scripts, flows and apps,
with for example a Staging Workspace making automatically PRs on a repo that pushes to a Prod workspace upon merge.

![Local development Setup](../4_local_development/local_development.png 'Local development Setup')

<iframe
style={{ aspectRatio: '16/9' }}
src="https://www.youtube.com/embed/es8FUC2M73o"
Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/18_instance_settings/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import TabItem from '@theme/TabItem';

# Instance settings

Instance settings are accessible to all [superadmins](../../core_concepts/16_roles_and_permissions/index.mdx#superadmin) of your Windmill instance. This is where you manage settings and features across all workspaces.
Instance settings are accessible to all [superadmins](../../core_concepts/16_roles_and_permissions/index.mdx#superadmin) of your Windmill instance. This is where you manage settings and features across all workspaces. One instance can have several workspaces.

This is from the Instance settings that you can see on which Windmill [version](https://github.com/windmill-labs/windmill/releases) your instance is running.

Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/4_local_development/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ The metadata file schema is available [here](../../core_concepts/13_json_schema_

### Editing and creating scripts locally

To summarize, here are the canonical process to edit and create scripts locally (details below):
To summarize, here are the canonical process to edit and create scripts locally using [Windmill CLI](../3_cli/index.mdx) (details below):

```bash
# optional - create a new script using the CLI (can be done manually)
Expand Down
Binary file modified docs/core_concepts/18_files_binary_data/s3_file_output.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 7 additions & 3 deletions docs/core_concepts/38_object_storage_in_windmill/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,14 @@ At the [workspace level](#workspace-object-storage), what is exclusive to the [E

Additionally, for [instance integration](#instance-object-storage), the Enterprise version offers advanced features such as large-scale log management and distributed dependency caching.

![Object storage in Windmill](./object_storage_in_windmill.png 'Object storage in Windmill')

## Workspace object storage

Connect your Windmill workspace to your S3 bucket or your Azure Blob storage to enable users to read and write from S3 without having to have access to the credentials.

![Workspace object storage infographic](../11_persistent_storage/s3_infographics.png 'Workspace object storage infographic')

Windmill S3 bucket browser will not work for buckets containing more than 20 files and uploads are limited to files < 50MB. Consider upgrading to Windmill [Enterprise Edition](/pricing) to use this feature with large buckets.

Once you've created an [S3 or Azure Blob resource](../../integrations/s3.mdx) in Windmill, go to the workspace settings > S3 Storage. Select the resource and click Save.
Expand All @@ -31,8 +35,6 @@ S3 files in Windmill are just pointers to the S3 object using its key. As such,
}
```

![Workspace object storage infographic](../11_persistent_storage/s3_infographics.png 'Workspace object storage infographic')

### Resources permissions

The resource can be set to be public with toggle "S3 resource details can be accessed by all users of this workspace".
Expand Down Expand Up @@ -352,7 +354,9 @@ Learn more about it in the [Data pipelines](../27_data_pipelines/index.mdx) sect

## Instance object storage

Under [Enterprise Edition](/pricing), Instance object storage offers advanced features to enhance performance and scalability at the [instance](../../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.
Under [Enterprise Edition](/pricing), instance object storage offers advanced features to enhance performance and scalability at the [instance](../../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.

![Instance object storage infographic](./instance_object_storage_infographic.png 'Instance object storage infographic')

![S3/Azure for Python/Go cache & large logs](../../core_concepts/20_jobs/s3_azure_cache.png "S3/Azure for Python/Go cache & large logs")

Expand Down
Binary file added ..._concepts/38_object_storage_in_windmill/instance_object_storage_infographic.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/core_concepts/38_object_storage_in_windmill/object_storage_in_windmill.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/integrations/aws-s3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ The workspace object storage is exclusive to the [Enterprise](/pricing) edition.

## Instance object storage

Under [Enterprise Edition](/pricing), Instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.
Under [Enterprise Edition](/pricing), instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.

![S3/Azure for Python/Go cache & large logs](../core_concepts/20_jobs/s3_azure_cache.png "S3/Azure for Python/Go cache & large logs")

Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/cloudflare-r2.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ The workspace object storage is exclusive to the [Enterprise](/pricing) edition.

## Instance object storage

Under [Enterprise Edition](/pricing), Instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.
Under [Enterprise Edition](/pricing), instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.

![S3/Azure for Python/Go cache & large logs](../core_concepts/20_jobs/s3_azure_cache.png "S3/Azure for Python/Go cache & large logs")

Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/s3.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ The workspace object storage is exclusive to the [Enterprise](/pricing) edition.

## Instance object storage

Under [Enterprise Edition](/pricing), Instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.
Under [Enterprise Edition](/pricing), instance object storage offers advanced features to enhance performance and scalability at the [instance](../advanced/18_instance_settings/index.mdx) level. This integration is separate from the [Workspace object storage](#workspace-object-storage) and provides solutions for large-scale log management and distributed dependency caching.

![S3/Azure for Python/Go cache & large logs](../core_concepts/20_jobs/s3_azure_cache.png "S3/Azure for Python/Go cache & large logs")

Expand Down
Loading