Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KOGITO-9982] Add persistence to SonataFlow CRD #529

Merged
8 changes: 8 additions & 0 deletions serverlessworkflow/modules/ROOT/pages/cloud/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,14 @@ xref:cloud/operator/service-discovery.adoc[]
Learn how to configure and use the {operator_name} Kubernetes service discovery
--

[.card]
--
[.card-title]
xref:cloud/operator/using-persistence.adoc[]
[.card-description]
Learn how to define the workflow `Persistence` field to allow the workflow to store its context
--

[.card]
--
[.card-title]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
= Using persistence in the SonataFlow Workflow CR
:compat-mode!:
// Metadata:
:description: Using persistence in the workflow instance to store its context
:keywords: sonataflow, workflow, serverless, operator, kubernetes, persistence


This document describes how to configure a SonataFlow instance to use persistence to store the flow's context in a relational database.

== Configuring the SonataFlow CR to use persistence

Kubernetes's pods are stateless by definition. In some scenarios, this can be a challenge for workloads that require maintaining the status of
the application regardless of the pod's lifecycle. In the case of SonataFlow, the context of the workflow is lost when the pod restarts.
jordigilh marked this conversation as resolved.
Show resolved Hide resolved
If your workflow requires recovery from such scenarios, you must to make these additions to your workflow CR:
Use the `persistence` field in the `SonataFlow` workflow spec to define the database service located in the same cluster.
There are 2 ways to accomplish this:

* Using the Platform CR's defined persistence
When the Platform CR is deployed with its persistence spec populated it enables workflows to leverage its configuration to populate the persistence
properties in the workflows.

[source,yaml,subs="attributes+"]
---
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
name: sonataflow-platform
spec:
persistence:
postgresql:
secretRef:
name: postgres-secrets
userKey: POSTGRES_USER
passwordKey: POSTGRES_PASSWORD
serviceRef:
name: postgres
port: 5432
databaseName: sonataflow
databaseSchema: shared
build:
config:
strategyOptions:
KanikoBuildCacheEnabled: "true"
---

The values of `POSTGRES_USER` and `POSTGRES_PASSWORD` are the keys in the secret that contains the credentials to connect to the postgreSQL instance.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The values of `POSTGRES_USER` and `POSTGRES_PASSWORD` are the keys in the secret that contains the credentials to connect to the postgreSQL instance.
The values of `POSTGRES_USER` and `POSTGRES_PASSWORD` are the secret keys containing the credentials to connect to the PostgreSQL instance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure because this change conveys that the keys are a secret, when in fact the keys are not a secret but are stored in an object of kind secret. It's a kubernetes terminology.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imho we can add the link:
...are the keys in the [secret](https://kubernetes.io/docs/concepts/configuration/secret/)

jordigilh marked this conversation as resolved.
Show resolved Hide resolved
The SonataFlow Workflow CR
is defined with its `Persistence` field defined as empty.

[source,yaml,subs="attributes+"]
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlow
metadata:
name: callbackstatetimeouts
annotations:
sonataflow.org/description: Callback State Timeouts Example k8s
sonataflow.org/version: 0.0.1
spec:
persistence: {}
...
---

This configuration signals the operator that the workflow requires persistence and that it expects its configuration to be populated accordingly.
The operator will add the relevant JDBC properties in the `application.properties`
generated and as part of the pod´s environment so that it can connect to the persistence service defined in the `Platform` CR.

Note that currently only PostgreSQL is the only persistence supported.
jordigilh marked this conversation as resolved.
Show resolved Hide resolved

* Using the custom defined persistence in the SonataFlow CR
jordigilh marked this conversation as resolved.
Show resolved Hide resolved

Alternatively, you can define a dedicated configuration in the SonataFlow CR instance using the same schema format found in the Platform CRD:
jordigilh marked this conversation as resolved.
Show resolved Hide resolved

[source,yaml,subs="attributes+"]
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlow
metadata:
name: callbackstatetimeouts
annotations:
sonataflow.org/description: Callback State Timeouts
sonataflow.org/version: 0.0.1
spec:
persistence:
postgresql:
secretRef:
name: postgres-secrets
userKey: POSTGRES_USER
passwordKey: POSTGRES_PASSWORD
serviceRef:
name: postgres
port: 5432
databaseName: sonataflow
databaseSchema: callbackstatetimeouts
...
---

Like in the Platform CR case, the values of the `POSTGRES_USER` and `POSTGRES_PASSWORD` are the secret keys in the secret that contain the credentials to connect to
the PostgreSQL instace.

== Conclusion
You can enable SQL persistence in your workflows by configuring each `SonataFlow` CR instance. And when the `SonataFlowPlatform` CR contains the persistence field configured,
the operator uses this information to configure those `SonataFlow` CRs that request persistence. When both the `Platform CR` and the `SonataFlow CR` contain persistence
configuration, the operator will use the `Persistence` values from the `SonataFlow` CR.

== Additional resources
* xref:cloud/operator/developing-workflows.adoc[]

include::../../../pages/_common-content/report-issue.adoc[]
Loading