Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kie-kogito-docs-664: Create a guide for the SonataFlow Operator driven Supporting Services Knative Eventing system configuration #685

Merged
merged 23 commits into from
Nov 9, 2024
Merged
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
a6c5132
kie-kogito-docs-664: Create a guide for the SonataFlow Operator drive…
wmedvede Nov 4, 2024
8160cbe
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
8b9ed74
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
598abdd
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
627918f
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
3c4d27f
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
24078b1
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
6caf5a4
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
7927a69
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
ca81089
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
eb7b63a
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
ee2bb86
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
59c07f2
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
6160c42
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
164b375
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
6438bca
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
ef22cee
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
5df4016
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
448eac9
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
f62fde9
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
2ce0992
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
1103859
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
fe57416
Update serverlessworkflow/modules/ROOT/pages/cloud/operator/supportin…
wmedvede Nov 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,241 @@ When you use the common PostgreSQL configuration, the database schema for each s
For example, `sonataflow-platform-example-data-index-service`.
====

== Configuring the supporting services Eventing system

In general, the following events are produced in a {product_name} installation:

* Workflow outgoing and incoming business events.
* {product_name} system events sent from the workflow to the Data Index and Job Service respectively.
* {product_name} system events sent from the Jobs Service to the Data Index Service.

The {operator_name} is designed to use the link:{knative_eventing_url}[Knative Eventing] system to resolve all the event communication between these services.

[NOTE]
====
In a regular {product_name} installation, the preferred method is to use the <<platform-scoped-eventing-system-configuration, Platform scoped Eventing System Configuration>>, while the <<service-scoped-eventing-system-configuration, Service scoped Eventing System configuration>> is reserved only for advanced use cases.
====

[#platform-scoped-eventing-system-configuration]
=== Platform-scoped Eventing system configuration

To configure a platform-scoped eventing system, you must use the field `spec.eventing.broker.ref` in the `SonataFlowPlatform` CR to refer to a Knative Eventing Broker.

This information signals the {operator_name} to automatically link the supporting services to `produce` and `consume` the events by using that Broker.

Additionally, workflows deployed in that namespace, that don't provide a custom eventing system configuration, will be linked to that Broker.
// TODO, uncomment when the workflows section is in.
// For more information about configuring the workflow eventing system, xref:cloud/operator/configuring-workflow-eventing-system.adoc[see].

The following `SonataFlowPlatform` CR fragment shows an example of such configuration:

.Platform scoped eventing system configuration example
[source,yam]
----
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
name: sonataflow-platform-example
namespace: example-namespace
spec:
eventing:
broker:
ref:
name: example-broker <1>
namespace: example-broker-namespace <2>
apiVersion: eventing.knative.dev/v1
kind: Broker
----

<1> Name of the Knative Eventing Broker.
<2> Optional: Defines the namespace of the Knative Eventing Broker. Defaults to the SonataFlowPlatform namespace. In general, we recommend creating the Knative Eventing Broker in the same namespace as the SonataFlowPlatform.

[NOTE]
====
In production environments, you must use a production-ready broker, like the link:{knative_eventing_kafka_broker_url}[Knative Kafka Broker].
====

[#service-scoped-eventing-system-configuration]
=== Service-scoped Eventing system configuration

A service scoped eventing system configuration provides the ability to do a fine-grained configuration of the Eventing system for the <<data-index-eventing-system-configuration, Data Index>> or the <<jos-service-eventing-system-configuration, Jobs Service>>.

[NOTE]
====
In a regular {product_name} installation, the preferred method is to use a <<platform-scoped-eventing-system-configuration, Platform-scoped Eventing System Configuration>>, while the service-scoped configuration is reserved only for advanced use cases.
====

[#data-index-eventing-system-configuration]
=== Data Index Eventing system configuration

To configure a service-scoped eventing system for the Data Index, you must use the field `spec.services.dataIndex.source.ref` in the `SonataFlowPlatform` CR to refer to a specific Knative Eventing Broker.

This information signals the {operator_name} to automatically link the Data Index to `consume` the {product_name} system events from that Broker.

.Data Index service scoped eventing system configuration example
[source,yam]
----
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
name: sonataflow-platform-example
spec:
services:
dataIndex:
source:
ref:
name: data-index-source-example-broker <1>
namespace: data-index-source-example-broker-namespace <2>
apiVersion: eventing.knative.dev/v1
kind: Broker
----

<1> Name of the Knative Eventing Broker to `consume` events from.
<2> Optional: Defines the namespace of the Knative Eventing Broker. Defaults to the SonataFlowPlatform namespace. In general, we recommend creating the Knative Eventing Broker in the same namespace as the SonataFlowPlatform.

[NOTE]
====
In production environments, you must use a production-ready broker, like the link:{knative_eventing_kafka_broker_url}[Knative Kafka Broker].
====

[#jos-service-eventing-system-configuration]
=== Jobs Service Eventing system configuration

To configure a service-scoped eventing system for the Jobs Service, you must use the fields `spec.services.jobService.source.ref` and `spec.services.jobService.sink.ref` in the `SonataFlowPlatform` CR.

This information signals the {operator_name} to automatically link the Jobs Service to `consume` and `produce` the {product_name} system events from that configuration respectively.

.Jobs Service scoped eventing system configuration example
[source,yam]
----
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
name: sonataflow-platform-example
spec:
services:
jobService:
source:
ref:
name: jobs-service-source-example-broker <1>
namespace: jobs-service-source-example-broker-namespace <2>
apiVersion: eventing.knative.dev/v1
kind: Broker
sink:
ref:
name: jobs-service-sink-example-broker <3>
namespace: jobs-service-sink-example-broker-namespace <4>
apiVersion: eventing.knative.dev/v1
kind: Broker
----

<1> Name of the Knative Eventing Broker to `consume` events from.
<2> Optional: Defines the namespace of the Knative Eventing Broker. Defaults to the SonataFlowPlatform namespace. In general, we recommend creating the Knative Eventing Broker in the same namespace as the SonataFlowPlatform.
<3> Name of the Knative Eventing Broker to `produce` events on.
<4> Optional: Defines the namespace of the Knative Eventing Broker. Defaults to the SonataFlowPlatform namespace. In general, we recommend creating the Knative Eventing Broker in the same namespace as the SonataFlowPlatform.

[NOTE]
====
In production environments, you must use production-ready brokers, like the link:{knative_eventing_kafka_broker_url}[Knative Kafka Broker].
====

[#cluster-scoped-eventing-system-configuration]
=== Cluster-scoped Eventing system configuration

When you use a <<cluster-scoped-deployment>> deployment, the supporting services are automatically linked to the `Broker` configured in the `SonataFlowPlatform` CR referred to by the `SonataFlowClusterPlatform` CR.

=== Eventing system configuration precedence rules

To configure the eventing system for a supporting service, the {operator_name} use the following precedence rules:

* If the supporting service has a configured eventing system, by using any of the <<data-index-eventing-system-configuration>> or <<jos-service-eventing-system-configuration>> respectively, that configuration applies.

* If the `SonataFlowPlatform` CR enclosing the supporting service, is configured with a <<platform-scoped-eventing-system-configuration>>, that configuration applies.

* If the current cluster, is configured with a <<cluster-scoped-eventing-system-configuration>>, that configuration apply.

* If none of the previous configurations exists, the supporting service is configured to produce direct HTTP calls to deliver events.

=== Eventing System linking objects

The linking of the supporting services with the Eventing System is produced by using Knative Eventing SinkBindings and Triggers.
These objects are automatically created by the {operator_name}, and facilitate the events production and consumption from the supporting services.

The following example shows the Knative Native eventing objects created for the `SonataFlowPlatform` CR:

.SonataFlowPlatform eventing system configuration example
[source,yaml]
----
apiVersion: sonataflow.org/v1alpha08
kind: SonataFlowPlatform
metadata:
name: sonataflow-platform-example
namespace: example-namespace
spec:
eventing:
broker:
ref:
name: example-broker <1>
apiVersion: eventing.knative.dev/v1
kind: Broker
services:
dataIndex: <2>
enabled: true
jobService: <3>
enabled: true
----

<1> Platform Broker configuration used by the Data Index, Jobs Service, and the workflows if not overridden.
<2> Data Index ephemeral deployment.
<3> Jobs Service ephemeral deployment.

.Knative Kafka Broker example used by the SonataFlowPlatform
[source,yaml]
----
apiVersion: eventing.knative.dev/v1
kind: Broker
metadata:
annotations:
eventing.knative.dev/broker.class: Kafka <1>
name: example-broker
namespace: example-namespace
spec:
config:
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config
namespace: knative-eventing
----

<1> Use the Kafka class to create a Kafka Knative Broker

.Knative Eventing Triggers created for the Data Index and Jobs Service events consumption
[source,bash]
----
kn trigger list -n example-namespace

NAME BROKER SINK AGE CONDITIONS READY REASON
data-index-jobs-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True
data-index-process-definition-e48b4e4bf73e22b90ecf7e093ff6b1eaf example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True
data-index-process-error-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True
data-index-process-node-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True
data-index-process-sla-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True
data-index-process-state-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True
data-index-process-variable-ac727d6051750888dedb72f697737c0dfbf example-broker service:sonataflow-platform-example-data-index-service 106s 7 OK / 7 True

jobs-service-create-job-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-jobs-service 106s 7 OK / 7 True
jobs-service-delete-job-fbf285df-c0a4-4545-b77a-c232ec2890e2 example-broker service:sonataflow-platform-example-jobs-service 106s 7 OK / 7 True
----

.Knative Eventing SinkBinding created for the Jobs Service events production
[source,bash]
----
kn source list -n example-namespace

NAME TYPE RESOURCE SINK READY
sonataflow-platform-example-jobs-service-sb SinkBinding sinkbindings.sources.knative.dev broker:example-broker True
----

== Advanced Supporting Services Configurations

To configure the advanced options for any of the supporting services you must use the `podTemplate` field respectively, for example `dataIndex.podTemplate`:
Expand Down