Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KOGITO-9971: [Guides] Dataindex deployment use cases with operator #519

Merged
merged 6 commits into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 4 additions & 1 deletion serverlessworkflow/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,10 @@
* Data Index
** xref:data-index/data-index-core-concepts.adoc[Core concepts]
** xref:data-index/data-index-service.adoc[Standalone service]
** xref:data-index/data-index-quarkus-extension.adoc[Quarkus Extension]
** xref:data-index/data-index-quarkus-extension.adoc[Quarkus Extensions]
** Operator
*** xref:data-index/data-index-usecase-singleton.adoc[]
*** xref:data-index/data-index-usecase-multi.adoc[]
//** Quarkus Extensions TODO: https://issues.redhat.com/browse/KOGITO-9463
* Use Cases
** xref:use-cases/orchestration-based-saga-pattern.adoc[Saga Orchestration]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@

link:{flow_examples_operator_url}/tree/main/infra/dataindex[Here] you can find the infrastructure kustomization required to deploy {data_index_ref} service and a postgresql database explained in this use case.

Thas folder woud contain the following four files:
domhanak marked this conversation as resolved.
Show resolved Hide resolved

* kustomization.yaml
* 01-postgres.yaml
* 02-dataindex.yaml
* application.properties

domhanak marked this conversation as resolved.
Show resolved Hide resolved
.`kustomization.yaml` resources that deploy {data_index_ref} deployment with persistence to a postgresql database
[source,yaml,subs="attributes+"]
----
resources:
- 01-postgres.yaml <1>
- 02-dataindex.yaml <2>

secretGenerator:
- name: postgres-secrets
literals:
- POSTGRES_USER=sonataflow
- POSTGRES_PASSWORD=sonataflow
- POSTGRES_DB=sonataflow
- PGDATA=/var/lib/postgresql/data/mydata

configMapGenerator:
- name: dataindex-properties
files:
- application.properties
----
<1> Postgres database deployment
<2> {data_index_ref} deployment

.`01_postgres.yaml` that deploys Postgresql database
[source,yaml,subs="attributes+"]
----
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
labels:
app.kubernetes.io/name: postgres
name: postgres-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/name: postgres
name: postgres
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: postgres
template:
metadata:
labels:
app.kubernetes.io/name: postgres
spec:
containers:
- name: postgres
image: postgres:13.2-alpine
imagePullPolicy: 'IfNotPresent'
ports:
- containerPort: 5432
volumeMounts:
- name: storage
mountPath: /var/lib/postgresql/data
envFrom:
- secretRef:
name: postgres-secrets
readinessProbe:
exec:
command: ["pg_isready"]
initialDelaySeconds: 15
timeoutSeconds: 2
livenessProbe:
exec:
command: ["pg_isready"]
initialDelaySeconds: 15
timeoutSeconds: 2
resources:
limits:
memory: "256Mi"
cpu: "500m"
volumes:
- name: storage
persistentVolumeClaim:
claimName: postgres-pvc
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: postgres
name: postgres
spec:
selector:
app.kubernetes.io/name: postgres
ports:
- port: 5432
----

.`02-dataindex.yaml` that deploys {data_index_ref} with persistence to the previous defined postgresql database
[source,yaml,subs="attributes+"]
----
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/name: data-index-service-postgresql
name: data-index-service-postgresql
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: data-index-service-postgresql
template:
metadata:
labels:
app.kubernetes.io/name: data-index-service-postgresql
spec:
containers:
- name: data-index-service-postgresql
image: quay.io/kiegroup/kogito-data-index-postgresql:latest
imagePullPolicy: Always
resources:
limits:
memory: "256Mi"
cpu: "500m"
ports:
- containerPort: 8080
name: http
protocol: TCP
env:
- name: KOGITO_DATA_INDEX_QUARKUS_PROFILE
value: http-events-support
- name: KUBERNETES_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: QUARKUS_DATASOURCE_USERNAME
valueFrom:
secretKeyRef:
key: POSTGRES_USER
name: postgres-secrets
- name: QUARKUS_DATASOURCE_PASSWORD
valueFrom:
secretKeyRef:
key: POSTGRES_PASSWORD
name: postgres-secrets
volumeMounts:
- name: application-config
mountPath: "/home/kogito/config"
livenessProbe:
failureThreshold: 3
httpGet:
path: /q/health/live
port: 8080
scheme: HTTP
initialDelaySeconds: 0
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 10
readinessProbe:
failureThreshold: 3
httpGet:
path: /q/health/ready
port: 8080
scheme: HTTP
initialDelaySeconds: 0
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 10
volumes:
- name: application-config
configMap:
name: dataindex-properties
initContainers:
- name: init-postgres
image: registry.access.redhat.com/ubi9/ubi-minimal:latest
imagePullPolicy: IfNotPresent
command: ['sh', '-c', 'until (echo 1 > /dev/tcp/postgres.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local/5432) >/dev/null 2>&1; do echo "Waiting for postgres server"; sleep 3; done;']
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: data-index-service-postgresql
name: data-index-service-postgresql
spec:
ports:
- name: http
port: 80
targetPort: 8080
selector:
app.kubernetes.io/name: data-index-service-postgresql
type: NodePort
----
.`application.properties` referenced by `kustomization.yaml`
[source,properties]
----
quarkus.http.port=8080
quarkus.http.cors=true
quarkus.http.cors.origins=/.*/

quarkus.datasource.jdbc.url=jdbc:postgresql://postgres:5432/sonataflow?currentSchema=data-index-service
quarkus.hibernate-orm.database.generation=update
quarkus.flyway.migrate-at-start=true

# Disable kafka client health check since the quarkus-http connector is being used instead.
quarkus.smallrye-health.check."io.quarkus.kafka.client.health.KafkaHealthCheck".enabled=false
----
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@

.Prerequisites
* Minikube installed with `registry` addon enabled
* `kubectl` {kubectl_prereq}
* SonataFlow operator installed if workflows are deployed. To install the operator you can see xref:cloud/operator/install-serverless-operator.adoc[].

[NOTE]
====
We recommend that you start Minikube with the following parameters, note that the `registry` addon must be enabled.

[source,shell]
----
minikube start --cpus 4 --memory 10240 --addons registry --addons metrics-server --insecure-registry "10.0.0.0/24" --insecure-registry "localhost:5000"
----

To verify that the registry addon was property added you can execute this command:

[source,shell]
----
minikube addons list | grep registry
----

----
| registry | minikube | enabled ✅ | Google |
| registry-aliases | minikube | disabled | 3rd party (unknown) |
| registry-creds | minikube | disabled | 3rd party (UPMC Enterprises) |
----
====
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
[[querying-dataindex-minikube]]
== Querying Data Index service on Minikube

You can use the public Data Index endpoint to play around with the GraphiQL interface.

.Procedure
This procedure apply to all use cases with that deploys the Data Index Service.

* Get the Data Index Url:
[source,shell]
----
minikube service data-index-service-postgresql --url -n my_usecase
----

* Open the GrahiqlUI

Using the url returned, open a browser window in the following url http://192.168.49.2:32409/graphiql/,

[NOTE]
====
that IP and port will be different in your installation, and don't forget to add the last slash "/" to the url, otherwise the GraphiqlUI won't be opened.
====


To see the process instances information you can execute this query:

[source,shell]
----
{
ProcessInstances {
id,
processId,
processName,
variables,
state,
endpoint,
serviceUrl,
start,
end
}
}
----

The results should be something like:

[source]
----
{
"data": {
"ProcessInstances": [
{
"id": "3ed8bf63-85c9-425d-9099-49bfb63608cb",
"processId": "greeting",
"processName": "workflow",
"variables": "{\"workflowdata\":{\"name\":\"John\",\"greeting\":\"Hello from JSON Workflow, \",\"language\":\"English\"}}",
"state": "COMPLETED",
"endpoint": "/greeting",
"serviceUrl": "http://greeting",
"start": "2023-09-13T06:59:24.319Z",
"end": "2023-09-13T06:59:24.400Z"
}
]
}
}
----

To see the jobs instances information, if any, you can execute this query:

[source]
----
{
Jobs {
id,
processId,
processInstanceId,
status,
expirationTime,
retries,
endpoint,
callbackEndpoint
}
}
----

The results should be something like:

[source]
----
{
"data": {
"Jobs": [
{
"id": "55c7aadb-3dff-4b97-af8e-cc45014b1c0d",
"processId": "callbackstatetimeouts",
"processInstanceId": "299886b7-2b78-4965-a701-16783c4162d8",
"status": "EXECUTED",
"expirationTime": null,
"retries": 0,
"endpoint": "http://jobs-service-postgresql/jobs",
"callbackEndpoint": "http://callbackstatetimeouts:80/management/jobs/callbackstatetimeouts/instances/299886b7-2b78-4965-a701-16783c4162d8/timers/-1"
}
]
}
}
----
Loading