Merge pull request #9 from th2-net/book-and-page

Migrated to Book and page concept

.github/workflows/build-dev-release.yml

@@ -0,0 +1,15 @@
+name: Build and publish dev release Docker image to Github Container Registry ghcr.io
+
+on: workflow_dispatch
+
+jobs:
+  build:
+    uses: th2-net/.github/.github/workflows/compound-java.yml@main
+    with:
+      build-target: 'Docker'
+      devRelease: true
+      createTag: true
+      docker-username: ${{ github.actor }}
+    secrets:
+      docker-password: ${{ secrets.GITHUB_TOKEN }}
+      nvd-api-key: ${{ secrets.NVD_APIKEY }}

.github/workflows/build-release.yml

@@ -0,0 +1,15 @@
+name: Build and publish release Docker image to Github Container Registry ghcr.io
+
+on: workflow_dispatch
+
+jobs:
+  build:
+    uses: th2-net/.github/.github/workflows/compound-java.yml@main
+    with:
+      build-target: 'Docker'
+      devRelease: false
+      createTag: true
+      docker-username: ${{ github.actor }}
+    secrets:
+      docker-password: ${{ secrets.GITHUB_TOKEN }}
+      nvd-api-key: ${{ secrets.NVD_APIKEY }}

.github/workflows/build-sanpshot.yml

@@ -0,0 +1,20 @@
+name: Build and publish Docker image to Github Container Registry ghcr.io
+
+on:
+  push:
+    branches-ignore:
+      - master
+      - version-*
+      - dependabot**
+    paths-ignore:
+      - README.md
+
+jobs:
+  build-job:
+    uses: th2-net/.github/.github/workflows/compound-java-dev.yml@main
+    with:
+      build-target: 'Docker'
+      docker-username: ${{ github.actor }}
+    secrets:
+      docker-password: ${{ secrets.GITHUB_TOKEN }}
+      nvd-api-key: ${{ secrets.NVD_APIKEY }}

.github/workflows/ci-unwelcome-words.yml

@@ -7,17 +7,17 @@ jobs:
   test:
     runs-on: ubuntu-20.04
     steps:
-      - uses: actions/checkout@v2
-        with:
-          ref: ${{ github.sha }}
-      - name: Checkout tool
-        uses: actions/checkout@v2
-        with:
-          repository: exactpro-th2/ci-github-action
-          ref: master
-          token: ${{ secrets.PAT_CI_ACTION }}
-          path: ci-github-action
-      - name: Run CI action
-        uses: ./ci-github-action
-        with:
-          ref: ${{ github.sha }}
+    - uses: actions/checkout@v4
+      with:
+        ref: ${{ github.sha }}
+    - name: Checkout tool
+      uses: actions/checkout@v4
+      with:
+        repository: exactpro-th2/ci-github-action
+        ref: master
+        token: ${{ secrets.PAT_CI_ACTION }}
+        path: ci-github-action
+    - name: Run CI action
+      uses: ./ci-github-action
+      with:
+        ref: ${{ github.sha }}

Dockerfile

@@ -1,4 +1,4 @@
-FROM gradle:6.8-jdk11 AS build
+FROM gradle:8.11-jdk11 AS build
 ARG release_version
 ARG nexus_url
 ARG nexus_user

README.md

@@ -1,61 +1,172 @@
-# Healer data processor (0.0.2)
+# Event healer processor (0.0.3)
 
-Healer data processor fixes wrong status of events. When a child event has a "failed" status and its parent
+Event healer processor fixes wrong status of events. When a child event has a "failed" status and its parent
 has a "success" status, the status of the parent is wrong. Healer finds the parent event and makes its status "failed", too.
 
+Sometimes parent event doesn't exist in cradle when the healer process its child event with "failed" states. 
+In this case processor repeats searching of the parent event with interval according to `processorSettings`
+
 ## Configuration
 
-There is an example of full configuration for the data processor
+There is an example of full configuration (infra-2.1) for the event healer processor
 
 ```yaml
-apiVersion: th2.exactpro.com/v1
+apiVersion: th2.exactpro.com/v2
 kind: Th2Box
 metadata:
   name: event-healer
 spec:
-  image-name: ghcr.io/th2-net/th2-crawler-event-healer
-  image-version: <verison>
+  imageName: ghcr.io/th2-net/th2-crawler-event-healer
+  imageVersion: 0.0.3-dev
   type: th2-conn
-  custom-config:
-    name: test-event-healer
-    version: 1.0.0
-    maxCacheCapacity: 1000
   pins:
-    - name: server
-      connection-type: grpc
-  extended-settings:
+    grpc:
+      client:
+        - name: to_data_provider
+          serviceClass: com.exactpro.th2.dataprovider.lw.grpc.DataProviderService
+          linkTo:
+            - box: lw-data-provider-grpc
+              pin: server
+        - name: to_data_provider_stream
+          serviceClass: com.exactpro.th2.dataprovider.lw.grpc.QueueDataProviderService
+          linkTo:
+            - box: lw-data-provider-grpc
+              pin: server
+  customConfig:
+    crawler:
+      from: 2024-11-14T00:00:00.00Z
+      intervalLength: PT10M
+      syncInterval: PT10M
+      awaitTimeout: 10
+      awaitUnit: SECONDS
+      events:
+        bookToScopes:
+          test_book: [ "script" ]
+    processorSettings:
+      maxCacheCapacity: 1024
+      updateUnsubmittedEventInterval: 1
+      updateUnsubmittedEventTimeUnit: SECONDS
+      updateUnsubmittedEventAttempts: 100
+  extendedSettings:
+    envVariables:
+      JAVA_TOOL_OPTIONS: >
+        -XX:+ExitOnOutOfMemoryError
+        -XX:+UseContainerSupport
+        -Dlog4j2.shutdownHookEnabled=false
+        -Xlog:gc,gc+heap*,gc+start,gc+metaspace::utc,level,tags
+        -XX:MaxRAMPercentage=38
+        -XX:MaxMetaspaceSize=80M
+        -XX:CompressedClassSpaceSize=12M
+        -XX:ReservedCodeCacheSize=35M
+        -XX:MaxDirectMemorySize=15M
+        -Ddatastax-java-driver.advanced.connection.init-query-timeout="5000 milliseconds"
+        -Ddatastax-java-driver.basic.request.timeout="10 seconds"
+    resources:
+      limits:
+        memory: 250Mi
+        cpu: 200m
+      requests:
+        memory: 100Mi
+        cpu: 50m
+```
+
+Please note the `th2-lw-data-provider` worked in gRPC mode is required for the current processor.
+The main configuration of `th2-lw-data-provider` described below, full documentation can be found by the (link)[https://github.com/th2-net/th2-lw-data-provider]
+
+```yaml
+apiVersion: th2.exactpro.com/v2
+kind: Th2CoreBox
+metadata:
+  name: lw-data-provider-grpc
+spec:
+  imageName: ghcr.io/th2-net/th2-lw-data-provider
+  imageVersion: 2.12.0-dev
+  type: th2-rpt-data-provider
+  customConfig:
+    grpcBackPressure: true
+    hostname: 0.0.0.0
+    port: 8080
+    mode: GRPC
+  extendedSettings:
+    envVariables:
+      JAVA_TOOL_OPTIONS: >
+        -XX:+ExitOnOutOfMemoryError
+        -XX:+UseContainerSupport 
+        -Dlog4j2.shutdownHookEnabled=false
+        -Xlog:gc,gc+heap*,gc+start,gc+metaspace::utc,level,tags
+        -XX:MaxRAMPercentage=85
+        -Ddatastax-java-driver.advanced.connection.init-query-timeout="5000 milliseconds"
+        -Ddatastax-java-driver.basic.request.timeout="10 seconds"
+    resources:
+      limits:
+        memory: 1000Mi
+        ephemeral-storage: 500Mi
+        cpu: 1000m
+      requests:
+        memory: 300Mi
+        ephemeral-storage: 500Mi
+        cpu: 50m
     service:
       enabled: true
-      type: ClusterIP
-      endpoints:
-        - name: 'grpc'
-          targetPort: 8080
-    envVariables:
-      JAVA_TOOL_OPTIONS: '-XX:+ExitOnOutOfMemoryError -XX:+UseContainerSupport -XX:MaxRAMPercentage=85'
-  resources:
-    limits:
-      memory: 200Mi
-      cpu: 200m
-    requests:
-      memory: 100Mi
-      cpu: 50m
+      clusterIP:
+        - name: grpc
+          containerPort: 8080
+  pins:
+    mq:
+      subscribers:
+        - name: from_codec_transport
+          attributes:
+            - from_codec
+            - transport-group
+            - subscribe
+        - name: from_codec_proto
+          attributes:
+            - from_codec
+            - parsed
+            - subscribe
+      publishers:
+        - name: to_codec
+          attributes:
+            - to_codec
+            - transport-group
+            - publish
+    grpc:
+      server:
+        - name: server
+          serviceClasses:
+            - com.exactpro.th2.dataprovider.lw.grpc.DataProviderService
+            - com.exactpro.th2.dataprovider.lw.grpc.QueueDataProviderService
 ```
 
-### Parameters description
+### Processor settings description
 
-+ name - the data processor name
-+ version - the data processor version
-+ maxCacheCapacity - the maximum capacity of the cache that stores 
-  events processed by Healer. Caching events is useful in order to 
++ **maxCacheCapacity** (_**1024** by default_) - the maximum capacity of the cache that stores
+  events processed by Healer. Caching events is useful in order to
   avoid their repeated retrieval from Cradle.
-  After reaching the maximum capacity, the least recent accessed event 
+  After reaching the maximum capacity, the least recent accessed event
   from the cache will be removed, so no overflow occurs.
-
-See th2-crawler in **Useful links** section to see the goal of *name* and *version* 
-parameters.
++ **updateUnsubmittedEventInterval** (_**1** by default_) - value of interval between attempts for updating parent event status if it doesn't exist
++ **updateUnsubmittedEventTimeUnit** (_**SECONDS** by default_) - time unit of interval between attempts for updating parent event status if it doesn't exist
++ **updateUnsubmittedEventAttempts** (_**100**_ by default_) - number of attempts to update parent event status if it doesn't exist
 
 # Useful links
 
+The event-healer based on the th2-processor-core project, you can look at its readme to read about processor configuration.   
+
++ th2-processor-core - https://github.com/th2-net/th2-processor-core-j
++ th2-lw-data-provider - https://github.com/th2-net/th2-lw-data-provider
 + th2-common - https://github.com/th2-net/th2-common-j
 
-+ th2-crawler - https://github.com/th2-net/th2-crawler
+## Release notes
+
+### 0.0.3
+* Migrated to th2 gradle plugin `0.1.4` (based on th2-bom: `4.8.0`)
+* Updated:
+  * common: `5.14.0-dev`
+  * common-utils: `2.3.0-dev`
+  * processor-core: `0.3.0-dev`
+  * cradle-cassandra: `5.4.4-dev`
+  * auto-service: `1.1.1`
+  * caffeine: `3.1.8`
+  * kotlin-logging: `3.0.5`
+  * kotlin: `1.8.22`