Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TH2-5226] infra-operator periodically deletes redundant resources from RabbitMQ instead of remove on start #119

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
03123a6
[TH2-5226] moved kotlin files to the src/kotlin dir
Nikita-Smirnov-Exactpro Aug 7, 2024
a19d4b5
[TH2-5226] Added RabbitMQGcTask
Nikita-Smirnov-Exactpro Aug 8, 2024
04e15b9
[TH2-5226] Updated libraries
Nikita-Smirnov-Exactpro Aug 8, 2024
a565d2d
[TH2-5226] Implemented RabbitMQGcTask.deleteRedundantExchanges
Nikita-Smirnov-Exactpro Aug 8, 2024
17879e0
[TH2-5226] added configure logging method
Nikita-Smirnov-Exactpro Aug 8, 2024
1add5b3
[TH2-5226] Refactored
Nikita-Smirnov-Exactpro Aug 8, 2024
98b1655
[TH2-5226] added wrapper closure into build script
Nikita-Smirnov-Exactpro Aug 9, 2024
d34bbb3
[TH2-5226] moved kotlin files to the src/kotlin dir
Nikita-Smirnov-Exactpro Aug 7, 2024
9391349
[TH2-5226] Added `rabbitMQManagement.cleanUpOnStart` option
Nikita-Smirnov-Exactpro Aug 13, 2024
bd6c67b
[TH2-5226] Corrected GitHub workflow
Nikita-Smirnov-Exactpro Aug 13, 2024
3e4130c
Merge remote-tracking branch 'origin/TH2-5226-hotfix-2' into TH2-5226
Nikita-Smirnov-Exactpro Aug 14, 2024
0b04c17
[TH2-5226] registered exception handler for informers
Nikita-Smirnov-Exactpro Aug 15, 2024
c4041b9
[TH2-5226] added deleteRabbitMQRubbish method
Nikita-Smirnov-Exactpro Aug 16, 2024
a6d9014
[TH2-5226] set true for clearOnStart by default
Nikita-Smirnov-Exactpro Aug 16, 2024
ada9bf0
[TH2-5226] refactored
Nikita-Smirnov-Exactpro Aug 16, 2024
34816d5
Merge remote-tracking branch 'origin/master' into TH2-5226
Nikita-Smirnov-Exactpro Aug 16, 2024
1bc6904
[TH2-5226] corrected after review
Nikita-Smirnov-Exactpro Aug 19, 2024
2e47d42
[TH2-5226] refactored after review
Nikita-Smirnov-Exactpro Aug 19, 2024
123585d
Update src/main/kotlin/com/exactpro/th2/infraoperator/util/RabbitMQUt…
Nikita-Smirnov-Exactpro Aug 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,20 +1,14 @@
name: Build and publish Docker images to github registry
name: Build and publish dev release Docker image to Github Container Registry ghcr.io

on:
push:
branches:
- master
- version-*
paths:
- gradle.properties
on: workflow_dispatch

jobs:
build:
uses: th2-net/.github/.github/workflows/compound-java.yml@main
with:
build-target: 'Docker'
runsOn: ubuntu-latest
gradleVersion: '7'
devRelease: true
createTag: true
docker-username: ${{ github.actor }}
secrets:
docker-password: ${{ secrets.GITHUB_TOKEN }}
Expand Down
15 changes: 15 additions & 0 deletions .github/workflows/build-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Build and publish release Docker image to Github Container Registry ghcr.io

on: workflow_dispatch

jobs:
build:
uses: th2-net/.github/.github/workflows/compound-java.yml@main
with:
build-target: 'Docker'
devRelease: false
createTag: true
docker-username: ${{ github.actor }}
secrets:
docker-password: ${{ secrets.GITHUB_TOKEN }}
nvd-api-key: ${{ secrets.NVD_APIKEY }}
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
name: Dev build and publish Docker images to github registry
name: Build and publish Docker image to Github Container Registry ghcr.io

on:
push:
branches-ignore:
- master
- version-*
- dev-version-*
- dependabot*
- dependabot**
paths-ignore:
- README.md

Expand All @@ -15,8 +14,6 @@ jobs:
uses: th2-net/.github/.github/workflows/compound-java-dev.yml@main
with:
build-target: 'Docker'
runsOn: ubuntu-latest
gradleVersion: '7'
docker-username: ${{ github.actor }}
# FIXME: strict scanner was disabled for 4.6.4 hotfix publishing and must be removed after that
strict-scanner: false
Expand Down
30 changes: 15 additions & 15 deletions .github/workflows/ci-unwelcome-words.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,19 @@ on:

jobs:
test:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
ref: ${{ github.sha }}
- name: Checkout tool
uses: actions/checkout@v2
with:
repository: exactpro-th2/ci-github-action
ref: master
token: ${{ secrets.PAT_CI_ACTION }}
path: ci-github-action
- name: Run CI action
uses: ./ci-github-action
with:
ref: ${{ github.sha }}
- uses: actions/checkout@v4
with:
ref: ${{ github.sha }}
- name: Checkout tool
uses: actions/checkout@v4
with:
repository: exactpro-th2/ci-github-action
ref: master
token: ${{ secrets.PAT_CI_ACTION }}
path: ci-github-action
- name: Run CI action
uses: ./ci-github-action
with:
ref: ${{ github.sha }}
16 changes: 6 additions & 10 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
FROM gradle:7.6-jdk11 AS build
FROM gradle:8.7-jdk11 AS build
ARG app_version=0.0.0
COPY ./ .
RUN gradle build -Prelease_version=${app_version}
RUN gradle --no-daemon clean build dockerPrepare -Prelease_version=${release_version}

RUN mkdir /home/app
RUN cp ./build/libs/*.jar /home/app/application.jar

FROM eclipse-temurin:11-alpine
COPY --from=build /home/app /home/app

WORKDIR /home/app/
ENTRYPOINT ["java", "-Dlog4j2.configurationFile=file:/var/th2/config/log4j2.properties", "-jar", "/home/app/application.jar"]
FROM adoptopenjdk/openjdk11:alpine
WORKDIR /home
COPY --from=build /home/gradle/build/docker .
ENTRYPOINT ["/home/service/bin/service", "/var/th2/config/log4j2.properties"]
17 changes: 14 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,6 @@ rabbitMQManagement:
persistence: true
# determines if the RabbitMQ resources are persistent or not

cleanUpOnStart: false
# if option is true, operator removes all queues and exchanges from RabbitMQ on start

schemaPermissions:
# this section describes what permissions schema RabbitMQ user will have on its own resources
# see RabbitMQ documentation to find out how permissions are described
Expand Down Expand Up @@ -147,5 +144,19 @@ openshift:

## Release notes

### 4.7.0
+ Improved clean rubbish from RabbitMQ on start to delete only redundant resources.
The `cleanUpOnStart` option has been removed, the clean rubbish function is enabled.
+ Migrated to th2 plugin `0.1.1`

+ Updated:
+ bom: `4.6.1`
+ kubernetes-client: `6.13.1`
+ force okhttp: `4.12.0`
+ force logging-interceptor: `4.12.0`
+ http-client: `5.2.0`
+ java-uuid-generator: `5.1.0`
+ kotlin-logging: `3.0.5`

### 4.6.4
+ Added `rabbitMQManagement.cleanUpOnStart` option
137 changes: 36 additions & 101 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,141 +1,76 @@
/*
* Copyright 2020-2021 Exactpro (Exactpro Systems Limited)
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

plugins {
id 'java'
id 'application'
id 'checkstyle'
id "io.gitlab.arturbosch.detekt" version "${detekt_version}"
id "org.jetbrains.kotlin.jvm" version "${kotlin_version}"
id "org.owasp.dependencycheck" version "${owaspVersion}"
alias(libs.plugins.detekt)
alias(libs.plugins.kotlin)
alias(libs.plugins.th2.component)
}

group = 'com.exactpro.th2'
version = release_version

kotlin {
jvmToolchain(11)
}

repositories {
mavenCentral()
}

checkstyle {
toolVersion = "10.12.0"
toolVersion = "10.12.4"
}

detekt {
buildUponDefaultConfig = true
autoCorrect = true
config = files("$rootDir/config/detekt/detekt.yml")
}

ext {
uuid_generator_version = '4.2.0'
okhttp_version = '4.10.0'
fabric8_version = '6.6.2'
rabbit_amqp_version = '5.16.0'
rabbit_http_version = '5.0.0'
kotlin_logging_version = '3.0.0' // 3.0.0 the las version supported 1.6.21
jetbrains_annotations_version = '24.0.1'
mockito_version = '3.11.2'
jupiter_version = '5.9.2'
guava_version = '32.0.1-jre'
snakeyaml_version = '2.0'
}

configurations.configureEach() {
resolutionStrategy {
force "com.google.guava:guava:$guava_version"
force "org.yaml:snakeyaml:$snakeyaml_version"
}
config.setFrom("$rootDir/config/detekt/detekt.yml")
}

dependencies {
api platform('com.exactpro.th2:bom:4.3.0')
implementation "com.fasterxml.jackson.dataformat:jackson-dataformat-yaml"
implementation group: 'com.fasterxml.jackson.module', name: 'jackson-module-kotlin'
implementation "com.fasterxml.uuid:java-uuid-generator:${uuid_generator_version}"

implementation "com.squareup.okhttp3:okhttp:${okhttp_version}"
implementation "com.fasterxml.jackson.module:jackson-module-kotlin"
implementation libs.java.uuid.generator

implementation "org.apache.commons:commons-text"

implementation "io.fabric8:kubernetes-client:${fabric8_version}"
implementation "com.rabbitmq:amqp-client:${rabbit_amqp_version}"
implementation "com.rabbitmq:http-client:${rabbit_http_version}"
implementation libs.kubernetes.client
implementation(libs.okhttp) {
because "okhttp:3.12.12 has vulnerabilities"
}
implementation(libs.logging.interceptor) {
because "logging-interceptor:3.12.12 has vulnerabilities"
}

implementation "org.slf4j:slf4j-api"
implementation "org.apache.logging.log4j:log4j-slf4j2-impl"
implementation "org.apache.logging.log4j:log4j-core"
implementation group: 'io.github.microutils', name: 'kotlin-logging', version: kotlin_logging_version
implementation "com.rabbitmq:amqp-client"
implementation libs.http.client

implementation "org.jetbrains:annotations:${jetbrains_annotations_version}"
implementation 'org.apache.logging.log4j:log4j-slf4j2-impl'
implementation 'org.apache.logging.log4j:log4j-core'
implementation libs.kotlin.logging
implementation "org.jetbrains:annotations"

implementation "io.prometheus:simpleclient"
implementation "io.prometheus:simpleclient_httpserver"
implementation "io.prometheus:simpleclient_hotspot"

testImplementation group: 'org.mockito', name: 'mockito-core', version: "${mockito_version}"

testImplementation "org.junit.jupiter:junit-jupiter-api:${jupiter_version}"
testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:${jupiter_version}"

detektPlugins("io.gitlab.arturbosch.detekt:detekt-formatting:${detekt_version}")
}

dependencyCheck {
formats=['SARIF', 'JSON', 'HTML']
failBuildOnCVSS=5
testImplementation libs.mockito.core
testImplementation libs.mockito.kotlin
testImplementation libs.junit.jupiter.api
testRuntimeOnly libs.junit.jupiter.engine

analyzers {
assemblyEnabled = false
nugetconfEnabled = false
nodeEnabled = false
}
detektPlugins libs.detekt.formatting
}

jar {
duplicatesStrategy = DuplicatesStrategy.INCLUDE
manifest {
attributes(
'Created-By': "${System.getProperty('java.version')} (${System.getProperty('java.vendor')})",
'Specification-Title': '',
'Specification-Vendor': 'Exactpro Systems LLC',
'Main-Class': 'com.exactpro.th2.infraoperator.Th2CrdController',
'Implementation-Title': project.archivesBaseName,
'Implementation-Vendor': 'Exactpro Systems LLC',
'Implementation-Vendor-Id': 'com.exactpro',
'Implementation-Version': project.version
)
}

from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
wrapper {
version '8.7'
distributionType Wrapper.DistributionType.BIN
}

test {
useJUnitPlatform()
}

compileKotlin {
kotlinOptions {
jvmTarget = "11"
}
}

compileTestKotlin {
kotlinOptions {
jvmTarget = "11"
}
}
dependencyCheck {
skipConfigurations += "checkstyle"
}
6 changes: 1 addition & 5 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1 @@
release_version = 4.6.4

kotlin_version = 1.6.21
detekt_version = 1.22.0
owaspVersion = 8.2.1
release_version = 4.7.0
27 changes: 27 additions & 0 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[versions]
kotlin = "1.8.22"
th2-plugin = "0.1.1"
jupiter = "5.10.3"
okhttp3 = "4.12.0"
detekt = "1.23.6"

[libraries]
kotlin-logging = { group = "io.github.microutils", name = "kotlin-logging", version = "3.0.5" }
okhttp = { group = "com.squareup.okhttp3", name = "okhttp", version.ref = "okhttp3" }
logging-interceptor = { group = "com.squareup.okhttp3", name = "logging-interceptor", version.ref = "okhttp3" }
kubernetes-client = { group = "io.fabric8", name = "kubernetes-client", version = "6.13.1" }
http-client = { group = "com.rabbitmq", name = "http-client", version = "5.2.0" }
java-uuid-generator = { group = "com.fasterxml.uuid", name = "java-uuid-generator", version = "5.1.0" }

mockito-core = { group = "org.mockito", name = "mockito-core", version = "5.12.0" }
mockito-kotlin = { group = "org.mockito.kotlin", name = "mockito-kotlin", version = "5.4.0" }
junit-jupiter-api = { group = "org.junit.jupiter", name = "junit-jupiter-api", version.ref = "jupiter" }
junit-jupiter-engine = { group = "org.junit.jupiter", name = "junit-jupiter-engine", version.ref = "jupiter" }

detekt-formatting = { group = "io.gitlab.arturbosch.detekt", name = "detekt-formatting", version.ref = "detekt" }

[plugins]
kotlin = { id = "org.jetbrains.kotlin.jvm", version.ref = "kotlin" }
detekt = { id = "io.gitlab.arturbosch.detekt", version.ref = "detekt" }

th2-component = { id = "com.exactpro.th2.gradle.component", version.ref = "th2-plugin" }
21 changes: 3 additions & 18 deletions gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,22 +1,7 @@
#
# Copyright 2020-2021 Exactpro (Exactpro Systems Limited)
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

#Mon May 04 17:09:53 MSK 2020
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.7-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.6-all.zip
Loading