forked from opensearch-project/index-management
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Error prevention stage 1 (opensearch-project#579)
* initial framework Signed-off-by: Joanne Wang <[email protected]> * Removed recursion from Explain Action to avoid stackoverflow in some situations (opensearch-project#419) Signed-off-by: Petar Dzepina <[email protected]> Signed-off-by: Joanne Wang <[email protected]> * enabled by default integrated Signed-off-by: Joanne Wang <[email protected]> * cleaned up comments and logs, created unit test and updated previous integration tests Signed-off-by: Joanne Wang <[email protected]> * added delete validation logic Signed-off-by: Joanne Wang <[email protected]> * fixed rollover validation unit tests Signed-off-by: Joanne Wang <[email protected]> * added validation info field to ManagedIndexMetaData Signed-off-by: Joanne Wang <[email protected]> * removed step context as input Signed-off-by: Joanne Wang <[email protected]> * added validationmetadata class Signed-off-by: Joanne Wang <[email protected]> * restored old integration tests and changed validation service output Signed-off-by: Joanne Wang <[email protected]> * before integrated validation meta data into managed index meta data Signed-off-by: Joanne Wang <[email protected]> * integrated validation meta data Signed-off-by: Joanne Wang <[email protected]> * working version Signed-off-by: Joanne Wang <[email protected]> * added validation mapping Signed-off-by: Joanne Wang <[email protected]> * fixed integ tests Signed-off-by: Joanne Wang <[email protected]> * renamed some values Signed-off-by: Joanne Wang <[email protected]> * before removing from managed index meta data Signed-off-by: Joanne Wang <[email protected]> * created validation result object in explain Signed-off-by: Joanne Wang <[email protected]> * testing Signed-off-by: Joanne Wang <[email protected]> * run fails Signed-off-by: Joanne Wang <[email protected]> * integration test for delete + added framework for force merge Signed-off-by: Joanne Wang <[email protected]> * removed step validation metadata and still testing explain results Signed-off-by: Joanne Wang <[email protected]> * before removing from managed index runner Signed-off-by: Joanne Wang <[email protected]> * removed from managed index runner Signed-off-by: Joanne Wang <[email protected]> * clean up and tests Signed-off-by: Joanne Wang <[email protected]> * all validation tests pass Signed-off-by: Joanne Wang <[email protected]> * removed validation result from all managed index meta data Signed-off-by: Joanne Wang <[email protected]> * restored old IT tests Signed-off-by: Joanne Wang <[email protected]> * fixed it tests, set explain validation to false Signed-off-by: Joanne Wang <[email protected]> * clean up Signed-off-by: Joanne Wang <[email protected]> * Implemented replica_count, open, read_only, read_write Signed-off-by: Angie Zhang <[email protected]> * Implemented replica_count, open, read_only, read_write Signed-off-by: Angie Zhang <[email protected]> * Fix Test cases Signed-off-by: Angie Zhang <[email protected]> * Fix Test cases Signed-off-by: Angie Zhang <[email protected]> * Fix messages Signed-off-by: Angie Zhang <[email protected]> * Fix comments; set validation disabled by default Signed-off-by: Angie Zhang <[email protected]> * Rename validation_service to action_validation; Fix some detekt issues Signed-off-by: Angie Zhang <[email protected]> Signed-off-by: Joanne Wang <[email protected]> Signed-off-by: Petar Dzepina <[email protected]> Signed-off-by: Angie Zhang <[email protected]> Co-authored-by: Joanne Wang <[email protected]> Co-authored-by: Petar <[email protected]> Co-authored-by: Joanne Wang <[email protected]>
- Loading branch information
4 people
authored
Nov 3, 2022
1 parent
50072af
commit 2438f48
Showing
60 changed files
with
1,904 additions
and
50 deletions.
There are no files selected for viewing
46 changes: 46 additions & 0 deletions
46
spi/src/main/kotlin/org.opensearch.indexmanagement.spi/indexstatemanagement/Validate.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.indexmanagement.spi.indexstatemanagement | ||
|
|
||
| import org.opensearch.cluster.service.ClusterService | ||
| import org.opensearch.common.io.stream.StreamInput | ||
| import org.opensearch.common.io.stream.StreamOutput | ||
| import org.opensearch.common.io.stream.Writeable | ||
| import org.opensearch.common.settings.Settings | ||
| import org.opensearch.monitor.jvm.JvmService | ||
| import java.util.Locale | ||
|
|
||
| abstract class Validate( | ||
| val settings: Settings, | ||
| val clusterService: ClusterService, | ||
| val jvmService: JvmService | ||
| ) { | ||
|
|
||
| var validationStatus = ValidationStatus.PASSED | ||
| var validationMessage: String? = "Starting Validation" | ||
|
|
||
| abstract fun execute(indexName: String): Validate | ||
|
|
||
| enum class ValidationStatus(val status: String) : Writeable { | ||
| PASSED("passed"), | ||
| RE_VALIDATING("re_validating"), | ||
| FAILED("failed"); | ||
|
|
||
| override fun toString(): String { | ||
| return status | ||
| } | ||
|
|
||
| override fun writeTo(out: StreamOutput) { | ||
| out.writeString(status) | ||
| } | ||
|
|
||
| companion object { | ||
| fun read(streamInput: StreamInput): ValidationStatus { | ||
| return valueOf(streamInput.readString().uppercase(Locale.ROOT)) | ||
| } | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
94 changes: 94 additions & 0 deletions
94
.../kotlin/org.opensearch.indexmanagement.spi/indexstatemanagement/model/ValidationResult.kt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| /* | ||
| * Copyright OpenSearch Contributors | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
|
|
||
| package org.opensearch.indexmanagement.spi.indexstatemanagement.model | ||
|
|
||
| import org.opensearch.common.Strings | ||
| import org.opensearch.common.io.stream.StreamInput | ||
| import org.opensearch.common.io.stream.StreamOutput | ||
| import org.opensearch.common.io.stream.Writeable | ||
| import org.opensearch.common.xcontent.LoggingDeprecationHandler | ||
| import org.opensearch.common.xcontent.NamedXContentRegistry | ||
| import org.opensearch.common.xcontent.ToXContent | ||
| import org.opensearch.common.xcontent.ToXContentFragment | ||
| import org.opensearch.common.xcontent.XContentBuilder | ||
| import org.opensearch.common.xcontent.XContentParser | ||
| import org.opensearch.common.xcontent.XContentParserUtils | ||
| import org.opensearch.common.xcontent.XContentType | ||
| import org.opensearch.indexmanagement.spi.indexstatemanagement.Validate | ||
| import java.io.ByteArrayInputStream | ||
| import java.nio.charset.StandardCharsets | ||
| import java.util.Locale | ||
|
|
||
| data class ValidationResult( | ||
| val validationMessage: String, | ||
| val validationStatus: Validate.ValidationStatus | ||
| ) : Writeable, ToXContentFragment { | ||
|
|
||
| override fun writeTo(out: StreamOutput) { | ||
| out.writeString(validationMessage) | ||
| validationStatus.writeTo(out) | ||
| } | ||
|
|
||
| override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { | ||
| builder | ||
| .field(VALIDATION_MESSAGE, validationMessage) | ||
| .field(VALIDATION_STATUS, validationStatus.toString()) | ||
| return builder | ||
| } | ||
|
|
||
| fun getMapValueString(): String { | ||
| return Strings.toString(this, false, false) | ||
| } | ||
|
|
||
| companion object { | ||
| const val VALIDATE = "validate" | ||
| const val VALIDATION_MESSAGE = "validation_message" | ||
| const val VALIDATION_STATUS = "validation_status" | ||
|
|
||
| fun fromStreamInput(si: StreamInput): ValidationResult { | ||
| val validationMessage: String? = si.readString() | ||
| val validationStatus: Validate.ValidationStatus? = Validate.ValidationStatus.read(si) | ||
|
|
||
| return ValidationResult( | ||
| requireNotNull(validationMessage) { "$VALIDATION_MESSAGE is null" }, | ||
| requireNotNull(validationStatus) { "$VALIDATION_STATUS is null" } | ||
| ) | ||
| } | ||
|
|
||
| fun fromManagedIndexMetaDataMap(map: Map<String, String?>): ValidationResult? { | ||
| val stepJsonString = map[VALIDATE] | ||
| return if (stepJsonString != null) { | ||
| val inputStream = ByteArrayInputStream(stepJsonString.toByteArray(StandardCharsets.UTF_8)) | ||
| val parser = XContentType.JSON.xContent().createParser(NamedXContentRegistry.EMPTY, LoggingDeprecationHandler.INSTANCE, inputStream) | ||
| parser.nextToken() | ||
| parse(parser) | ||
| } else { | ||
| null | ||
| } | ||
| } | ||
|
|
||
| fun parse(xcp: XContentParser): ValidationResult { | ||
| var validationMessage: String? = null | ||
| var validationStatus: Validate.ValidationStatus? = null | ||
|
|
||
| XContentParserUtils.ensureExpectedToken(XContentParser.Token.START_OBJECT, xcp.currentToken(), xcp) | ||
| while (xcp.nextToken() != XContentParser.Token.END_OBJECT) { | ||
| val fieldName = xcp.currentName() | ||
| xcp.nextToken() | ||
|
|
||
| when (fieldName) { | ||
| VALIDATION_MESSAGE -> validationMessage = xcp.text() | ||
| VALIDATION_STATUS -> validationStatus = Validate.ValidationStatus.valueOf(xcp.text().uppercase(Locale.ROOT)) | ||
| } | ||
| } | ||
|
|
||
| return ValidationResult( | ||
| requireNotNull(validationMessage) { "$VALIDATION_MESSAGE is null" }, | ||
| requireNotNull(validationStatus) { "$VALIDATION_STATUS is null" } | ||
| ) | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.