Fix dissect ingest processor parsing empty brackets failed #9255

gaobinlong · 2023-08-11T06:18:28Z

Description

Fix the bug of dissect ingest processor parse empty brackets failed, and add some unit test code and yaml rest test.
The bug is caused by that when walking through the original string, the cursor i cannot move to next when one or more matches are found because i already jumps to then end of the match(i += lookAheadMatches), so for loop should be changed to while loop.

Related Issues

Relates to #7223.

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Gao Binlong <[email protected]>

github-actions · 2023-08-11T06:45:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/22437/
CommitID: 24f2ae9
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

opensearch-trigger-bot · 2023-08-11T06:45:58Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer.git]
Compatible components: [https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git]

BUILD SUCCESSFUL in 21m 53s

opensearch-trigger-bot · 2023-08-11T06:49:24Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]
Compatible components: [https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 22m 33s

opensearch-trigger-bot · 2023-08-11T06:49:39Z

Compatibility status:



> Task :checkCompatibility
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/reporting.git]
Compatible components: [https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

BUILD SUCCESSFUL in 30m 43s

Signed-off-by: Gao Binlong <[email protected]>

github-actions · 2023-10-08T11:12:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/27414/
CommitID: 8a79835
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-08T11:37:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/27415/
CommitID: 5b8d247
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

gaobinlong · 2023-10-09T14:11:34Z

Both the failed test org.opensearch.persistent.PersistentTasksExecutorFullRestartIT.testFullClusterRestart and org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testUpdateOperations seem to be flaky tests.

dblock · 2023-10-10T21:19:46Z

Both the failed test org.opensearch.persistent.PersistentTasksExecutorFullRestartIT.testFullClusterRestart and org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testUpdateOperations seem to be flaky tests.

Find the GitHub issue for those/open one if it doesn't exist. Force push/rebase to retry gradle checks until a green one.

github-actions · 2023-10-11T10:55:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/27597/
CommitID: fff66b4

gaobinlong · 2023-10-11T14:04:38Z

The flaky tests have related issues: #5145, #10026. Now all checks have passed yet.

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

dblock · 2023-10-13T18:54:48Z

There was a CHANGELOG conflict, so I resolved it changing the language slightly to "Fix failure in dissect ingest processor parsing empty brackets", lmk if that's incorrect.

github-actions · 2023-10-13T19:28:02Z

Gradle Check (Jenkins) Run Completed with:

RESULT: ❌
URL: https://build.ci.opensearch.org/job/gradle-check/27848/
CommitID: b240237
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Gao Binlong <[email protected]>

github-actions · 2023-10-16T06:48:20Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/27915/
CommitID: 840fe0e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-17T03:10:50Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28084/
CommitID: 9c2514a
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-17T10:32:34Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testSearchAggregationWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.cluster.allocation.AwarenessAllocationIT.testThreeZoneOneReplicaWithForceZoneValueAndLoadAwareness

URL: https://build.ci.opensearch.org/job/gradle-check/28112/
CommitID: e03b2b1
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

* Fix dissect ingest processor parsing empty brackets failed Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Add assertion Signed-off-by: Gao Binlong <[email protected]> --------- Signed-off-by: Gao Binlong <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> (cherry picked from commit 7b62e2f) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…h-project#9255) * Fix dissect ingest processor parsing empty brackets failed Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Add assertion Signed-off-by: Gao Binlong <[email protected]> --------- Signed-off-by: Gao Binlong <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Siddhant Deshmukh <[email protected]>

…h-project#9255) * Fix dissect ingest processor parsing empty brackets failed Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Add assertion Signed-off-by: Gao Binlong <[email protected]> --------- Signed-off-by: Gao Binlong <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>

* Fix dissect ingest processor parsing empty brackets failed Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Add assertion Signed-off-by: Gao Binlong <[email protected]> --------- Signed-off-by: Gao Binlong <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> (cherry picked from commit 7b62e2f) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…10667) * Fix dissect ingest processor parsing empty brackets failed * Modify change log * Modify change log * Add assertion --------- (cherry picked from commit 7b62e2f) Signed-off-by: Gao Binlong <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]>

…h-project#9255) * Fix dissect ingest processor parsing empty brackets failed Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Modify change log Signed-off-by: Gao Binlong <[email protected]> * Add assertion Signed-off-by: Gao Binlong <[email protected]> --------- Signed-off-by: Gao Binlong <[email protected]> Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

Fix dissect ingest processor parsing empty brackets failed

29372cb

Signed-off-by: Gao Binlong <[email protected]>

gaobinlong added 2 commits August 11, 2023 14:23

Modify change log

24f2ae9

Signed-off-by: Gao Binlong <[email protected]>

Modify change log

c558848

Signed-off-by: Gao Binlong <[email protected]>

gaobinlong added 2 commits October 8, 2023 18:52

Add assertion

8a79835

Signed-off-by: Gao Binlong <[email protected]>

merge upstream main

5b8d247

Signed-off-by: Gao Binlong <[email protected]>

Merge remote-tracking branch 'upstream/main' into fix_dissect_bug

fff66b4

dblock approved these changes Oct 13, 2023

View reviewed changes

Merge branch 'main' into fix_dissect_bug

b240237

Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>

Merge main

840fe0e

Signed-off-by: Gao Binlong <[email protected]>

Merge remote-tracking branch 'upstream/main' into fix_dissect_bug

9c2514a

Merge remote-tracking branch 'upstream/main' into fix_dissect_bug

e03b2b1

dblock added the backport 2.x Backport to 2.x branch label Oct 17, 2023

dblock merged commit 7b62e2f into opensearch-project:main Oct 17, 2023
17 checks passed

opensearch-trigger-bot bot mentioned this pull request Oct 17, 2023

[Backport 2.x] Fix dissect ingest processor parsing empty brackets failed #10667

Merged

gaobinlong mentioned this pull request Jan 10, 2024

[BUG] Dissect ingest processor cannot parse empty value with brackets #7223

Closed

BrewTestBot mentioned this pull request Feb 21, 2024

opensearch 2.12.0 Homebrew/homebrew-core#163463

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix dissect ingest processor parsing empty brackets failed #9255

Fix dissect ingest processor parsing empty brackets failed #9255

gaobinlong commented Aug 11, 2023 •

edited

Loading

github-actions bot commented Aug 11, 2023

opensearch-trigger-bot bot commented Aug 11, 2023

opensearch-trigger-bot bot commented Aug 11, 2023

opensearch-trigger-bot bot commented Aug 11, 2023

github-actions bot commented Oct 8, 2023

github-actions bot commented Oct 8, 2023

gaobinlong commented Oct 9, 2023

dblock commented Oct 10, 2023

github-actions bot commented Oct 11, 2023

gaobinlong commented Oct 11, 2023

dblock commented Oct 13, 2023

github-actions bot commented Oct 13, 2023

github-actions bot commented Oct 16, 2023

github-actions bot commented Oct 17, 2023

github-actions bot commented Oct 17, 2023

Fix dissect ingest processor parsing empty brackets failed #9255

Fix dissect ingest processor parsing empty brackets failed #9255

Conversation

gaobinlong commented Aug 11, 2023 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Aug 11, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Aug 11, 2023

opensearch-trigger-bot bot commented Aug 11, 2023

opensearch-trigger-bot bot commented Aug 11, 2023

github-actions bot commented Oct 8, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 8, 2023

Gradle Check (Jenkins) Run Completed with:

gaobinlong commented Oct 9, 2023

dblock commented Oct 10, 2023

github-actions bot commented Oct 11, 2023

Gradle Check (Jenkins) Run Completed with:

gaobinlong commented Oct 11, 2023

dblock commented Oct 13, 2023

github-actions bot commented Oct 13, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 16, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 17, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 17, 2023

Gradle Check (Jenkins) Run Completed with:

gaobinlong commented Aug 11, 2023 •

edited

Loading