[BUG] GeoPointShapeQueryTests.testQueryLinearRing fails with UnsupportedOperationException #11688

rishabhmaurya · 2023-12-28T22:20:16Z

Describe the bug

GeoPointShapeQueryTests.testQueryLinearRing fails with UnsupportedOperationException -

Here is the failed test - https://build.ci.opensearch.org/job/gradle-check/31539/testReport/junit/org.opensearch.search.geo/GeoPointShapeQueryTests/testQueryLinearRing/

The failure is not reproducible on mac, amazonlinux and ubuntu when I tried to reproduce them locally with the same seed.

Relates to PR - #11039

java.lang.UnsupportedOperationException: line ring cannot be serialized using GeoJson
	at __randomizedtesting.SeedInfo.seed([D24747E5E9B452AE:CFF1D9D2D64D1980]:0)
	at org.opensearch.common.geo.GeoJson$3.visit(GeoJson.java:561)
	at org.opensearch.common.geo.GeoJson$3.visit(GeoJson.java:543)
	at org.opensearch.geometry.LinearRing.visit(LinearRing.java:84)
	at org.opensearch.common.geo.GeoJson.getGeoJsonName(GeoJson.java:543)
	at org.opensearch.common.geo.GeoJson.toXContent(GeoJson.java:103)
	at org.opensearch.index.query.AbstractGeometryQueryBuilder.doXContent(AbstractGeometryQueryBuilder.java:452)
	at org.opensearch.index.query.AbstractQueryBuilder.toXContent(AbstractQueryBuilder.java:101)
	at org.opensearch.core.xcontent.XContentBuilder.value(XContentBuilder.java:887)
	at org.opensearch.core.xcontent.XContentBuilder.value(XContentBuilder.java:880)
	at org.opensearch.core.xcontent.XContentBuilder.field(XContentBuilder.java:872)
	at org.opensearch.search.builder.SearchSourceBuilder.innerToXContent(SearchSourceBuilder.java:1350)
	at org.opensearch.search.builder.SearchSourceBuilder.toXContent(SearchSourceBuilder.java:1490)
	at org.opensearch.core.xcontent.XContentHelper.toXContent(XContentHelper.java:46)
	at org.opensearch.search.builder.SearchSourceBuilder.toString(SearchSourceBuilder.java:1810)
	at org.opensearch.action.search.SearchRequest.buildDescription(SearchRequest.java:715)
	at org.opensearch.action.search.SearchTask.getDescription(SearchTask.java:83)
	at org.opensearch.tasks.TaskManager.register(TaskManager.java:210)
	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:99)
	at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110)
	at org.opensearch.client.node.NodeClient.doExecute(NodeClient.java:97)
	at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:476)
	at org.opensearch.client.support.AbstractClient.execute(AbstractClient.java:463)
	at org.opensearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:66)
	at org.opensearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:73)
	at org.opensearch.search.geo.GeoPointShapeQueryTests.testQueryLinearRing(GeoPointShapeQueryTests.java:140)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.lang.IllegalStateException: Failed to close the XContentBuilder
		at org.opensearch.core.xcontent.XContentBuilder.close(XContentBuilder.java:1044)
		at org.opensearch.core.xcontent.XContentHelper.toXContent(XContentHelper.java:41)
		... 50 more
	Caused by: java.io.IOException: Unclosed object or array found
		at org.opensearch.common.xcontent.json.JsonXContentGenerator.close(JsonXContentGenerator.java:494)
		at org.opensearch.core.xcontent.XContentBuilder.close(XContentBuilder.java:1042)
		... 51 more

Related component

Build

To Reproduce

Here are the steps to reproduce it, although its not reproducible -

https://build.ci.opensearch.org/job/gradle-check/31539/testReport/junit/org.opensearch.search.geo/GeoPointShapeQueryTests/testQueryLinearRing/

./gradlew ':server:test' --tests "org.opensearch.search.geo.GeoPointShapeQueryTests.testQueryLinearRing" -Dtests.seed=E6A53B1422E4284A -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=hu-HU -Dtests.timezone=Pacific/Funafuti -Druntime.java=21

Expected behavior

Ideally such serialization/deserialization failures should be deterministic not be environment dependent.

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

rishabhmaurya · 2023-12-29T21:01:09Z

Found that it fails here -

OpenSearch/server/src/main/java/org/opensearch/tasks/TaskManager.java

Line 210 in e8df9db

    
           logger.trace("register {} [{}] [{}] [{}]", task.getId(), type, action, task.getDescription());

So when trace logs are enabled and it goes to line 210 and calls task.getDescription() - it results in following exception. When trace logs are not enabled, this line never gets called and tests executes normally.

java.lang.UnsupportedOperationException: line ring cannot be serialized using GeoJson
	at __randomizedtesting.SeedInfo.seed([D24747E5E9B452AE:CFF1D9D2D64D1980]:0)
	at org.opensearch.common.geo.GeoJson$3.visit(GeoJson.java:561)
	at org.opensearch.common.geo.GeoJson$3.visit(GeoJson.java:543)
	at org.opensearch.geometry.LinearRing.visit(LinearRing.java:84)
	at org.opensearch.common.geo.GeoJson.getGeoJsonName(GeoJson.java:543)
	at org.opensearch.common.geo.GeoJson.toXContent(GeoJson.java:103)
	at org.opensearch.index.query.AbstractGeometryQueryBuilder.doXContent(AbstractGeometryQueryBuilder.java:452)
	at org.opensearch.index.query.AbstractQueryBuilder.toXContent(AbstractQueryBuilder.java:101)
	at org.opensearch.core.xcontent.XContentBuilder.value(XContentBuilder.java:887)
	at org.opensearch.core.xcontent.XContentBuilder.value(XContentBuilder.java:880)
	at org.opensearch.core.xcontent.XContentBuilder.field(XContentBuilder.java:872)
	at org.opensearch.search.builder.SearchSourceBuilder.innerToXContent(SearchSourceBuilder.java:1350)
	at org.opensearch.search.builder.SearchSourceBuilder.toXContent(SearchSourceBuilder.java:1490)
	at org.opensearch.core.xcontent.XContentHelper.toXContent(XContentHelper.java:46)
	at org.opensearch.search.builder.SearchSourceBuilder.toString(SearchSourceBuilder.java:1810)
	at org.opensearch.action.search.SearchRequest.buildDescription(SearchRequest.java:715)
	at org.opensearch.action.search.SearchTask.getDescription(SearchTask.java:83)
	at org.opensearch.tasks.TaskManager.register(TaskManager.java:210)

peternied · 2024-01-31T16:52:49Z

[Triage - attendees 1 2 3 4 5 6 7 8]
@rishabhmaurya Thanks for filing.

@andrross It sounds like there are more details to this issue, could you update the description to better fit what should be addressed.

ajaymovva · 2024-02-01T10:00:46Z

Facing the same issue: https://build.ci.opensearch.org/job/gradle-check/33006/testReport/junit/org.opensearch.search.geo/GeoPointShapeQueryTests/testQueryLinearRing/

PR: #12111

shwetathareja · 2024-03-19T13:24:23Z

PR - #8992 observed same failure here - https://build.ci.opensearch.org/job/gradle-check/35326/

shwetathareja · 2024-03-19T16:28:19Z

#12252 (comment)

shwetathareja · 2024-03-19T18:15:36Z

This test seems like a serial offender, causing gradle check across multiple PRs to fail. can this be muted @andrross ?

rishabhmaurya · 2024-03-19T19:15:08Z

Or we can remove -
task.getDescription() from logger.trace("register {} [{}] [{}] [{}]", task.getId(), type, action, task.getDescription()); to fix it. Not sure how viable this option is.

shwetathareja · 2024-03-20T03:27:58Z

Or we can remove -
task.getDescription() from logger.trace("register {} [{}] [{}] [{}]", task.getId(), type, action, task.getDescription()); to fix it. Not sure how viable this option is.

@rishabhmaurya : It wouldn't be preferred as it is a trace log and may be useful in the context of debugging for other type of tasks. We should fix the test.

shwetathareja · 2024-03-20T07:00:34Z

As @rishabhmaurya pointed out, right now it is failing when trace logging is enabled.
This would be consistent failure when trace is enabled, so we need to check during gradle-check why some runs are with trace enabled and while others are not.

I looked into the test, it is a negative and it expects this same exception, just that from a different flow, but when trace logging is enabled, that exception is pre-empted via task.getDescription() call

    public void testQueryLinearRing() throws Exception {
        XContentBuilder xcb = createDefaultMapping();
        client().admin().indices().prepareCreate("test").setMapping(xcb).get();
        ensureGreen();

        LinearRing linearRing = new LinearRing(new double[] { -25, -35, -25 }, new double[] { -25, -35, -25 });

        try {
            // LinearRing extends Line implements Geometry: expose the build process
            GeoShapeQueryBuilder queryBuilder = new GeoShapeQueryBuilder(defaultGeoFieldName, linearRing);
            SearchRequestBuilder searchRequestBuilder = new SearchRequestBuilder(client(), SearchAction.INSTANCE);
            searchRequestBuilder.setQuery(queryBuilder);
            searchRequestBuilder.setIndices("test");
            searchRequestBuilder.get();       --------->  // here it fails during search task register before it could start the execution, which would have thrown the same exception 
        } catch (SearchPhaseExecutionException e) {
            assertThat(
                e.getCause().getMessage(),
                containsString("Field [" + defaultGeoFieldName + "] does not support LINEARRING queries")
            );
        }
    }

I don't think we can control the logging behavior in TaskManager as it is generic across all tasks.
Our options to fix the test

Expect UnsupportedOperationException in the test and assert on it as well
Change log level for this test? What if somebody introduces another log at debug, this would start to fail again

I would prefer Option 1.
@peternied / Others : Please feel free to share your thoughts if you think there is a better way to handle it.

shwetathareja · 2024-03-20T08:44:55Z

Raised the PR to fix it - #12783

dblock · 2024-03-20T20:30:50Z

The fix in #12783 doesn't look right, reopening so we don't lose it, cc: @Bukhtawar @shwetathareja (#12783 (comment))

peternied · 2024-03-21T19:54:02Z

Does anyone know why is a LinearRing should not be representable in OpenSearch? From the GeoJSON RFC [1] its a polygon... seems easier for our users (and test cases) by representing it as a polygon when outputted.

[1] https://datatracker.ietf.org/doc/html/rfc7946#section-3.1.6

rishabhmaurya · 2024-03-21T21:07:16Z

@andrross @shwetathareja how about this fix - #12844 ?
And we should still address - #12324 independently.

rishabhmaurya added bug Something isn't working untriaged labels Dec 28, 2023

github-actions bot added the Build Build Tasks/Gradle Plugin, groovy scripts, build tools, Javadoc enforcement. label Dec 28, 2023

rishabhmaurya added flaky-test Random test failure that succeeds on second run and removed Build Build Tasks/Gradle Plugin, groovy scripts, build tools, Javadoc enforcement. labels Dec 28, 2023

andrross added the Geospatial label Dec 28, 2023

vikasvb90 added the Search Search query, autocomplete ...etc label Jan 30, 2024

github-project-automation bot added this to Search Project Board Jan 30, 2024

github-project-automation bot moved this to 🆕 New in Search Project Board Jan 30, 2024

peternied removed the untriaged label Jan 31, 2024

ajaymovva mentioned this issue Feb 1, 2024

[Backport 2.x] AdmissionControl Framework with CPU Based Admission Controller #12111

Merged

8 tasks

andrross mentioned this issue Feb 14, 2024

[BUG] Some GeoShapeQueryBuilder instances cannot be toString()'d #12324

Open

shiv0408 mentioned this issue Mar 19, 2024

Created new ReplicaShardBatchAllocator #8992

Merged

6 tasks

rishabhmaurya mentioned this issue Mar 19, 2024

[DerivedFields] PR1 - DerivedFieldScript and query execution logic #12746

Merged

8 tasks

shwetathareja mentioned this issue Mar 20, 2024

Light weight Transport action to verify local term before fetching cluster-state from remote #12252

Merged

8 tasks

shwetathareja mentioned this issue Mar 20, 2024

Fixing flaky test GeoPointShapeQueryTests.testQueryLinearRing #12783

Merged

8 tasks

shwetathareja closed this as completed in #12783 Mar 20, 2024

github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board Mar 20, 2024

dblock reopened this Mar 20, 2024

github-project-automation bot moved this from ✅ Done to 🏗 In progress in Search Project Board Mar 20, 2024

github-actions bot added the untriaged label Mar 20, 2024

This was referenced Mar 21, 2024

Implement serialization for LinearRing. #12831

Closed

Catch task description error #12834

Merged

rishabhmaurya mentioned this issue Mar 21, 2024

Fix LinearRing serialization issue while emitting task description in… #12844

Closed

6 tasks

reta closed this as completed in #12834 Mar 22, 2024

github-project-automation bot moved this from 🏗 In progress to ✅ Done in Search Project Board Mar 22, 2024

reta added v2.14.0 and removed untriaged labels Mar 22, 2024

dblock mentioned this issue Mar 22, 2024

[Backport 2.x] Catch task description error (#12834) #12858

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] GeoPointShapeQueryTests.testQueryLinearRing fails with UnsupportedOperationException #11688

[BUG] GeoPointShapeQueryTests.testQueryLinearRing fails with UnsupportedOperationException #11688

rishabhmaurya commented Dec 28, 2023 •

edited

Loading

rishabhmaurya commented Dec 29, 2023

peternied commented Jan 31, 2024

ajaymovva commented Feb 1, 2024

shwetathareja commented Mar 19, 2024

shwetathareja commented Mar 19, 2024

shwetathareja commented Mar 19, 2024

rishabhmaurya commented Mar 19, 2024

shwetathareja commented Mar 20, 2024

shwetathareja commented Mar 20, 2024 •

edited

Loading

shwetathareja commented Mar 20, 2024

dblock commented Mar 20, 2024

peternied commented Mar 21, 2024

rishabhmaurya commented Mar 21, 2024 •

edited

Loading

[BUG] GeoPointShapeQueryTests.testQueryLinearRing fails with UnsupportedOperationException #11688

[BUG] GeoPointShapeQueryTests.testQueryLinearRing fails with UnsupportedOperationException #11688

Comments

rishabhmaurya commented Dec 28, 2023 • edited Loading

Describe the bug

Related component

To Reproduce

Expected behavior

Additional Details

rishabhmaurya commented Dec 29, 2023

peternied commented Jan 31, 2024

ajaymovva commented Feb 1, 2024

shwetathareja commented Mar 19, 2024

shwetathareja commented Mar 19, 2024

shwetathareja commented Mar 19, 2024

rishabhmaurya commented Mar 19, 2024

shwetathareja commented Mar 20, 2024

shwetathareja commented Mar 20, 2024 • edited Loading

shwetathareja commented Mar 20, 2024

dblock commented Mar 20, 2024

peternied commented Mar 21, 2024

rishabhmaurya commented Mar 21, 2024 • edited Loading

rishabhmaurya commented Dec 28, 2023 •

edited

Loading

shwetathareja commented Mar 20, 2024 •

edited

Loading

rishabhmaurya commented Mar 21, 2024 •

edited

Loading