-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] "arithmetic_exception: long overflow" while running sort query on date field #16709
Comments
[Triage] Can you please share some sample data/commands to re-produce this ?
Output
Thank you |
Hi @prudhvigodithi ,
When we execute the query against all these 14 indices we are getting arithmetic exceptions . This is how we are executing the query.
This is the data distribution on the indices When we tried to execute the same query against smaller dataset(i.e. against 10% of the above mentioned data volume) , we did not face any issues. We are currently using OS version 2.11 & these are sample document Document with dueDate value
Document without dueDate value
|
This may have been fixed by #12676, which was merged for OpenSearch 2.13. Are you able to try upgrading to 2.13 or later and testing? |
Hi @msfroh , We upgraded the OS version to 2.13 but still we are seeing the same issue |
Hello, everyone. I replicated the issue by creating an index with different combinations of date formats for the dueDate field. In my environment, the error was triggered when custom date formats were used (i.e., yyyy-MM-dd HH:mm:ss, yyyy-MM-dd), even when using each one of them alone. When using "strict_date_optional_time" alone or paired with "epoch_millis" the issue did not show up.
|
Thanks @NamrataNerli and @noeg-1, following is the stack trace of the error.
|
After multiple tries finally I was able to reproduce the error, I have created a small
I can see this error across the Thank you. |
I have a draft PR #16796 while we discuss on more on this, @noeg-1 and @NamrataNerli can you please try to clone my fork, run the cluster and see you can re-produce the error? With the fix part of the PR I dont see the long overflow error anymore.
Thanks. |
Hello, @prudhvigodithi. I have cloned the fork, run the cluster, and tried to trigger the Arithmetic Exception. The exception was not triggered in any of the tests I ran. I tested with the provided conditions, ingesting increasing amounts of documents with missing dueDate values, up to 20,000 documents. |
Did some deep dive and looks like the issue is coming from joda time, tried adding the flow below at high level.
From the logs
The reason from beginning I was not able to reproduce the error on Since we dont want to backport this to Also thanks a lot @noeg-1 for the testing. I have more generic solution to my fork (#16796) can you please test one more time if the results are as expected (from my end with the above script #16709 (comment) I dont see the error anymore). Once testing is done I will create a PR a |
@msfroh tagging you here since you've been looking into similar (or same?) issue before, may be missing some edge cases? Thanks @prudhvigodithi for looking into it. |
Hello, everyone. @prudhvigodithi, I have tested again, using the same conditions shared for reproducing the error. Tried again with up to 20,000 documents with missing values for dueDate. From my end, the Arithmetic Exception was not triggered in any of the tests |
Thanks, I have the created a PR for |
output |
Describe the bug
We are getting "arithmetic_exception: long overflow" intermittently when we sort the results on date field.
This is the query we are using:
This is error message :
Related component
Search
To Reproduce
We are using this date format on dueDate field.
This is query
Note: Some of the documents/records that are inserted on the index do have value for dueDate field and some of them do.
We tried changing the date format to this
yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis||strict_date_optional_time
, but still we were getting arithmetic_exception.Expected behavior
We want to sort on date field without having any issues.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: