Handle case with nested list of objects

[krishy91]

Description

These changes add support for processing a list of objects in a nested field. Currently, if the configuration looks like below and if the nested field contains a list of nested objects then the request fails:

curl --location --request PUT 'http://localhost:9200/_ingest/pipeline/embeddings-nested' \
--header 'Content-Type: application/json' \
--data '{
    "processors": [
        {
            "text_embedding": {
                "model_id": "o-bPa4sBu3zYeKjn7baf",
                "field_map": {
                    "nested-field": {
                        "text-field": "vector-field"
                    }
                }
            }
        }
    ]
}'

Example Request with Nested List

curl --location 'http://localhost:9200/_ingest/pipeline/embeddings-nested/_simulate?verbose=true' \
--header 'Content-Type: application/json' \
--data '{
    "docs": [
        {
            "_id": "1",
            "_source": {
                "nested-field": [
                    {
                        "text-field": "This is another test"
                    },
                    {
                        "text-field": "This is a third field"
                    },
                    {
                        "text-field": "This is a fourth field"
                    }
                ]
            }
        }
    ]
}'

Issues Resolved

#469

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[navneet1v]

@krishy91 is this PR still a draft PR or ready for review?

[krishy91]

@krishy91 is this PR still a draft PR or ready for review?

Hi! I'm still working on it! I will update here once its ready!

[navneet1v]

Hi @krishy91 before we can review the changes please do the following:

1. Remove the draft keyword from title of the PR.
2. Make sure that all your github actions are successful.

Please let ping on this PR once the above things are done.

[codecov]

Codecov Report

Attention: Patch coverage is 89.65517% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 84.42%. Comparing base (bd1dfbf) to head (54419ea).
Report is 17 commits behind head on main.

Files	Patch %	Lines
...rch/neuralsearch/processor/InferenceProcessor.java	89.65%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #477      +/-   ##
============================================
+ Coverage     84.33%   84.42%   +0.09%     
- Complexity      533      609      +76     
============================================
  Files            40       48       +8     
  Lines          1564     1843     +279     
  Branches        244      282      +38     
============================================
+ Hits           1319     1556     +237     
- Misses          133      162      +29     
- Partials        112      125      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[krishy91]

Hi @navneet1v ! The Pull Request passed the workflows.

[br3no]

We would really like to use this feature, is there anything we can do to help in getting this merged?

[manzke]

Solves: #469 - PR is already in use and we would like to see it included in 2.12. or earlier.

[krishy91]

Hi, we would really like to have this fix in the next possible 2.x release. Need two more reviews @navneet1v

[navneet1v]

Will be checking this PR today.

[krishy91]

Thanks @navneet1v! Let me know if I can do anything to speed up the merge process

[br3no]

@navneet1v gently reminder to look into this PR. We would like to move away from using this branch.

[navneet1v]

Looked into the PR, on high level looks good. Please add Integration tests for this change.

[navneet1v]

Also code coverage is dropped for new lines which are added please ensure that those lines are covered.

[krishy91]

Looked into the PR, on high level looks good. Please add Integration tests for this change.

Thanks @navneet1v! Added integration test for the case containing list of nested objects

[krishy91]

Also code coverage is dropped for new lines which are added please ensure that those lines are covered.

Added addtional test with objects having nesting depth of 2. The InferenceProcessor has 98% coverage for lines now. How can I rerun the report?

[krishy91]

@navneet1v did you get a chance to look at the PR again after I resolved the issues you mentioned in your review?