Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fork Sync: Update from parent repository #143

Open
wants to merge 1,914 commits into
base: main
Choose a base branch
from

Conversation

neetikasinghal
Copy link
Owner

No description provided.

@neetikasinghal neetikasinghal requested a review from sohami as a code owner August 23, 2023 23:08
dependabot bot and others added 29 commits October 1, 2024 09:14
…s/repository-azure (#16133)

* Bump com.azure:azure-core-http-netty in /plugins/repository-azure

Bumps [com.azure:azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-core-http-netty_1.15.3...azure-core-http-netty_1.15.4)

---
updated-dependencies:
- dependency-name: com.azure:azure-core-http-netty
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…mps in node stats (#15611)

---------

Signed-off-by: Lakshya Taragi <[email protected]>
…ture (#16135)

* Bump org.jline:jline in /test/fixtures/hdfs-fixture

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.3 to 3.27.0.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.3...jline-3.27.0)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* Remove Identity FeatureFlag

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Mark IdentityService as internal

Signed-off-by: Craig Perkins <[email protected]>

* Add @internalapi annotation

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
…ed handler (#16154)

* Ensure RestHandler.Wrapper delegates all implementations to the wrapper handler

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Fix typo

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
* Fix client-rest-high-level tests for ppc64le

Signed-off-by: prachi-gaonkar <[email protected]>

* Fix client-rest-high-level tests for ppc64le

Signed-off-by: prachi-gaonkar <[email protected]>

---------

Signed-off-by: prachi-gaonkar <[email protected]>
* Add bwc version 2.17.2

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Update Version.java

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
…on dynamic limit settings (#15986)

* Add changes to block calls in cat shards, indices and segments based on dynamic limit settings

Signed-off-by: Sumit Bansal <[email protected]>
…16080)

* Separate Remote State and Publication enabled and configured methods

Signed-off-by: Shivansh Arora <[email protected]>
* add `Strings#isDigits` API

inspiration taken from [this SO answer][SO].

note that the stream is not parallelised to avoid the overhead of this
as the method is intended to be called primarily with shorter strings
where the time to set up would take longer than the actual check.

[SO]: https://stackoverflow.com/a/35150400

Signed-off-by: Ralph Ursprung <[email protected]>

* add `phone` & `phone-search` analyzer + tokenizer

this is largely based on [elasticsearch-phone] and internally uses
[libphonenumber].
this intentionally only ports a subset of the features: only `phone` and
`phone-search` are supported right now, `phone-email` can be added
if/when there's a clear need for it.

using `libphonenumber` is required since parsing phone numbers is a
non-trivial task (even though it might seem trivial at first glance!),
as can be seen in the list [falsehoods programmers believe about phone
numbers][falsehoods].

this allows defining the region to be used when analysing a phone
number. so far only the generic "unkown" region (`ZZ`) had been used
which worked as long as international numbers were prefixed with `+` but
did not work when using local numbers (e.g. a number stored as
`+4158...` was not matched against a number entered as `004158...` or
`058...`).

example configuration for an index:
```json
{
  "index": {
    "analysis": {
      "analyzer": {
        "phone": {
          "type": "phone"
        },
        "phone-search": {
          "type": "phone-search"
        },
        "phone-ch": {
          "type": "phone",
          "phone-region": "CH"
        },
        "phone-search-ch": {
          "type": "phone-search",
          "phone-region": "CH"
        }
      }
    }
  }
}
```
this creates four analyzers: `phone` and `phone-search` which do not
explicitly specify a region and thus fall back to `ZZ` (unknown region,
regional version of international dialing prefix (e.g. `00` instead of
`+` in most of europe) will not be recognised) and `phone-ch` and
`phone-search-ch` which will try to parse the phone number as a swiss
phone number (thus e.g. `00` as a prefix is recognised as the
international dialing prefix).

note that the analyzer is (currently) not meant to find phone numbers in
large text documents - instead it should be used on fields which contain
just the phone number (though extra text will be ignored) and it
collects the whole content of the field into a `String` in memory,
making it unsuitable for large field values.

this has been implemented in a new plugin which is however part of the
central opensearch repository as it was deemed too big an overhead to
have it in a separate repository but not important enough to bundle it
directly in `analysis-common` (see the discussion on the issue and the
PR for further details).

note that the new plugin has been added to the exclude list of the
javadoc check as this check is overzealous and also complains in many
cases where it shouldn't (e.g. on overridden methods - which it should
theoretically not do - or constructors which don't even exist). the
check first needs to be improved before this exclusion could be removed.

closes #11326

[elasticsearch-phone]: https://github.com/purecloudlabs/elasticsearch-phone
[libphonenumber]: https://github.com/google/libphonenumber
[falsehoods]: https://github.com/google/libphonenumber/blob/master/FALSEHOODS.md

Signed-off-by: Ralph Ursprung <[email protected]>

---------

Signed-off-by: Ralph Ursprung <[email protected]>
…16194)

* Fix warnings from SLF4J on startup when repository-s3 is installed

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Fix precommit

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
* Adding _list/shards API

Signed-off-by: Harsh Garg <[email protected]>
* Update Apache Lucene to 9.12.0

Signed-off-by: Andriy Redko <[email protected]>

* change to IOContext READONCE in locations where the file is not expected to be read multiple times.

Signed-off-by: Marc Handalian <[email protected]>

* Use READ IOContext for all non Segment* files when copying node-node

Signed-off-by: Marc Handalian <[email protected]>

* Fixing more test failures

Signed-off-by: Andriy Redko <[email protected]>

* Move Composite912Codec under org.opensearch.index.codec.composite.composite912 package

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Marc Handalian <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
* Avoid deep copy and other allocation improvements
* Refactoring based on PR Comments and added JavaDocs
* Added more comments
* Added character for Triggering Jenkins build
* Changes to cover collectZeroDocEntries method
* Updated comment based on change in method's functionality
* Added test to cover branches in collectZeroDocEntriesIfRequired
* Rebased and resolved changelog conflict

---------

Signed-off-by: expani <[email protected]>
* cancellation related

Signed-off-by: Kiran Prakash <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Kiran Prakash <[email protected]>

* add better cancellation reason

Signed-off-by: Kiran Prakash <[email protected]>

* Update DefaultTaskCancellationTests.java

Signed-off-by: Kiran Prakash <[email protected]>

* refactor

Signed-off-by: Kiran Prakash <[email protected]>

* refactor

Signed-off-by: Kiran Prakash <[email protected]>

* Update DefaultTaskCancellation.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update DefaultTaskCancellation.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update DefaultTaskCancellation.java

Signed-off-by: Kiran Prakash <[email protected]>

* Update DefaultTaskSelectionStrategy.java

Signed-off-by: Kiran Prakash <[email protected]>

* refactor

Signed-off-by: Kiran Prakash <[email protected]>

* refactor node level threshold

Signed-off-by: Kiran Prakash <[email protected]>

* use query group task

Signed-off-by: Kaushal Kumar <[email protected]>

* code clean up and refactorings

Signed-off-by: Kaushal Kumar <[email protected]>

* add unit tests and fix existing ones

Signed-off-by: Kaushal Kumar <[email protected]>

* uncomment the test case

Signed-off-by: Kaushal Kumar <[email protected]>

* update CHANGELOG

Signed-off-by: Kaushal Kumar <[email protected]>

* fix imports

Signed-off-by: Kaushal Kumar <[email protected]>

* add queryGroupService

Signed-off-by: Kaushal Kumar <[email protected]>

* refactor and add UTs for new constructs

Signed-off-by: Kaushal Kumar <[email protected]>

* fix javadocs

Signed-off-by: Kaushal Kumar <[email protected]>

* remove code clutter

Signed-off-by: Kaushal Kumar <[email protected]>

* change annotation version and task selection strategy

Signed-off-by: Kaushal Kumar <[email protected]>

* rename a util class

Signed-off-by: Kaushal Kumar <[email protected]>

* remove wrappers from resource type

Signed-off-by: Kaushal Kumar <[email protected]>

* apply spotless

Signed-off-by: Kaushal Kumar <[email protected]>

* address comments

Signed-off-by: Kaushal Kumar <[email protected]>

* add rename changes

Signed-off-by: Kaushal Kumar <[email protected]>

* address comments

Signed-off-by: Kaushal Kumar <[email protected]>

* initial changes

Signed-off-by: Kaushal Kumar <[email protected]>

* refactor changes and logical bug fix

Signed-off-by: Kaushal Kumar <[email protected]>

* add chanegs

Signed-off-by: Kaushal Kumar <[email protected]>

* address comments

Signed-off-by: Kaushal Kumar <[email protected]>

* temp changes

Signed-off-by: Kaushal Kumar <[email protected]>

* add UTs

Signed-off-by: Kaushal Kumar <[email protected]>

* add changelog

Signed-off-by: Kaushal Kumar <[email protected]>

* add task completion listener hook

Signed-off-by: Kaushal Kumar <[email protected]>

* add remaining pieces to make the feature functional

Signed-off-by: Kaushal Kumar <[email protected]>

* extend stats and fix bugs

Signed-off-by: Kaushal Kumar <[email protected]>

* fix bugs and add logic to make SBP work with wlm

Signed-off-by: Kaushal Kumar <[email protected]>

* address comments

Signed-off-by: Kaushal Kumar <[email protected]>

* fix bugs and SBP ITs

Signed-off-by: Kaushal Kumar <[email protected]>

* add missed applyCluster state change

Signed-off-by: Kaushal Kumar <[email protected]>

* address comments

Signed-off-by: Kaushal Kumar <[email protected]>

* decouple queryGroupService and cancellationService

Signed-off-by: Kaushal Kumar <[email protected]>

* replace StateApplier with StateListener interface

Signed-off-by: Kaushal Kumar <[email protected]>

* fix precommit errors

Signed-off-by: Kaushal Kumar <[email protected]>

---------

Signed-off-by: Kiran Prakash <[email protected]>
Signed-off-by: Kaushal Kumar <[email protected]>
Co-authored-by: Kiran Prakash <[email protected]>
* Segmented cache changes for TieredCache

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding change log

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Allow segment number to be power of two

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Moving common tiered cache IT methods to a common base class

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Adding disk took time IT test with multiple segment

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Correcting changelog

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing invalid segment count variable name

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Introducing new settings for size for respective cache tier

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Changing the default segmentCount logic

Signed-off-by: Sagar Upadhyaya <[email protected]>

* Fixing missing java doc issue

Signed-off-by: Sagar Upadhyaya <[email protected]>

---------

Signed-off-by: Sagar Upadhyaya <[email protected]>
Signed-off-by: Sagar <[email protected]>
* Refactoring builder tests

Signed-off-by: Bharathwaj G <[email protected]>

* adding date tests

Signed-off-by: Bharathwaj G <[email protected]>

---------

Signed-off-by: Bharathwaj G <[email protected]>
* Change successfulSearchShardIndices to Set<Index>

Signed-off-by: David Zane <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Ankit Jain <[email protected]>

---------

Signed-off-by: David Zane <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
Co-authored-by: Ankit Jain <[email protected]>
kkewwei and others added 30 commits December 27, 2024 15:29
…ository-azure (#16918)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.17.2 to 1.18.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.17.2...v1.18.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
…ixtures/hdfs-fixture (#16919)

* Bump org.apache.commons:commons-text in /test/fixtures/hdfs-fixture

Bumps org.apache.commons:commons-text from 1.12.0 to 1.13.0.

---
updated-dependencies:
- dependency-name: org.apache.commons:commons-text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Introduce auxiliary transport to NetworkPlugin and add gRPC plugin.

Auxiliary transports are optional lifecycle components provided by
network plugins which run in parallel to the http server/native
transport. They are distinct from the existing NetworkPlugin
interfaces of 'getTransports' and 'getHttpTransports' as auxiliary
transports are optional. Each AuxTransport implements it's own
'aux.transport.type' and 'aux.transport.<type>.ports' setting. Since
Security.java initializes previous to Node.java during bootstrap
socket binding permissions are granted based on
'aux.transport.<type>.ports' for each enabled 'aux.transport.type',
falling back to a default if no ports are specified.

Signed-off-by: Finn Carroll <[email protected]>
…ha256() methods (#16923)

* Update script supports java.lang.String.sha1() and java.lang.String.sha256() methods

Signed-off-by: Gao Binlong <[email protected]>

* Modify change log

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
* Make extended plugins optional

Signed-off-by: Craig Perkins <[email protected]>

* Make extended plugins optional

Signed-off-by: Craig Perkins <[email protected]>

* Load extensions for classpath plugins

Signed-off-by: Craig Perkins <[email protected]>

* Ensure only single instance for each classpath extension

Signed-off-by: Craig Perkins <[email protected]>

* Add test for classpath plugin extended plugin loading

Signed-off-by: Craig Perkins <[email protected]>

* Modify test to allow optional extended plugin

Signed-off-by: Craig Perkins <[email protected]>

* Only optional extended plugins

Signed-off-by: Craig Perkins <[email protected]>

* Add additional warning message

Signed-off-by: Craig Perkins <[email protected]>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <[email protected]>

* Add tag to make extended plugin optional

Signed-off-by: Craig Perkins <[email protected]>

* Only send plugin names when serializing PluginInfo

Signed-off-by: Craig Perkins <[email protected]>

* Keep track of optional extended plugins in separate set

Signed-off-by: Craig Perkins <[email protected]>

* Include in ser/de of PluginInfo

Signed-off-by: Craig Perkins <[email protected]>

* Change to 3_0_0

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
…res/hdfs-fixture (#16951)

* Bump ch.qos.logback:logback-core in /test/fixtures/hdfs-fixture

Bumps [ch.qos.logback:logback-core](https://github.com/qos-ch/logback) from 1.5.12 to 1.5.16.
- [Commits](qos-ch/logback@v_1.5.12...v_1.5.16)

---
updated-dependencies:
- dependency-name: ch.qos.logback:logback-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
* add workload management IT
Signed-off-by: Ruirui Zhang <[email protected]>

* address comments
Signed-off-by: Ruirui Zhang <[email protected]>

---------

Signed-off-by: Ruirui Zhang <[email protected]>
…s/repository-azure (#16952)

* Bump com.azure:azure-core-http-netty in /plugins/repository-azure

Bumps [com.azure:azure-core-http-netty](https://github.com/Azure/azure-sdk-for-java) from 1.15.5 to 1.15.7.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-core-http-netty_1.15.5...azure-core-http-netty_1.15.7)

---
updated-dependencies:
- dependency-name: com.azure:azure-core-http-netty
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* Updating SHAs

Signed-off-by: dependabot[bot] <[email protected]>

* Update changelog

Signed-off-by: dependabot[bot] <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
In some cases, when we create a term query over a `match_only_text`
field, it may still try to compute scores, which prevents early
termination. We should *always* use a constant score query when
querying `match_only_text`, since we don't have the statistics
required to compute scores.

---------

Signed-off-by: Michael Froh <[email protected]>
Avoids exception when querying unmapped field when star tree experimental
feature is enables.

---------

Signed-off-by: expani <[email protected]>
* Use async client for delete blob or path in S3 Blob Container

Signed-off-by: Ashish Singh <[email protected]>

* Fix UTs

Signed-off-by: Ashish Singh <[email protected]>

* Fix failures in S3BlobStoreRepositoryTests

Signed-off-by: Ashish Singh <[email protected]>

* Fix S3BlobStoreRepositoryTests

Signed-off-by: Ashish Singh <[email protected]>

* Fix failures in S3RepositoryThirdPartyTests

Signed-off-by: Ashish Singh <[email protected]>

* Fix failures in S3RepositoryPluginTests

Signed-off-by: Ashish Singh <[email protected]>

---------

Signed-off-by: Ashish Singh <[email protected]>
* Fix shallow v1 snapshot failures on closed index

Signed-off-by: Shubh Sahu <[email protected]>

* UT fix

Signed-off-by: Shubh Sahu <[email protected]>

* Adding UT

Signed-off-by: Shubh Sahu <[email protected]>

* small fix

Signed-off-by: Shubh Sahu <[email protected]>

* Addressing comments

Signed-off-by: Shubh Sahu <[email protected]>

* Addressing comments

Signed-off-by: Shubh Sahu <[email protected]>

* Modifying IT to restore snapshot

Signed-off-by: Shubh Sahu <[email protected]>

---------

Signed-off-by: Shubh Sahu <[email protected]>
Co-authored-by: Shubh Sahu <[email protected]>
…in flat_object/keyword field (#16974)

Signed-off-by: kkewwei <[email protected]>
Signed-off-by: kkewwei <[email protected]>
…16881)

* use the correct type to widen the sort fields when merging top docs

Signed-off-by: panguixin <[email protected]>

* fix

Signed-off-by: panguixin <[email protected]>

* apply commments

Signed-off-by: panguixin <[email protected]>

* changelog

Signed-off-by: panguixin <[email protected]>

* add more tests

Signed-off-by: panguixin <[email protected]>

---------

Signed-off-by: panguixin <[email protected]>
* Fix multi-value sort for unsigned long

Signed-off-by: panguixin <[email protected]>

* Add initial rest-api-spec tests

Signed-off-by: Andriy Redko <[email protected]>

* add more rest tests

Signed-off-by: panguixin <[email protected]>

* fix

Signed-off-by: panguixin <[email protected]>

* fix

Signed-off-by: panguixin <[email protected]>

* Extend MultiValueMode with dedicated support of unsigned_long doc values

Signed-off-by: Andriy Redko <[email protected]>

* Add CHANGELOG.md, minor cleanups

Signed-off-by: Andriy Redko <[email protected]>

* Correct the license headers

Signed-off-by: Andriy Redko <[email protected]>

* Correct the @publicapi version

Signed-off-by: Andriy Redko <[email protected]>

* Replace SingletonSortedNumericUnsignedLongValues with LongToSortedNumericUnsignedLongValues (as per review comments)

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: panguixin <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: Andriy Redko <[email protected]>
…ension & unformatted input (#16993)

* `phone-search` analyzer: don't emit int'l prefix

this was an oversight in the initial implementation: if the tokenizer
emits the international calling prefix in the search analyzer then all
documents with the same international calling prefix will match.

e.g. when searching for `+1-555-123-4567` not only documents with this
number would match but also any other document with a `1` token (i.e.
any other number with this prefix).

thus the search functionality is currently broken for this analyzer,
making it useless.

the test coverage has now been extended to cover these and other
use-cases.

Signed-off-by: Ralph Ursprung <[email protected]>

* `phone-search` analyzer: don't emit extension & unformatted input

if these tokens are emitted it meant that phone numbers with other
international dialling prefixes still matched.

e.g. searching for `+1 1234` would also match a number stored as
`+2 1234`, which was wrong.

the tokens still need to be emited for the `phone` analyzer, e.g. when
the user only enters the extension / local number it should still match,
the same is with the other ngrams: these are needed for
search-as-you-type style queries where the user input needs to match
against partial phone numbers.

Signed-off-by: Ralph Ursprung <[email protected]>

* `phone-search` analyzer: don't emit sip/tel prefix

in line with the previous two commits, this is something else the search
analyzer shouldn't emit since otherwise searching for any number with
such a prefix will match _any_ document with the same prefix.

Signed-off-by: Ralph Ursprung <[email protected]>

---------

Signed-off-by: Ralph Ursprung <[email protected]>
…ery flow (#16760)

* Update search only replica recovery flow

This PR includes multiple changes to search replica recovery.
1. Change search only replica copies to recover as empty store instead of PEER. This will run a store recovery that syncs segments from remote store directly and eliminate any primary communication.
2. Remove search replicas from the in-sync allocation ID set and update routing table to exclude them from allAllocationIds.  This ensures primaries aren't tracking or validating the routing table for any search replica's presence.
3. Change search replica validation to require remote store.  There are versions of the above changes that are still possible with primary based node-node replication, but I don't think they are worth making  at this time.

Signed-off-by: Marc Handalian <[email protected]>

* more coverage

Signed-off-by: Marc Handalian <[email protected]>

* add changelog entry

Signed-off-by: Marc Handalian <[email protected]>

* add assertions that Search Replicas are not in the in-sync id set nor the AllAllocationIds set in the routing table

Signed-off-by: Marc Handalian <[email protected]>

* update async task to only run if the FF is enabled and we are a remote store cluster.

This check had previously only checked for segrep

Signed-off-by: Marc Handalian <[email protected]>

* clean up max shards logic

Signed-off-by: Marc Handalian <[email protected]>

* remove search replicas from check during renewPeerRecoveryRetentionLeases

Signed-off-by: Marc Handalian <[email protected]>

* Revert "update async task to only run if the FF is enabled and we are a remote store cluster."

reverting this, we already check for remote store earlier.

This reverts commit 48ca1a3.

Signed-off-by: Marc Handalian <[email protected]>

* Add more tests for failover case

Signed-off-by: Marc Handalian <[email protected]>

* Update remotestore restore logic and add test ensuring we can restore only writers when red

Signed-off-by: Marc Handalian <[email protected]>

* Fix Search replicas to honor node level recovery limits

Signed-off-by: Marc Handalian <[email protected]>

* Fix translog UUID mismatch on existing store recovery.

This commit adds PR feedback and recovery tests post node restart.

Signed-off-by: Marc Handalian <[email protected]>

* Fix spotless

Signed-off-by: Marc Handalian <[email protected]>

* Fix bug with remote restore and add more tests

Signed-off-by: Marc Handalian <[email protected]>

---------

Signed-off-by: Marc Handalian <[email protected]>
* fix case insensitive and escaped query on wildcard

Signed-off-by: gesong.samuel <[email protected]>

* add changelog

Signed-off-by: gesong.samuel <[email protected]>

---------

Signed-off-by: gesong.samuel <[email protected]>
Signed-off-by: Michael Froh <[email protected]>
Co-authored-by: gesong.samuel <[email protected]>
Co-authored-by: Michael Froh <[email protected]>
…om 1.27.0-alpha to 1.29.0-alpha (#17000)

Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: kkewwei <[email protected]>
Signed-off-by: kkewwei <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.