CatIndexTool implementation #1582

dbwiddis · 2023-11-02T01:22:52Z

Description

Implements Cat Index Tool with edge case handling and tests

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

dbwiddis · 2023-11-02T04:22:28Z

Test failures unassociated with this PR:

46 tests completed, 1 failed, 9 skipped
Tests with failures:
 - org.opensearch.ml.rest.RestMLDeployModelActionIT.testReDeployModel

Tests pass locally:

➜  ml-commons git:(cat-index) ./gradlew :opensearch-ml-algorithms:test --tests 'CatIndexToolTests'   
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.1.1
  OS Info               : Mac OS X 13.6 (x86_64)
  JDK Version           : 17 (Amazon Corretto JDK)
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/amazon-corretto-17.jdk/Contents/Home
  Random Testing Seed   : BBF6D34AA5D9DD5B
  In FIPS 140 mode      : false
=======================================

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/8.1.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 5s
11 actionable tasks: 1 executed, 10 up-to-date

dhrubo-os · 2023-11-02T20:39:36Z

ml-algorithms/src/main/java/org/opensearch/ml/engine/tools/CatIndexTool.java

+        if (parameters.containsKey("indices")) {
+            @SuppressWarnings("unchecked")
+            List<String> indexList = gson.fromJson(parameters.get("indices"), List.class);
+            indices = indexList.toArray(new String[0]);


what happens if a request comes to cat an system index?

Huh. Looks like I need to use the getSettings approach here

Fixed to match REST API

Signed-off-by: Daniel Widdis <[email protected]>

Updated version of awssdk (opensearch-project#1605) (opensearch-project#1606) Signed-off-by: Owais Kazi <[email protected]> (cherry picked from commit 7e44eb2) Co-authored-by: Owais Kazi <[email protected]> create agent Signed-off-by: Jing Zhang <[email protected]> CatIndexTool implementation (opensearch-project#1582) * CatIndexTool implementation Signed-off-by: Daniel Widdis <[email protected]> * Match implementation to REST API to respect index permissions Signed-off-by: Daniel Widdis <[email protected]> * Javadoc fixes Signed-off-by: Daniel Widdis <[email protected]> * Reduce fields to exactly match _cat/indices Signed-off-by: Daniel Widdis <[email protected]> * Add TODO, clean up unused code/imports Signed-off-by: Daniel Widdis <[email protected]> * Rebase with upstream Signed-off-by: Daniel Widdis <[email protected]> * Remove alias getters/setters accidentally kept when rebasing Signed-off-by: Daniel Widdis <[email protected]> * Update test json format to list Signed-off-by: Daniel Widdis <[email protected]> * Remove unused modelId Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Arjun kumar Giri <[email protected]>

#1611) * Add support for including intermediay tool response to agent response Signed-off-by: Arjun kumar Giri <[email protected]> * Add include intermediary response support to react agent runner Signed-off-by: Arjun kumar Giri <[email protected]> * Rebase from upstream Updated version of awssdk (#1605) (#1606) Signed-off-by: Owais Kazi <[email protected]> (cherry picked from commit 7e44eb2) Co-authored-by: Owais Kazi <[email protected]> create agent Signed-off-by: Jing Zhang <[email protected]> CatIndexTool implementation (#1582) * CatIndexTool implementation Signed-off-by: Daniel Widdis <[email protected]> * Match implementation to REST API to respect index permissions Signed-off-by: Daniel Widdis <[email protected]> * Javadoc fixes Signed-off-by: Daniel Widdis <[email protected]> * Reduce fields to exactly match _cat/indices Signed-off-by: Daniel Widdis <[email protected]> * Add TODO, clean up unused code/imports Signed-off-by: Daniel Widdis <[email protected]> * Rebase with upstream Signed-off-by: Daniel Widdis <[email protected]> * Remove alias getters/setters accidentally kept when rebasing Signed-off-by: Daniel Widdis <[email protected]> * Update test json format to list Signed-off-by: Daniel Widdis <[email protected]> * Remove unused modelId Signed-off-by: Daniel Widdis <[email protected]> --------- Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: Arjun kumar Giri <[email protected]> * Revert "Rebase from upstream" This reverts commit 49cff61. Signed-off-by: Arjun kumar Giri <[email protected]> * Add unit tests Signed-off-by: Arjun kumar Giri <[email protected]> * Rebase upstream changes Signed-off-by: Arjun kumar Giri <[email protected]> * Fixed PR feedback Signed-off-by: Arjun kumar Giri <[email protected]> --------- Signed-off-by: Arjun kumar Giri <[email protected]> Signed-off-by: Daniel Widdis <[email protected]> Signed-off-by: arjunkumargiri <[email protected]> Co-authored-by: Arjun kumar Giri <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:22 — with GitHub Actions Failure

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:22 — with GitHub Actions Error

dbwiddis force-pushed the cat-index branch from 665586d to 027d06e Compare November 2, 2023 01:25

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:26 — with GitHub Actions Error

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:26 — with GitHub Actions Failure

dbwiddis force-pushed the cat-index branch from 027d06e to 3d03512 Compare November 2, 2023 01:37

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:37 — with GitHub Actions Failure

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:37 — with GitHub Actions Error

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:37 — with GitHub Actions Failure

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 01:37 — with GitHub Actions Error

dbwiddis force-pushed the cat-index branch from 3d03512 to 3790f8c Compare November 2, 2023 04:04

dbwiddis temporarily deployed to ml-commons-cicd-env November 2, 2023 04:05 — with GitHub Actions Inactive

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 04:05 — with GitHub Actions Error

dbwiddis temporarily deployed to ml-commons-cicd-env November 2, 2023 04:05 — with GitHub Actions Inactive

dbwiddis had a problem deploying to ml-commons-cicd-env November 2, 2023 04:05 — with GitHub Actions Failure

dbwiddis requested review from jngz-es, ylwu-amzn and dhrubo-os November 2, 2023 04:23

dhrubo-os reviewed Nov 2, 2023

View reviewed changes

dbwiddis had a problem deploying to ml-commons-cicd-env November 4, 2023 20:09 — with GitHub Actions Failure

dbwiddis temporarily deployed to ml-commons-cicd-env November 4, 2023 20:09 — with GitHub Actions Inactive

dbwiddis had a problem deploying to ml-commons-cicd-env November 4, 2023 20:09 — with GitHub Actions Error

dbwiddis temporarily deployed to ml-commons-cicd-env November 4, 2023 20:09 — with GitHub Actions Inactive

dbwiddis requested a review from dhrubo-os November 4, 2023 20:10

dbwiddis temporarily deployed to ml-commons-cicd-env November 4, 2023 20:24 — with GitHub Actions Inactive

dbwiddis temporarily deployed to ml-commons-cicd-env November 8, 2023 00:10 — with GitHub Actions Inactive

Rebase with upstream

eab03af

Signed-off-by: Daniel Widdis <[email protected]>

dbwiddis force-pushed the cat-index branch from c0df260 to eab03af Compare November 8, 2023 00:11

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 00:11 — with GitHub Actions Error

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 00:11 — with GitHub Actions Failure

Remove alias getters/setters accidentally kept when rebasing

3013eee

Signed-off-by: Daniel Widdis <[email protected]>

dbwiddis temporarily deployed to ml-commons-cicd-env November 8, 2023 01:33 — with GitHub Actions Inactive

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 01:33 — with GitHub Actions Error

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 01:33 — with GitHub Actions Failure

dbwiddis temporarily deployed to ml-commons-cicd-env November 8, 2023 01:33 — with GitHub Actions Inactive

Update test json format to list

14c47a3

Signed-off-by: Daniel Widdis <[email protected]>

dbwiddis temporarily deployed to ml-commons-cicd-env November 8, 2023 07:00 — with GitHub Actions Inactive

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 07:00 — with GitHub Actions Failure

dbwiddis mentioned this pull request Nov 8, 2023

Implement Index Mapping Tool #1609

Merged

5 tasks

Remove unused modelId

33c3b01

Signed-off-by: Daniel Widdis <[email protected]>

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 16:09 — with GitHub Actions Error

dbwiddis temporarily deployed to ml-commons-cicd-env November 8, 2023 16:09 — with GitHub Actions Inactive

dbwiddis had a problem deploying to ml-commons-cicd-env November 8, 2023 16:09 — with GitHub Actions Failure

dbwiddis temporarily deployed to ml-commons-cicd-env November 8, 2023 16:10 — with GitHub Actions Inactive

dhrubo-os approved these changes Nov 8, 2023

View reviewed changes

dhrubo-os merged commit 7d1741c into opensearch-project:feature/agent_framework_dev Nov 8, 2023
4 of 7 checks passed

dbwiddis deleted the cat-index branch November 8, 2023 20:29

ohltyler mentioned this pull request Nov 14, 2023

[FEATURE] Tool development for Cat Index Tool #1547

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CatIndexTool implementation #1582

CatIndexTool implementation #1582

dbwiddis commented Nov 2, 2023 •

edited

Loading

dbwiddis commented Nov 2, 2023

dhrubo-os Nov 2, 2023

dbwiddis Nov 2, 2023

dbwiddis Nov 4, 2023

CatIndexTool implementation #1582

CatIndexTool implementation #1582

Conversation

dbwiddis commented Nov 2, 2023 • edited Loading

Description

Issues Resolved

Check List

dbwiddis commented Nov 2, 2023

dhrubo-os Nov 2, 2023

Choose a reason for hiding this comment

dbwiddis Nov 2, 2023

Choose a reason for hiding this comment

dbwiddis Nov 4, 2023

Choose a reason for hiding this comment

dbwiddis commented Nov 2, 2023 •

edited

Loading