Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support Transform as an ISM action #733

Closed
tanqiuliu opened this issue Apr 8, 2023 · 1 comment
Closed

[FEATURE] Support Transform as an ISM action #733

tanqiuliu opened this issue Apr 8, 2023 · 1 comment
Assignees
Labels
action Index management actions enhancement New request ism action

Comments

@tanqiuliu
Copy link
Contributor

tanqiuliu commented Apr 8, 2023

Is your feature request related to a problem?
I would like to apply different kinds of aggregation on different subset of data from the source index as a part of the index lifecycle. Currently, transform supports dataSelectionQuery while rollup does not. Also, transform appears to support more aggregation types compared with rollup.

What solution would you like?
Support Transform as an ISM action. Similar to ISM rollup, the ISM transform should contain the following fields:

{
  "description": "Creating transform through ISM",
  "target_index": "target",
  "page_size": 1000,
  "data_selection_query": {
    "match_all": {}
  }
  "groups": [
    {
      "date_histogram": {
        "fixed_interval": "60m",
        "source_field": "order_date",
        "target_field": "order_date",
        "timezone": "America/Los_Angeles"
      }
    },
    {
      "terms": {
        "source_field": "customer_gender",
        "target_field": "customer_gender"
      }
    },
    {
      "terms": {
        "source_field": "day_of_week",
        "target_field": "day_of_week"
      }
    }
  ],
  "aggregations": [
    {
      "source_field": "taxless_total_price",
      "metrics": [
        {
          "sum": {}
        }
      ]
    },
    {
      "source_field": "total_quantity",
      "metrics": [
        {
          "avg": {}
        },
        {
          "max": {}
        }
      ]
    }
  ]
}

What alternatives have you considered?
Periodically monitor the lifecycle of index and keep creating transform jobs. But native support in ISM would be cleaner.

Do you have any additional context?
I can pick up the implementation if accepted

@tanqiuliu tanqiuliu changed the title [FEATURE] Support Transform as an action of ISM [FEATURE] Support Transform as an ISM action Apr 8, 2023
@tanqiuliu
Copy link
Contributor Author

Created a PR #760

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action Index management actions enhancement New request ism action
Projects
Status: Done
Development

No branches or pull requests

4 participants