Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Rename ingest processor supports overriding target field if exists #12942

Closed
gaobinlong opened this issue Mar 27, 2024 · 3 comments
Labels
enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing v2.14.0 v3.0.0 Issues and PRs related to version 3.0.0

Comments

@gaobinlong
Copy link
Collaborator

Is your feature request related to a problem? Please describe

The rename ingest processor always throws error when the target field exists in the document, which is not friendly:

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
        "rename": {
          "field": "a",
          "target_field": "b"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "a":1,
        "b":2
      }
    }
  ]
}

, response:

{
  "docs": [
    {
      "error": {
        "root_cause": [
          {
            "type": "illegal_argument_exception",
            "reason": "field [b] already exists"
          }
        ],
        "type": "illegal_argument_exception",
        "reason": "field [b] already exists"
      }
    }
  ]
}

. Just like the Hard_rename rule in the Modify filter plugin of fluent-bit, the rename ingest processor can support something like override_target, which can be used to override the target field if it does exist.

Describe the solution you'd like

Rename ingest processor supports overriding target field if exists.

Related component

Indexing

Describe alternatives you've considered

No response

Additional context

No response

@gaobinlong gaobinlong added enhancement Enhancement or improvement to existing feature or request untriaged labels Mar 27, 2024
@github-actions github-actions bot added the Indexing Indexing, Bulk Indexing and anything related to indexing label Mar 27, 2024
@peternied
Copy link
Member

[Triage - attendees 1 2 3 4 5]
@gaobinlong Thanks for filing this issue, please create a pull request to add this functionality.

Note; it sounds like this scenario might be better served with a new processor named 'move'

@gaobinlong
Copy link
Collaborator Author

[Triage - attendees 1 2 3 4 5] @gaobinlong Thanks for filing this issue, please create a pull request to add this functionality.

Note; it sounds like this scenario might be better served with a new processor named 'move'

Thank you, I've created a PR for this issue, and I think move processor could do more things than the rename processor such as move multiple fields to an object field(just like move multiple files to a directory in Linux), will dive deep into it.

@reta reta added the v2.14.0 label Apr 12, 2024
@github-project-automation github-project-automation bot moved this to Planned work items in Test roadmap format Apr 12, 2024
@reta reta added the v3.0.0 Issues and PRs related to version 3.0.0 label Apr 12, 2024
@gaobinlong
Copy link
Collaborator Author

Closed by #12990.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Indexing Indexing, Bulk Indexing and anything related to indexing v2.14.0 v3.0.0 Issues and PRs related to version 3.0.0
Projects
Status: Planned work items
Development

No branches or pull requests

3 participants