Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derived field updates for 2.17 #8222

Closed
wants to merge 5 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 71 additions & 2 deletions _field-types/supported-field-types/derived.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ Despite the potential performance impact of query-time computations, the flexibi

Currently, derived fields have the following limitations:

- **Aggregation, scoring, and sorting**: Not yet supported.
- **Scoring and sorting**: Not yet supported.
- **With 2.17 Aggregations are supported with limitations**: Unsupported aggregation types: Geo, Significant Terms/Text and Scripted Metric
- **Dashboard support**: These fields are not displayed in the list of available fields in OpenSearch Dashboards. However, you can still use them for filtering if you know the derived field name.
- **Chained derived fields**: One derived field cannot be used to define another derived field.
- **Join field type**: Derived fields are not supported for the [join field type]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/join/).
- **Concurrent segment search**: Derived fields are not supported for [concurrent segment search]({{site.url}}{{site.baseurl}}/search-plugins/concurrent-segment-search/).

We are planning to address these limitations in future versions.

Expand Down Expand Up @@ -541,6 +541,75 @@ The response specifies highlighting in the `url` field:
```
</details>

## Aggregations

Most aggregation types are supported with derived fields since 2.17. Geo Aggregations, Significant Terms/Text and Scripted Metric are not supported.
For example, the following request creates a simple terms aggregation on the `method` derived field:

```json
POST /logs/_search
{
"size": 0,
"aggs": {
"methods": {
"terms": {
"field": "method"
}
}
}
}
```
{% include copy-curl.html %}
The response contains the following buckets:

<details markdown="block">
<summary>
Response
</summary>
{: .text-delta}

```json
{
"took" : 12,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 5,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"methods" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "GET",
"doc_count" : 2
},
{
"key" : "POST",
"doc_count" : 2
},
{
"key" : "DELETE",
"doc_count" : 1
}
]
}
}
}
```
</details>

## Performance

Derived fields are not indexed but are computed dynamically by retrieving values from the `_source` field or doc values. Thus, they run more slowly. To improve performance, try the following:
Expand Down
Loading