Skip to content

Commit

Permalink
Merge branch '8.x' into hide-inference-service-api
Browse files Browse the repository at this point in the history
  • Loading branch information
elasticmachine authored Nov 19, 2024
2 parents 092f2db + df1130f commit 4f1b993
Show file tree
Hide file tree
Showing 119 changed files with 2,348 additions and 1,167 deletions.
6 changes: 6 additions & 0 deletions docs/changelog/116277.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
pr: 116277
summary: Update Semantic Query To Handle Zero Size Responses
area: Vector Search
type: bug
issues:
- 116083
5 changes: 5 additions & 0 deletions docs/changelog/116915.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 116915
summary: Improve message about insecure S3 settings
area: Snapshot/Restore
type: enhancement
issues: []
5 changes: 5 additions & 0 deletions docs/changelog/116918.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 116918
summary: Split searchable snapshot into multiple repo operations
area: Snapshot/Restore
type: enhancement
issues: []
5 changes: 5 additions & 0 deletions docs/changelog/116931.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 116931
summary: Enable built-in Inference Endpoints and default for Semantic Text
area: "Machine Learning"
type: enhancement
issues: []
5 changes: 5 additions & 0 deletions docs/changelog/116942.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 116942
summary: Fix handling of bulk requests with semantic text fields and delete ops
area: Relevance
type: bug
issues: []
5 changes: 5 additions & 0 deletions docs/changelog/116995.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 116995
summary: "Apm-data: disable date_detection for all apm data streams"
area: Data streams
type: enhancement
issues: []
2 changes: 2 additions & 0 deletions docs/reference/esql/esql-language.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Detailed reference documentation for the {esql} language:
* <<esql-enrich-data>>
* <<esql-process-data-with-dissect-and-grok>>
* <<esql-implicit-casting>>
* <<esql-time-spans>>

include::esql-syntax.asciidoc[]
include::esql-commands.asciidoc[]
Expand All @@ -23,3 +24,4 @@ include::multivalued-fields.asciidoc[]
include::esql-process-data-with-dissect-grok.asciidoc[]
include::esql-enrich-data.asciidoc[]
include::implicit-casting.asciidoc[]
include::time-spans.asciidoc[]
18 changes: 6 additions & 12 deletions docs/reference/esql/esql-syntax.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -157,21 +157,15 @@ FROM employees
==== Timespan literals

Datetime intervals and timespans can be expressed using timespan literals.
Timespan literals are a combination of a number and a qualifier. These
qualifiers are supported:

* `millisecond`/`milliseconds`/`ms`
* `second`/`seconds`/`sec`/`s`
* `minute`/`minutes`/`min`
* `hour`/`hours`/`h`
* `day`/`days`/`d`
* `week`/`weeks`/`w`
* `month`/`months`/`mo`
* `quarter`/`quarters`/`q`
* `year`/`years`/`yr`/`y`
Timespan literals are a combination of a number and a temporal unit. The
supported temporal units are listed in <<esql-time-spans-table, time span unit>>.
More examples of the usages of time spans can be found in
<<esql-time-spans, Use time spans in ES|QL>>.


Timespan literals are not whitespace sensitive. These expressions are all valid:

* `1day`
* `1 day`
* `1 day`

2 changes: 2 additions & 0 deletions docs/reference/esql/functions/binary.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ Supported types:

include::types/greater_than_or_equal.asciidoc[]

[[esql-add]]
==== Add `+`
[.text-center]
image::esql/functions/signature/add.svg[Embedded,opts=inline]
Expand All @@ -98,6 +99,7 @@ Supported types:

include::types/add.asciidoc[]

[[esql-subtract]]
==== Subtract `-`
[.text-center]
image::esql/functions/signature/sub.svg[Embedded,opts=inline]
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/esql/functions/examples/count.asciidoc

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 22 additions & 18 deletions docs/reference/esql/implicit-casting.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<titleabbrev>Implicit casting</titleabbrev>
++++

Often users will input `date`, `ip`, `version`, `date_period` or `time_duration` as simple strings in their queries for use in predicates, functions, or expressions. {esql} provides <<esql-type-conversion-functions, type conversion functions>> to explicitly convert these strings into the desired data types.
Often users will input `date`, `date_period`, `time_duration`, `ip` or `version` as simple strings in their queries for use in predicates, functions, or expressions. {esql} provides <<esql-type-conversion-functions, type conversion functions>> to explicitly convert these strings into the desired data types.

Without implicit casting users must explicitly code these `to_X` functions in their queries, when string literals don't match the target data types they are assigned or compared to. Here is an example of using `to_datetime` to explicitly perform a data type conversion.

Expand All @@ -18,7 +18,10 @@ FROM employees
| LIMIT 1
----

Implicit casting improves usability, by automatically converting string literals to the target data type. This is most useful when the target data type is `date`, `ip`, `version`, `date_period` or `time_duration`. It is natural to specify these as a string in queries.
[discrete]
[[esql-implicit-casting-example]]
==== Implicit casting example
Implicit casting automatically converts string literals to the target data type. This allows users to specify string values for types like `date`, `date_period`, `time_duration`, `ip` and `version` in their queries.

The first query can be coded without calling the `to_datetime` function, as follows:

Expand All @@ -31,35 +34,36 @@ FROM employees
| LIMIT 1
----

[float]
=== Implicit casting support
[discrete]
[[esql-implicit-casting-supported-operations]]
==== Operations that support implicit casting

The following table details which {esql} operations support implicit casting for different data types.

[%header.monospaced.styled,format=dsv,separator=|]
|===
||ScalarFunction*|Operator*|<<esql-group-functions, GroupingFunction>>|<<esql-agg-functions, AggregateFunction>>
|DATE|Y|Y|Y|N
|IP|Y|Y|Y|N
|VERSION|Y|Y|Y|N
|BOOLEAN|Y|Y|Y|N
|DATE_PERIOD/TIME_DURATION|Y|N|Y|N
|ScalarFunctions|Operators|<<esql-group-functions, GroupingFunctions>>|<<esql-agg-functions, AggregateFunctions>>
DATE|Y|Y|Y|N
DATE_PERIOD/TIME_DURATION|Y|N|Y|N
IP|Y|Y|Y|N
VERSION|Y|Y|Y|N
BOOLEAN|Y|Y|Y|N
|===

ScalarFunction* includes:
ScalarFunctions includes:

<<esql-conditional-functions-and-expressions, Conditional Functions and Expressions>>
* <<esql-conditional-functions-and-expressions, Conditional Functions and Expressions>>

<<esql-date-time-functions, Date and Time Functions>>
* <<esql-date-time-functions, Date and Time Functions>>

<<esql-ip-functions, IP Functions>>
* <<esql-ip-functions, IP Functions>>


Operator* includes:
Operators includes:

<<esql-binary-operators, Binary Operators>>
* <<esql-binary-operators, Binary Operators>>

<<esql-unary-operators, Unary Operator>>
* <<esql-unary-operators, Unary Operator>>

<<esql-in-operator, IN>>
* <<esql-in-operator, IN>>

111 changes: 111 additions & 0 deletions docs/reference/esql/time-spans.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
[[esql-time-spans]]
=== {esql} time spans

++++
<titleabbrev>Time spans</titleabbrev>
++++

Time spans represent intervals between two datetime values. There are currently two supported types of time spans:

* `DATE_PERIOD` specifies intervals in years, quarters, months, weeks and days
* `TIME_DURATION` specifies intervals in hours, minutes, seconds and milliseconds

A time span requires two elements: an integer value and a temporal unit.

Time spans work with grouping functions such as <<esql-bucket, BUCKET>>, scalar functions such as <<esql-date_trunc, DATE_TRUNC>> and arithmetic operators such as <<esql-add, `+`>> and <<esql-subtract, `-`>>. Convert strings to time spans using <<esql-to_dateperiod, TO_DATEPERIOD>>, <<esql-to_timeduration, TO_TIMEDURATION>>, or the cast operators `::DATE_PERIOD`, `::TIME_DURATION`.

[discrete]
[[esql-time-spans-examples]]
==== Examples of using time spans in {esql}


With `BUCKET`:
[source.merge.styled,esql]
----
include::{esql-specs}/bucket.csv-spec[tag=docsBucketWeeklyHistogramWithSpan]
----
[%header.monospaced.styled,format=dsv,separator=|]
|===
include::{esql-specs}/bucket.csv-spec[tag=docsBucketWeeklyHistogramWithSpan-result]
|===


With `DATE_TRUNC`:
[source.merge.styled,esql]
----
include::{esql-specs}/date.csv-spec[tag=docsDateTrunc]
----
[%header.monospaced.styled,format=dsv,separator=|]
|===
include::{esql-specs}/date.csv-spec[tag=docsDateTrunc-result]
|===


With `+` and/or `-`:
[source.merge.styled,esql]
----
include::{esql-specs}/date.csv-spec[tag=docsNowWhere]
----
[%header.monospaced.styled,format=dsv,separator=|]
|===
include::{esql-specs}/date.csv-spec[tag=docsNowWhere-result]
|===


When a time span is provided as a named parameter in string format, `TO_DATEPERIOD`, `::DATE_PERIOD`, `TO_TIMEDURATION` or `::TIME_DURATION` can be used to convert to its corresponding time span value for arithmetic operations like `+` and/or `-`.
[source, esql]
----
POST /_query
{
"query": """
FROM employees
| EVAL x = hire_date + ?timespan::DATE_PERIOD, y = hire_date - TO_DATEPERIOD(?timespan)
""",
"params": [{"timespan" : "1 day"}]
}
----

When a time span is provided as a named parameter in string format, it can be automatically converted to its corresponding time span value in grouping functions and scalar functions, like `BUCKET` and `DATE_TRUNC`.
[source, esql]
----
POST /_query
{
"query": """
FROM employees
| WHERE hire_date >= "1985-01-01T00:00:00Z" AND hire_date < "1986-01-01T00:00:00Z"
| STATS hires_per_week = COUNT(*) BY week = BUCKET(hire_date, ?timespan)
| SORT week
""",
"params": [{"timespan" : "1 week"}]
}
----

[source, esql]
----
POST /_query
{
"query": """
FROM employees
| KEEP first_name, last_name, hire_date
| EVAL year_hired = DATE_TRUNC(?timespan, hire_date)
""",
"params": [{"timespan" : "1 year"}]
}
----

[discrete]
[[esql-time-spans-table]]
==== Supported temporal units
[%header.monospaced.styled,format=dsv,separator=|]
|===
Temporal Units|Valid Abbreviations
year|y, yr, years
quarter|q, quarters
month|mo, months
week|w, weeks
day|d, days
hour|h, hours
minute|min, minutes
second|s, sec, seconds
millisecond|ms, milliseconds
|===
2 changes: 1 addition & 1 deletion gradle/build.versions.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[versions]
asm = "9.6"
asm = "9.7.1"
jackson = "2.15.0"
junit5 = "5.8.1"
spock = "2.1-groovy-3.0"
Expand Down
Loading

0 comments on commit 4f1b993

Please sign in to comment.