Skip to content

Commit

Permalink
Update and edit logsdb docs for logsdb / synthetic source GA (#118303)
Browse files Browse the repository at this point in the history
* Update licensing; fix screenshots; edit generally

* Small edit for clarity and style

* Update docs/reference/index-modules.asciidoc

Co-authored-by: Liam Thompson <[email protected]>

* Apply changes from review

Co-authored-by: Liam Thompson <[email protected]>

* Address review comments

* Match similar change from review

* More changes from review

* Apply suggestions from review

Co-authored-by: shainaraskas <[email protected]>

* Apply suggestions from review

Co-authored-by: shainaraskas <[email protected]>

* Update docs/reference/data-streams/logs.asciidoc

Co-authored-by: shainaraskas <[email protected]>

* Apply suggestions from review

Co-authored-by: shainaraskas <[email protected]>

* Apply suggestions from review

* Change to general subscription note

* Apply suggestions from review

Co-authored-by: Oleksandr Kolomiiets <[email protected]>

* Apply suggestions from review

Co-authored-by: Oleksandr Kolomiiets <[email protected]>

* Apply suggestions from review; additional edits

* Apply suggestions from review; clarity tweaks

* Restore previous paragraph structure and context

---------

Co-authored-by: Liam Thompson <[email protected]>
Co-authored-by: shainaraskas <[email protected]>
Co-authored-by: Oleksandr Kolomiiets <[email protected]>
  • Loading branch information
4 people authored Dec 11, 2024
1 parent a8a4a7b commit ae9bb90
Show file tree
Hide file tree
Showing 9 changed files with 112 additions and 111 deletions.
197 changes: 103 additions & 94 deletions docs/reference/data-streams/logs.asciidoc

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/reference/data-streams/tsds.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ metrics data. Only use a TSDS if you typically add metrics data to {es} in near
real-time and `@timestamp` order.

A TSDS is only intended for metrics data. For other timestamped data, such as
logs or traces, use a regular data stream.
logs or traces, use a <<logs-data-stream,logs data stream>> or regular data stream.

[discrete]
[[differences-from-regular-data-stream]]
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file modified docs/reference/images/index-mgmt/management-index-templates.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 2 additions & 3 deletions docs/reference/index-modules.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,9 @@ Index mode supports the following values:

`standard`::: Standard indexing with default settings.

`time_series`::: Index mode optimized for storage of metrics documented in <<tsds-index-settings,TSDS Settings>>.
`tsds`::: _(data streams only)_ Index mode optimized for storage of metrics. For more information, see <<tsds-index-settings>>.

`logsdb`::: Index mode optimized for storage of logs. It applies default sort settings on the `hostname` and `timestamp` fields and uses <<synthetic-source,synthetic `_source`>>. <<index-modules-index-sorting,Index sorting>> on different fields is still allowed.
preview:[]
`logsdb`::: _(data streams only)_ Index mode optimized for <<logs-data-stream,logs>>.

[[routing-partition-size]] `index.routing_partition_size`::

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/indices/index-mgmt.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ This value is the time period for which your data is guaranteed to be stored. Da
Elasticsearch at a later time.

[role="screenshot"]
image::images/index-mgmt/management-data-stream.png[Data stream details]
image::images/index-mgmt/management-data-stream-fields.png[Data stream details]

* To view more information about a data stream, such as its generation or its
current index lifecycle policy, click the stream's name. From this view, you can navigate to *Discover* to
Expand Down
6 changes: 3 additions & 3 deletions docs/reference/indices/put-index-template.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -115,10 +115,10 @@ See <<create-index-template,create an index template>>.
`index_mode`::
(Optional, string) Type of data stream to create. Valid values are `null`
(regular data stream) and `time_series` (<<tsds,time series data stream>>).
(standard data stream), `time_series` (<<tsds,time series data stream>>) and `logsdb`
(<<logs-data-stream,logs data stream>>).
+
If `time_series`, each backing index has an `index.mode` index setting of
`time_series`.
The template's `index_mode` sets the `index.mode` of the backing index.
=====

`index_patterns`::
Expand Down
11 changes: 2 additions & 9 deletions docs/reference/mapping/fields/synthetic-source.asciidoc
Original file line number Diff line number Diff line change
@@ -1,17 +1,10 @@
[[synthetic-source]]
==== Synthetic `_source`

IMPORTANT: Synthetic `_source` is Generally Available only for TSDB indices
(indices that have `index.mode` set to `time_series`). For other indices,
synthetic `_source` is in technical preview. Features in technical preview may
be changed or removed in a future release. Elastic will work to fix
any issues, but features in technical preview are not subject to the support SLA
of official GA features.

Though very handy to have around, the source field takes up a significant amount
of space on disk. Instead of storing source documents on disk exactly as you
send them, Elasticsearch can reconstruct source content on the fly upon retrieval.
Enable this by using the value `synthetic` for the index setting `index.mapping.source.mode`:
To enable this https://www.elastic.co/subscriptions[subscription] feature, use the value `synthetic` for the index setting `index.mapping.source.mode`:

[source,console,id=enable-synthetic-source-example]
----
Expand All @@ -30,7 +23,7 @@ PUT idx
----
// TESTSETUP

While this on the fly reconstruction is *generally* slower than saving the source
While this on-the-fly reconstruction is _generally_ slower than saving the source
documents verbatim and loading them at query time, it saves a lot of storage
space. Additional latency can be avoided by not loading `_source` field in queries when it is not needed.

Expand Down

0 comments on commit ae9bb90

Please sign in to comment.