Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Clarifying CSV limitations #175208

Merged
merged 3 commits into from
Jan 25, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion docs/user/reporting/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ You access the options from the *Share* menu in the toolbar. The sharing options

* *PNG Reports* — Generate and download PNG files of dashboards and visualizations. PNG reports are a link:https://www.elastic.co/subscriptions[subscription feature].

* *CSV Reports* — Generate and download CSV files of saved searches.
* *CSV Reports* &mdash; Generate and download CSV files of saved searches. <<csv-limitations,Certain limitations apply>>.

* *CSV Download* &mdash; Generate and download CSV files of *Lens* visualizations.

Expand Down Expand Up @@ -72,6 +72,12 @@ NOTE: Reports are stored in {es} and managed by the `kibana-reporting` {ilm}
more about {ilm-init} policies, refer to the {es}
{ref}/index-lifecycle-management.html[{ilm-init} documentation].

[float]
[[csv-limitations]]
=== CSV reports limitations

include::reporting-csv-limitations.asciidoc[]

[float]
[[share-a-direct-link]]
== Share direct links
Expand Down
10 changes: 10 additions & 0 deletions docs/user/reporting/reporting-csv-limitations.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
We recommend using CSV reports to export moderate amounts of data only. The feature enables analysis of data in external tools, but it is not intended for bulk export or to backup Elasticsearch data. Report timeout and incomplete data issues are likely if you are exporting data where:

- More than 250 MB of data is being exported
- Data is stored on slow storage tiers
- Any shard needed for the search is unavailable
- Network latency between nodes is high
- Cross-cluster search is used
- ES|QL is used and result row count exceeds the limits of ES|QL queries

Use filters to create multiple smaller reports, or extract the data you need directly with the Elasticsearch APIs. See Point in time API, or SQL with CSV response data format. Reporting parameters can be adjusted to overcome some of these limiting scenarios. Results are dependent on data size, availability, and latency factors and are not guaranteed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use filters to create multiple smaller reports, or extract the data you need directly with the Elasticsearch APIs. See Point in time API, or SQL with CSV response data format. Reporting parameters can be adjusted to overcome some of these limiting scenarios. Results are dependent on data size, availability, and latency factors and are not guaranteed.
To work around the limitations, use filters to create multiple smaller reports, or extract the data you need directly with the Elasticsearch APIs. See Point in time API, or SQL with CSV response data format. Reporting parameters can be adjusted to overcome some of these limiting scenarios. Results are dependent on data size, availability, and latency factors and are not guaranteed.

I think scroll should be included here as well, and the phrases should be linked.

Point-in-time API: https://www.elastic.co/guide/en/elasticsearch/reference/current/point-in-time-api.html
Scroll API: https://www.elastic.co/guide/en/elasticsearch/reference/current/scroll-api.html
SQL with CSV response: https://www.elastic.co/guide/en/elasticsearch/reference/current/sql-rest-format.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I also added a link to reporting settings and mentioned ES|QL with csv response type