Skip to content

Commit

Permalink
[ES3] Consolidate tech preview limitations + serverless differences, … (
Browse files Browse the repository at this point in the history
  • Loading branch information
leemthompo authored Nov 21, 2024
1 parent 1680745 commit 353b9dd
Show file tree
Hide file tree
Showing 4 changed files with 141 additions and 43 deletions.
7 changes: 2 additions & 5 deletions serverless/index-serverless-elasticsearch.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
++++

include::./pages/what-is-elasticsearch-serverless.asciidoc[leveloffset=+2]
include::./pages/serverless-differences.asciidoc[leveloffset=+2]

include::./pages/pricing.asciidoc[leveloffset=+2]

Expand Down Expand Up @@ -43,8 +44,4 @@ include::./pages/search-your-data-semantic-search-elser.asciidoc[leveloffset=+4]

include::./pages/explore-your-data.asciidoc[leveloffset=+2]

include::./pages/search-playground.asciidoc[leveloffset=+2]

include::./pages/serverless-differences.asciidoc[leveloffset=+2]

include::./pages/technical-preview-limitations.asciidoc[leveloffset=+2]
include::./pages/search-playground.asciidoc[leveloffset=+2]
153 changes: 135 additions & 18 deletions serverless/pages/serverless-differences.asciidoc
Original file line number Diff line number Diff line change
@@ -1,17 +1,111 @@
[[elasticsearch-differences]]
= Differences from other {es} offerings
++++
<titleabbrev>Serverless differences</titleabbrev>
++++

// :description: Understand how {es-serverless} differs from Elastic Cloud Hosted and self-managed offerings.
// :keywords: serverless, elasticsearch

++++
<titleabbrev>Serverless differences</titleabbrev>
++++
<<what-is-elasticsearch-serverless,{es-serverless}>> handles all the infrastructure management for you, providing a fully managed {es} service.

If you've used {es} before, you'll notice some differences in how you work with the service on {serverless-full}, because a number of APIs and settings are not required for serverless projects.

This guide helps you understand what's different, what's available, and how to work effectively when running {es} on {serverless-full}.

[discrete]
[[elasticsearch-differences-serverless-infrastructure-management]]
== Fully managed infrastructure

{es-serverless} manages all infrastructure automatically, including:

* Cluster scaling and optimization
* Node management and allocation
* Shard distribution and replication
* Resource utilization and monitoring

This fully managed approach means many traditional {es} infrastructure APIs and settings are not available to end users, as detailed in the following sections.

[discrete]
[[elasticsearch-differences-serverless-apis-availability]]
== API availability

Because {es-serverless} manages infrastructure automatically, certain APIs are not available, while others remain fully accessible.

[TIP]
====
Refer to the https://www.elastic.co/docs/api/doc/elasticsearch-serverless[{es-serverless} API reference] for a complete list of available APIs.
====

The following categories of operations are unavailable:

Infrastructure operations::
* All `_nodes/*` operations
* All `_cluster/*` operations
* Most `_cat/*` operations, except for index-related operations such as `/_cat/indices` and `/_cat/aliases`

Storage and backup::
* All `_snapshot/*` operations
* Repository management operations

Index management::
* `indices/close` operations
* `indices/open` operations
* Recovery and stats operations
* Force merge operations

When attempting to use an unavailable API, you'll receive a clear error message:

[source,json]
----
{
"error": {
"root_cause": [
{
"type": "api_not_available_exception",
"reason": "Request for uri [/<API_ENDPOINT>] with method [<METHOD>] exists but is not available when running in serverless mode"
}
],
"status": 410
}
}
----

[discrete]
[[elasticsearch-differences-serverless-settings-availability]]
== Settings availability

In {es-serverless}, you can only configure {ref}/index-modules.html#index-modules-settings[index-level settings].
Cluster-level settings and node-level settings are not required by end users and the `elasticsearch.yml` file is fully managed by Elastic.

Available settings::
*Index-level settings*: Settings that control how {es} documents are processed, stored, and searched are available to end users. These include:
* Analysis configuration
* Mapping parameters
* Search/query settings
* Indexing settings such as `refresh_interval`

Managed settings::
*Infrastructure-related settings*: Settings that affect cluster resources or data distribution are not available to end users. These include:
* Node configurations
* Cluster topology
* Shard allocation
* Resource management

[discrete]
[[elasticsearch-differences-serverless-feature-categories]]
== Feature availability

Some features that are available in Elastic Cloud Hosted and self-managed offerings are not available in {es-serverless}.
These features have either been replaced by a new feature, or are not applicable in the new Serverless architecture:
These features have either been replaced by a new feature, are planned to be released in future, or are not applicable in the new serverless architecture.

[discrete]
[[elasticsearch-differences-serverless-features-replaced]]
=== Replaced features

* **Index lifecycle management ({ilm-init})** is not available, in favor of <<index-management,**data stream lifecycle**>>.
These features have been replaced by a new feature and are therefore not available on {es-serverless}:

* *Index lifecycle management ({ilm-init})* is not available, in favor of <<index-management,**data stream lifecycle**>>.
+
In an Elastic Cloud Hosted or self-managed environment, {ilm-init} lets you automatically transition indices through data tiers according to your
performance needs and retention requirements. This allows you to balance hardware costs with performance. {es-serverless} eliminates this
Expand All @@ -20,21 +114,44 @@ complexity by optimizing your cluster performance for you.
Data stream lifecycle is an optimized lifecycle tool that lets you focus on the most common lifecycle management needs, without unnecessary
hardware-centric concepts like data tiers.
+
* **Watcher** is not available, in favor of Kibana Alerts**.
* *Watcher* is not available, in favor of <<rules-alerts, *Alerts*>>.
+
Kibana Alerts allows rich integrations across use cases like APM, metrics, security, and uptime. Prepackaged rule types simplify setup and
hide the details of complex, domain-specific detections, while providing a consistent interface across Kibana.

[discrete]
[[elasticsearch-differences-serverless-feature-planned]]
=== Planned features

The following features are planned for future support in all {serverless-full} projects:

* Reindexing from remote clusters
* Cross-project search and replication
* Snapshot and restore
* Migrations from non-serverless deployments
* Audit logging
* Authentication realms (native/SAML/OIDC/Kerberos/JWT)
* Clone index API
* Traffic filtering and VPCs

The following {es-serverless} project-specific features are planned for future support:

* {ref}/behavioral-analytics-overview.html[Behavioral Analytics]
* {ref}/search-application-overview.html[Search Applications]
* {enterprise-search-ref}/crawler.html[Managed web crawler]
+
You can use the https://github.com/elastic/crawler[self-managed web crawler] in the meantime.
+
* Certain APIs, API parameters, index, cluster and node level settings are not available. Refer to our
<<elasticsearch-http-apis,REST API references>> for a list of available APIs.
+
{es-serverless} manages the underlying Elastic cluster for you, optimizing nodes, shards, and replicas for your use case.
Because of this, various management and monitoring APIs, API parameters and settings are not available on Serverless.
+
* {ref}/search-aggregations-metrics-scripted-metric-aggregation.html[Scripted metric aggregations] are not available.
* {ref}/es-native-connectors.html[Elastic managed Search connectors]
+
You can use {ref}/es-build-connector.html[self-managed Search connectors] in the meantime.

.Other limitations
[IMPORTANT]
====
For serverless technical preview limitations, refer to <<elasticsearch-technical-preview-limitations,Technical preview limitations>>.
====
[discrete]
[[elasticsearch-differences-serverless-feature-unavailable]]
=== Unplanned features

The following features are not available in {es-serverless} and are not planned for future support:

* https://www.elastic.co/guide/en/cloud/current/ec-custom-bundles.html[Custom plugins and bundles]
* https://www.elastic.co/guide/en/elasticsearch/hadoop/current/reference.html[{es} for Apache Hadoop]
* {ref}/search-aggregations-metrics-scripted-metric-aggregation.html[Scripted metric aggregations]
18 changes: 3 additions & 15 deletions serverless/pages/technical-preview-limitations.asciidoc
Original file line number Diff line number Diff line change
@@ -1,22 +1,10 @@
[[elasticsearch-technical-preview-limitations]]
= Technical preview limitations
= Limitations

// :description: Review the limitations that apply to {es-serverless} projects.
// :keywords: serverless, elasticsearch

The following are currently not available:

* Custom plugins and custom bundles
* Reindexing from remote clusters
* Cross-cluster search and cross-cluster replication
* Snapshot and restore
* Clone index API
* Migrations from non-serverless {es} deployments. In the interim, you can <<elasticsearch-ingest-data-through-logstash,use Logstash>> to move data to and from serverless projects.
* Audit logging
* {es} for Apache Hadoop
// This page is not included in the index file, so it is not visible in the navigation menu anymore. HTTP redirects will be set up.

Currently, workloads outside of the following ranges may experience higher latencies (greater than sub-second):

* Search queries on indices greater than 150GB
* Index queries per second (QPS) greater than 1000
* Search queries per second (QPS) greater than 1800
Refer to <<serverless-differences,{es-serverless} differences>> for a comparison of {es-serverless} with other Elastic Cloud offerings.
6 changes: 1 addition & 5 deletions serverless/pages/what-is-elasticsearch-serverless.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ Elasticsearch allows you to build custom applications. Whether you have structur
If you haven't used {es} before, first learn the basics in the https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-intro.html[core {es} documentation].
====


{es} is one of the three available project types on <<intro,{serverless-full}>>.

This project type enables you to use the core functionality of {es}: searching, indexing, storing, and analyzing data of all shapes and sizes.
Expand All @@ -32,12 +31,9 @@ a| [.card-title]#<<elasticsearch-get-started,*Create a project →*>># +
Get started by creating your first {es} project on serverless.

| 🔌
a| [.card-title]#<<elasticsearch-get-started,*Connect to {es} →*>># +
a| [.card-title]#<<elasticsearch-connecting-to-es-serverless-endpoint,*Connect to {es} →*>># +
Learn how to connect your applications to your {es-serverless} endpoint.

// TODO add coming link to new page about connecting to your serverless endpoint
// <<elasticsearch-connecting-to-es-serverless-endpoint,*Connect your application →*>>

| ⤵️
a| [.card-title]#<<elasticsearch-ingest-your-data,*Ingest data →*>># +
Learn how to get your data into {es} and start building your search application.
Expand Down

0 comments on commit 353b9dd

Please sign in to comment.