Skip to content

Commit

Permalink
Merge pull request opensearch-project#432 from stockholmux/oss-to-os-…
Browse files Browse the repository at this point in the history 
…blog-post

edit pass
  • Loading branch information
@stockholmux
stockholmux authored Oct 29, 2021
2 parents 1b6e469 + 19ce3c4 commit 6d57afa
Showing 1 changed file with 22 additions and 23 deletions.
45 changes: 22 additions & 23 deletions _posts/2021-10-29-moving-from-opensource-elasticsearch-to-opensearch.markdown
View file
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@
---
layout: post
title: "Moving from Open Source Elasticsearch to OpenSearch"
title: "Moving from open source Elasticsearch to OpenSearch"
authors:
- vemsarat
date: 2021-10-29
categories:
- technical-posts
twittercard:
description: "As OpenSearch is now shipping out generally available releases, I wanted to provide an overview of how moving from Open Source Elasticsearch to OpenSearch looks like, and help you prepare for the move... "
description: "As OpenSearch is now shipping out generally available releases, I wanted to provide an overview of how moving from open source Elasticsearch to OpenSearch looks like, and help you prepare for the move... "
---
As OpenSearch is now shipping out generally available releases, I wanted to provide an overview of how moving from Open Source Elasticsearch to OpenSearch looks like, and help you prepare for the move.
As OpenSearch is now shipping out generally available releases, I wanted to provide an overview of how moving from open source Elasticsearch to OpenSearch looks like, and help you prepare for the move.

### What is Elasticsearch OSS/OpenSearch Upgrade?
### What is an open source Elasticsearch to OpenSearch Upgrade?

An *upgrade* is moving a cluster to either new minor version or new major version. Major version upgrades come with breaking changes which is why it should be carefully upgraded via recommended paths. Minor version upgrades are relatively simpler.
An *upgrade* is moving a cluster to either a new minor version or new major version. Major version upgrades come with breaking changes which is why it should be carefully upgraded via recommended paths. Minor version upgrades are relatively simple.

When talking about upgrades, settings and indices are upgraded:

* Cluster Settings - All dynamic settings which are supported in the new cluster will be migrated and the ones which were deprecated will be *archived*.
* Indices - All compatible indices(i.e. all your data) will be readable/writable by the new software version.
* Indices - All compatible indices (i.e. all your data) will be readable/writable by the new software version.

Static settings defined in `elasticsearch.yml`/`opensearch.yml` are not moved automatically during an upgrade.
Static settings defined in `elasticsearch.yml` / `opensearch.yml` are not moved automatically during an upgrade.
OpenSearch 1.1.0 is bundled with a new [upgrade tool](https://opensearch.org/docs/latest/upgrade-to/upgrade-to/#upgrade-tool) which connects to an Elasticsearch cluster and moves static settings which are defined in `elasticsearch.yml` to `opensearch.yml` automatically.

Yay! Now you know what an upgrade is, lets buckle up for the upgrade.

### Preparing for the upgrade

If you run Open Source Elasticsearch or older minor versions of OpenSearch and you would like to upgrade to the latest OpenSearch (1.1.0), here are few things you should take care of.
If you run open source Elasticsearch or older minor versions of OpenSearch and you would like to upgrade to the latest OpenSearch (1.1.0), below are a few things you should take care of.

First, **always** take a backup of your data. Take a snapshot of your existing cluster, you could follow the guidance from the earlier [blog post](https://opensearch.org/blog/technical-posts/2021/07/how-to-upgrade-from-opendistro-to-opensearch/).
Second, understand the version compatibility between the existing cluster to the version you are moving into. As OpenSearch was a fork from the last Open Source version of Elasticsearch, 7.10.2:
Second, understand the version compatibility between the existing cluster to the version you are moving into. As OpenSearch was a fork from the last open source version of Elasticsearch, 7.10.2:

* Indices are compatible with current and previous major versions (i.e. OpenSearch 1.x could read/write indices from Open Source Elasticsearch 6.x and 7.x).
* Wire compatibility works with all versions of current major and last minor of the previous major version (i.e. OpenSearch 1.x could join an Open Source Elasticsearch 6.8.x and 7.x cluster).
* Indices are compatible with current and previous major versions (i.e. OpenSearch 1.x could read/write indices from open source Elasticsearch 6.x and 7.x).
* Wire compatibility works with all versions of current major and last minor of the previous major version (i.e. OpenSearch 1.x could join an open source Elasticsearch 6.8.x and 7.x cluster).

Indices that were created prior to Open Source Elasticsearch 6.x will have to be either re-indexed or deleted in order to upgrade the cluster to OpenSearch. Having incompatible indices will fail the cluster to start.
Indices that were created prior to open source Elasticsearch 6.x will have to be either re-indexed or deleted in order to upgrade the cluster to OpenSearch. Having incompatible indices will cause a failure of the cluster to start.

Use [Reindex API](https://opensearch.org/docs/latest/opensearch/rest-api/document-apis/reindex/) to migrate your data from indices created in versions prior to 6.x to a new version. Here is an example:

Expand All @@ -51,25 +51,25 @@ POST /_reindex
}
```

### Support Matrix for Elasticsearch OSS to OpenSearch
### Support Matrix for open source Elasticsearch to OpenSearch

|Elasticsearch OSS |Recommended Upgrade Path |
|Open Source Elasticsearch|Recommended Upgrade Path |
|--- |--- |
|7.0 to 7.10.2 |Restart/Rolling upgrade to OpenSearch 1.x |
|6.8 |Restart/Rolling upgrade to 7.10.2 |
|6.0 to 6.7 |Restart/Rolling upgrade to 6.8 |
|5.6 |Restart/Rolling upgrade to 6.8 |
|5.0 to 5.5 |Restart/Rolling upgrade to 5.6 |

Finally, now that you are ready for the upgrade, you could download the latest version of OpenSearch from the [OpenSearch downloads page](https://opensearch.org/downloads.html), and if you need help with installation instructions follow the install and [configure guide](https://opensearch.org/docs/opensearch/install/index/) on the documentation website.
Finally, now that you are ready for the upgrade, you can download the latest version of OpenSearch from the [OpenSearch downloads page](https://opensearch.org/downloads.html), and if you need help with installation instructions follow the install and [configure guide](https://opensearch.org/docs/opensearch/install/index/) on the documentation website.

As outlined in the earlier [blog post](https://opensearch.org/blog/technical-posts/2021/07/how-to-upgrade-from-opendistro-to-opensearch/), as Open Source Elasticsearch is the foundation for OpenDistro and it has the same upgrade paths. You could follow through the steps for the path you choose to upgrade your cluster.
As outlined in the earlier [blog post](https://opensearch.org/blog/technical-posts/2021/07/how-to-upgrade-from-opendistro-to-opensearch/), open source Elasticsearch is the foundation for Open Distro and it has the same upgrade paths. You can follow through the steps for the path you choose to upgrade your cluster.

### Clients and Tools

As you are moving your workloads to OpenSearch, there are probably few tools like Beats, Logstash, Fluentd, Fluent Bit etc you have used with Elasticsearch OSS and would like to use them with OpenSearch. Unfortunately few clients and tools have version checks builtin and do not work out of the box.
As you are moving your workloads to OpenSearch, there are probably a few tools like Beats, Logstash, Fluentd, and Fluent Bit you have used with open source Elasticsearch and would like to use them with OpenSearch. Unfortunately some clients and tools have version checks builtin and do not work out of the box.

To work around this, OpenSearch is built with a compatibility flag which will respond back with the version field as `7.10.2`. It is a cluster setting and you could set it via:
To work around this, OpenSearch is built with a compatibility flag which will respond back with the version field as `7.10.2`. It is a cluster setting and you can set it via:

```
PUT _cluster/settings
Expand All @@ -82,16 +82,15 @@ PUT _cluster/settings
}
```

OpenSearch [documentation](https://opensearch.org/docs/latest/clients/agents-and-ingestion-tools/index/) lists down a compatibility matrix for each tool and pointers to download each.
OpenSearch [documentation](https://opensearch.org/docs/latest/clients/agents-and-ingestion-tools/index/) provides links to download and compatibility matrices for each tool.

As many users use Logstash to ingest data into the cluster, opensearch-project has built a [Logstash output plugin](https://opensearch.org/downloads.html) to work with OpenSearch.
As many users use Logstash to ingest data into the cluster, the OpenSearch project has built a [Logstash output plugin](https://opensearch.org/downloads.html) to work specifically with OpenSearch.

The project announced [native client support](https://opensearch.org/blog/community/2021/08/community-clients/), a few clients like [Python, NodeJS and JS](https://opensearch.org/blog/community/2021/09/opensearch-py-js-go/) contributed by the community are now ready for production use.
Since the project announced [native client support](https://opensearch.org/blog/community/2021/08/community-clients/) a few clients like [Python, NodeJS, and Go](https://opensearch.org/blog/community/2021/09/opensearch-py-js-go/) are now ready for production use.

### Going Forward

Now that you are empowered with this knowledge, choose the right path which is suitable for your workload and always remember to take a backup.

I hope this helps in choosing the right path and let us know if you any feedback or suggestions.

In the coming days, look for a partner spotlight post that goes into more detail about migrating from open source Elasticsearch to OpenSearch.

0 comments on commit 6d57afa

Please sign in to comment.