OpenSearch - Docker - Compose

Dockerized cluster architecture for OpenSearch with compose.

Key concepts

• OpenSearch is the successor of OpenDistro
• OpenSearch = Elasticsearch
• OpenSearch Dashboards = Kibana

Cluster setup

Raise your host's ulimits for ElasticSearch to handle high I/O :

sudo sysctl -w vm.max_map_count=512000
# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`

Now, we will generate the certificates for the cluster :

# You may want to edit the OPENDISTRO_DN variable first
bash generate-certs.sh

Start the cluster :

docker-compose up -d

Wait about 30 seconds and run securityadmin to initialize the security plugin :

docker-compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"

Find all the configuration files in the container's /usr/share/opensearch/config/opensearch-security directory. You might want to mount them as volumes.

Access OpenSearch Dashboards through https://localhost:5601

Default username is admin and password is admin

Take a look at OpenSearch's internal users documentation to add, remove or update a user.

Hot-warm architecture setup

Use a hot-warm cluster architecture if you have data that you rarely want to update or search so you can place them on lower-cost storage nodes.

Hot-warm architecture cluster setup instructions...

Raise your host's ulimits for ElasticSearch to handle high I/O :

sudo sysctl -w vm.max_map_count=512000
# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`

Now, we will generate the certificates for the cluster :

# You may want to edit the OPENDISTRO_DN variable first
bash generate-certs-hot-warm.sh

Adjust Xms/Xmx parameters and start the cluster :

docker-compose -f docker-compose.hot-warm.yml up -d

Wait about 60 seconds and run securityadmin to initialize the security plugin :

docker-compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"

Find all the configuration files in the container's /usr/share/opensearch/config/opensearch-security directory. You might want to mount them as volumes.

Access OpenSearch Dashboards through https://localhost:5601

Default username is admin and password is admin

Take a look at OpenSearch's internal users documentation to add, remove or update a user.

To add an index to a warm node :

PUT newindex
{
  "settings": {
    "index.routing.allocation.require.temp": "warm"
  }
}

You might want to use Index State Management (ILM) to automatically move old indices from hot to warm nodes.

Why OpenSearch

• Fully open source (including plugins)
• Fully under Apache 2.0 license
• Advanced security plugin (free)
• Alerting plugin (free)
• Allows you to perform SQL queries against ElasticSearch
• Maintained by AWS and used for its cloud services