Provider 2.x support (#62)

Signed-off-by: Prudhvi Godithi <[email protected]>

.github/workflows/test.yml

@@ -29,20 +29,19 @@ jobs:
         go-version: [1.18.x]
         os: [ubuntu-latest]
         os-major-version:
-          - "1-opensearch"
+          - "2-opensearch"
         include:
-        - os-major-version: "1-opensearch"
-          version: 1.1.0
+        - os-major-version: "2-opensearch"
+          version: 2
           oss-image: "opensearchproject/opensearch"
-          OS_OPENDISTRO_IMAGE: "opensearchproject/opensearch:1.1.0"
-          OS_DASHBOARD_IMAGE: "opensearchproject/opensearch-dashboards:1.1.0"
+          OS_IMAGE: "opensearchproject/opensearch:2"
+          OS_DASHBOARD_IMAGE: "opensearchproject/opensearch-dashboards:2"
           OPENSEARCH_PREFIX: "plugins.security"
-          OSS_ENV_VAR: "plugins.security.disabled=true"
     needs: [lint]
     name: Test against OS ${{ matrix.os-major-version }} on ${{ matrix.go-version }}/${{ matrix.os }}
     runs-on: ${{ matrix.os }}
     env:
-      OS_OPENDISTRO_IMAGE: "${{matrix.OS_OPENDISTRO_IMAGE}}"
+      OS_IMAGE: "${{matrix.OS_IMAGE}}"
       OS_DASHBOARD_IMAGE: "${{matrix.OS_DASHBOARD_IMAGE}}"
       OPENSEARCH_PREFIX: "${{matrix.OPENSEARCH_PREFIX}}"
       OSS_ENV_VAR: "${{matrix.OSS_ENV_VAR}}"
@@ -92,21 +91,13 @@ jobs:
     - name: Wait for Opensearch
       # ensure that OS has come up and is available
       run: |
-        ./script/wait-for-endpoint --timeout=20 http://localhost:9200
-    - name: Warm up OpenDistro/Opensearch
-      # - OpenDistro lazily initializes its indexes, see
-      #   https://github.com/opendistro-for-elasticsearch/alerting/issues/60
-      run: |
-        if [ -n "$OS_OPENDISTRO_IMAGE" ]; then
-          ./script/wait-for-endpoint --timeout=120 http://admin:admin@localhost:9220
-          curl -s -v -X POST -H 'Content-type: application/json' -d '{"name":"_warmup","type":"slack","slack":{"url": "http://www.example.com"}}' http://admin:admin@localhost:9220/_opendistro/_alerting/destinations
-        fi
+        ./script/wait-for-endpoint --timeout=20 http://admin:admin@localhost:9200
     - name: Dump docker logs on failure
       if: failure()
       uses: jwalton/gh-docker-logs@v2
     - name: Run the tests
       run: |
-        export OPENSEARCH_URL=http://localhost:9200
+        export OPENSEARCH_URL=http://admin:admin@localhost:9200
         export TF_LOG=INFO
         TF_ACC=1 go test ./... -v -parallel 20 -cover -short
         # check goreleaser config for deprecations

README.md

@@ -20,9 +20,76 @@ This provider will target compatibility with major versions of Opensearch, each
 
 | Opensearch version | Supported          |
 | -----------        | ---------          |
-| 1.x                | :white_check_mark: |
-| 2.x                | :x:                |
+| 2.x                | :white_check_mark: |
 
+### Supported Functionalities 
+
+Examples of resources can be found in the examples directory.
+
+#### OpenSearch and OpenSearch Dashboards
+
+- [x] [Cluster Settings](https://opensearch.org/docs/latest/api-reference/cluster-api/cluster-settings/)
+- [x] [Audit Config](https://opensearch.org/docs/latest/security/audit-logs/index/)
+- [x] [Component templates](https://opensearch.org/docs/latest/dashboards/im-dashboards/component-templates/)
+- [x] [Index and Composable templates](https://opensearch.org/docs/latest/im-plugin/index-templates/)
+- [x] [Data Streams](https://opensearch.org/docs/2.9/dashboards/im-dashboards/datastream/)
+- [x] [Ingest Pipeline](https://opensearch.org/docs/2.9/api-reference/ingest-apis/create-update-ingest/)
+- [x] [Security](https://opensearch.org/docs/latest/security/index/)
+- [x] [Snapshot Repository](https://opensearch.org/docs/2.9/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore/#register-repository)
+- [x] [Anomaly Detection](https://opensearch.org/docs/latest/observing-your-data/ad/index/)
+- [x] [Index State Management](https://opensearch.org/docs/latest/im-plugin/ism/index/)
+- [x] [Dashboards Visualization](https://opensearch.org/docs/latest/dashboards/visualize/viz-index/)
+- [x] [Dashboards Tenant](https://opensearch.org/docs/latest/security/multi-tenancy/tenant-index/)
+- [x] [Alerting Monitors](https://opensearch.org/docs/latest/observing-your-data/alerting/monitors/)
+
+
+### Running tests locally
+
+```sh
+./script/install-tools
+export OSS_IMAGE="opensearchproject/opensearch:2"
+docker-compose up -d
+docker-compose ps -a
+export OPENSEARCH_URL=http://admin:admin@localhost:9200
+export TF_LOG=INFO
+TF_ACC=1 go test ./... -v -parallel 20 -cover -short
+```
+
+#### To Run Specific Test
+```sh
+cd provider/
+TF_ACC=2 go test -run TestAccOpensearchOpenDistroDashboardTenant  -v -cover -short
+```
+
+#### Fix the go-lint errors
+
+```sh
+golangci-lint run --out-format=github-actions 
+```
+
+
+### Debugging this provider
+
+Build the executable, and start in debug mode:
+
+```console
+$ go build
+$ ./terraform-provider-opensearch -debuggable # or start in debug mode in your IDE
+{"@level":"debug","@message":"plugin address","@timestamp":"2022-05-17T10:10:04.331668+01:00","address":"/var/folders/32/3mbbgs9x0r5bf991ltrl3p280010fs/T/plugin1346340234","network":"unix"}
+Provider started, to attach Terraform set the TF_REATTACH_PROVIDERS env var:
+
+        TF_REATTACH_PROVIDERS='{"registry.terraform.io/opensearch-project/opensearch":{"Protocol":"grpc","ProtocolVersion":5,"Pid":79075,"Test":true,"Addr":{"Network":"unix","String":"/var/folders/32/3mbbgs9x0r5bf991ltrl3p280010fs/T/plugin1346340234"}}}'
+```
+
+In another terminal, you can test your terraform code:
+
+```console
+$ cd <my-project/terraform>
+$ export TF_REATTACH_PROVIDERS=<env var above>
+$ terraform apply
+```
+
+The local provider will be used instead, and you should see debug information printed to the terminal.
 
 ## Version and Branching
 As of now, this terraform-provider-opensearch repository maintains 2 branches:

docker-compose.yml

@@ -2,7 +2,7 @@ version: '3'
 
 services:
   opensearch:
-    image: ${OSS_IMAGE}
+    image:  ${OSS_IMAGE}
     hostname: opensearch
     container_name: opensearch
     networks: 
@@ -15,6 +15,7 @@ services:
       - network.publish_host=127.0.0.1
       - logger.org.opensearch=warn
       - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
+      - "plugins.security.ssl.http.enabled=false"
       - ${OSS_ENV_VAR:-FOO=bar}
     command: ${OS_COMMAND}
     ulimits:
@@ -27,30 +28,6 @@ services:
         hard: -1
     ports:
       - 9200:9200
-  opendistro:
-    image: ${OS_OPENDISTRO_IMAGE:-rwgrim/docker-noop}
-    hostname: opensearch-opendistro
-    container_name: opensearch-opendistro
-    environment:
-      - cluster.name=opendistro
-      - bootstrap.memory_lock=true
-      - discovery.type=single-node
-      - path.repo=/tmp
-      - ${OPENSEARCH_PREFIX:-opendistro_security}.ssl.http.enabled=false
-      - http.port=9220
-      - network.publish_host=127.0.0.1
-      - logger.org.opensearch=warn
-      - "OPENSEARCH_JAVA_OPTS=-Xms1g -Xmx1g"
-    ulimits:
-      nproc: 65536
-      nofile:
-        soft: 65536
-        hard: 65536
-      memlock:
-        soft: -1
-        hard: -1
-    ports:
-      - 9220:9220
 networks:
   opensearch:
     driver: bridge

docs/index.md

@@ -64,7 +64,7 @@ EOF
 
 - `aws_access_key` (String) The access key for use with AWS opensearch Service domains
 - `aws_assume_role_arn` (String) Amazon Resource Name of an IAM Role to assume prior to making AWS API calls.
-- `aws_assume_role_external_id` (Optional) - External ID configured in the role to assume prior to making AWS API calls.
+- `aws_assume_role_external_id` (String) External ID configured in the IAM policy of the IAM Role to assume prior to making AWS API calls.
 - `aws_profile` (String) The AWS profile for use with AWS opensearch Service domains
 - `aws_region` (String) The AWS region for use in signing of AWS opensearch requests. Must be specified in order to use AWS URL signing with AWS OpenSearch endpoint exposed on a custom DNS domain.
 - `aws_secret_key` (String) The secret key for use with AWS opensearch Service domains

docs/resources/composable_index_template.md

@@ -3,12 +3,12 @@
 page_title: "opensearch_composable_index_template Resource - terraform-provider-opensearch"
 subcategory: ""
 description: |-
-  Provides an Composable index template resource. This resource uses the /_index_template endpoint of the API that is available since version 7.8. Use opensearch_index_template if you are using older versions or if you want to keep using legacy Index Templates in versions 7.8+.
+  Provides an Composable index template resource. This resource uses the /_index_template endpoint of the API that is available since version 2.0.0. Use opensearch_index_template if you are using older versions or if you want to keep using legacy Index Templates.
 ---
 
 # opensearch_composable_index_template (Resource)
 
-Provides an Composable index template resource. This resource uses the `/_index_template` endpoint of the API that is available since version 7.8. Use `opensearch_index_template` if you are using older versions or if you want to keep using legacy Index Templates in versions 7.8+.
+Provides an Composable index template resource. This resource uses the `/_index_template` endpoint of the API that is available since version 2.0.0. Use `opensearch_index_template` if you are using older versions or if you want to keep using legacy Index Templates.
 
 ## Example Usage
 

docs/resources/dashboard_object.md

@@ -3,15 +3,12 @@
 page_title: "opensearch_dashboard_object Resource - terraform-provider-opensearch"
 subcategory: ""
 description: |-
-  Provides an OpenSearch Dashboards object resource. This resource interacts directly with the underlying OpenSearch index backing Dashboards, so the format must match what Dashboards the version of Dashboards is expecting. Dashboards v5 and v6 will export all objects in Dashboards v5 format, so the exported objects cannot be used as a source for body in this resource - directly pulling the JSON from a Dashboards index of the same version of OpenSearch targeted by the provider is a workaround.
-  With the removal of mapping types in OpenSearch, the Dashboards index changed from v5 to >= v6, previously the document mapping type had the Dashboards object type, however, the _type going forward is doc and the type is within the document, see below. Using v5 doc types in v6 and above will result in errors from OpenSearch after one or more document types are used.
+  Provides an OpenSearch Dashboards object resource. This resource interacts directly with the underlying OpenSearch index backing Dashboards, so the format must match what Dashboards the version of Dashboards is expecting. Dashboards with older versions - directly pulling the JSON from a Dashboards index of the same version of OpenSearch targeted by the provider is a workaround.
 ---
 
 # opensearch_dashboard_object (Resource)
 
-Provides an OpenSearch Dashboards object resource. This resource interacts directly with the underlying OpenSearch index backing Dashboards, so the format must match what Dashboards the version of Dashboards is expecting. Dashboards v5 and v6 will export all objects in Dashboards v5 format, so the exported objects cannot be used as a source for `body` in this resource - directly pulling the JSON from a Dashboards index of the same version of OpenSearch targeted by the provider is a workaround.
-
-With the removal of mapping types in OpenSearch, the Dashboards index changed from v5 to >= v6, previously the document mapping type had the Dashboards object type, however, the `_type` going forward is `doc` and the type is within the document, see below. Using v5 doc types in v6 and above will result in errors from OpenSearch after one or more document types are used.
+Provides an OpenSearch Dashboards object resource. This resource interacts directly with the underlying OpenSearch index backing Dashboards, so the format must match what Dashboards the version of Dashboards is expecting. Dashboards with older versions - directly pulling the JSON from a Dashboards index of the same version of OpenSearch targeted by the provider is a workaround.
 
 ## Example Usage