[FEATURE] Index patterns resource type

[arichtman-srt]

Is your feature request related to a problem?

I'm unable to create all the necessary objects by way of Terraform.
Index Patterns, which I think you need for data streams, are not present in the provider.
Though it might be a Kibana concept?

What solution would you like?

A resource type opensearch_index_pattern that operates about the same as everything else in the provider.

What alternatives have you considered?

Combination of local_file with jsonencode and null resource of local_script to manually curl the API.

Do you have any additional context?

https://www.elastic.co/guide/en/kibana/7.17/index-patterns.html

[prudhvigodithi]

[Untriage]
Something as follows should be useful.

resource "opensearch_index_pattern" "logsdata" {
  name           = "log"
  pattern = "log-*"
  timeFieldName = "@timestamp"
}

@arichtman-srt can you take a look and contribute this index_pattern setting to the provider?
Adding @phillbaker

[prudhvigodithi]

From code side I did found the index_patterns settings https://github.com/opensearch-project/terraform-provider-opensearch/blob/main/provider/resource_opensearch_role.go#L354.

[alemairebe]

@prudhvigodithi
It is doable as per doc at https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/dashboard_object

resource "opensearch_dashboard_object" "test_index_pattern_v7" {
  body = <<EOF
[
  {
    "_id": "index-pattern:cloudwatch",
    "_type": "doc",
    "_source": {
      "type": "index-pattern",
      "index-pattern": {
        "title": "cloudwatch-*",
        "timeFieldName": "timestamp"
      }
    }
  }
]
EOF
}

[jgournet]

Just a few things based on the opensearch_dashboard_object approach:

1. I needed to remove

"_type": "doc",

otherwise terraform wants to change this at every plan/apply - I guess it's related to the opensearch version.
2) Anyone knows how to set a particular index-pattern as "default" ?
3) there's an issue where just visiting any of those index templates in kibana will populate all the available fields into the index-pattern itself; so when re-applying terraform, it wants to wipe all that data. I'm not sure yet of the impact (does it reset all popularity counter too ?)

=> @rblcoder : Because of those 3 points above, I think there's a case for actually creating a dedicated terraform resource. do you think it's worth re-opening ?

[rblcoder]

@PhilippReinke would it be possible for you to look into the query above regarding index patterns?

[rblcoder]

A working example of index pattern
#184 (comment)

[PhilippReinke]

Hi :)

here is an example index-pattern that is based on the comment @rblcoder linked but without tenant.

By default you've got an index of the form "security-auditlog-*" and with the latest OpenSearch version you can create an index-pattern with

terraform {
  required_providers {
    opensearch = {
      source = "opensearch-project/opensearch"
    }
  }
}

provider "opensearch" {
  url         = "https://localhost:9200"
  username    = "admin"
  password    = "myStrongPassword123@456"
  healthcheck = "false"
  insecure    = "true"
}

resource "opensearch_dashboard_object" "test_pattern" {
  body = <<EOF
[
  {
    "_id": "index-pattern:test_pattern",
    "_source": {
      "index-pattern": {
        "title": "security-auditlog-*",
        "timeFieldName": "@timestamp",
        "fields": "[{\"count\":0,\"name\":\"@timestamp\",\"type\":\"date\",\"esTypes\":[\"date\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"count\":0,\"name\":\"_id\",\"type\":\"string\",\"esTypes\":[\"_id\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"count\":0,\"name\":\"_index\",\"type\":\"string\",\"esTypes\":[\"_index\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"count\":0,\"name\":\"_score\",\"type\":\"number\",\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"_source\",\"type\":\"_source\",\"esTypes\":[\"_source\"],\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"_type\",\"type\":\"string\",\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_category\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_category.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_category\"}}},{\"count\":0,\"name\":\"audit_cluster_name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_cluster_name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_cluster_name\"}}},{\"count\":0,\"name\":\"audit_compliance_doc_version\",\"type\":\"number\",\"esTypes\":[\"long\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"count\":0,\"name\":\"audit_compliance_operation\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_compliance_operation.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_compliance_operation\"}}},{\"count\":0,\"name\":\"audit_format_version\",\"type\":\"number\",\"esTypes\":[\"long\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"count\":0,\"name\":\"audit_node_host_address\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_node_host_address.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_node_host_address\"}}},{\"count\":0,\"name\":\"audit_node_host_name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_node_host_name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_node_host_name\"}}},{\"count\":0,\"name\":\"audit_node_id\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_node_id.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_node_id\"}}},{\"count\":0,\"name\":\"audit_node_name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_node_name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_node_name\"}}},{\"count\":0,\"name\":\"audit_request_body\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_request_body.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_request_body\"}}},{\"count\":0,\"name\":\"audit_request_effective_user\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_request_effective_user.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_request_effective_user\"}}},{\"count\":0,\"name\":\"audit_request_effective_user_is_admin\",\"type\":\"boolean\",\"esTypes\":[\"boolean\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"count\":0,\"name\":\"audit_request_layer\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_request_layer.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_request_layer\"}}},{\"count\":0,\"name\":\"audit_request_origin\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_request_origin.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_request_origin\"}}},{\"count\":0,\"name\":\"audit_request_privilege\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_request_privilege.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_request_privilege\"}}},{\"count\":0,\"name\":\"audit_request_remote_address\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_request_remote_address.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_request_remote_address\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Accept\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Accept-Encoding\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Accept-Encoding.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Accept-Encoding\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Accept-Language\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Accept-Language.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Accept-Language\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Accept.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Accept\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Connection\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Connection.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Connection\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Cookie\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Cookie.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Cookie\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Host\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Host.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Host\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-Dest\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-Dest.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Sec-Fetch-Dest\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-Mode\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-Mode.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Sec-Fetch-Mode\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-Site\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-Site.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Sec-Fetch-Site\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-User\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-Fetch-User.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Sec-Fetch-User\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-GPC\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Sec-GPC.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Sec-GPC\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.Upgrade-Insecure-Requests\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.Upgrade-Insecure-Requests.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.Upgrade-Insecure-Requests\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.User-Agent\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.User-Agent.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.User-Agent\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.sec-ch-ua\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.sec-ch-ua-mobile\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.sec-ch-ua-mobile.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.sec-ch-ua-mobile\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.sec-ch-ua-platform\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_headers.sec-ch-ua-platform.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.sec-ch-ua-platform\"}}},{\"count\":0,\"name\":\"audit_rest_request_headers.sec-ch-ua.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_headers.sec-ch-ua\"}}},{\"count\":0,\"name\":\"audit_rest_request_method\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_method.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_method\"}}},{\"count\":0,\"name\":\"audit_rest_request_path\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_rest_request_path.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_rest_request_path\"}}},{\"count\":0,\"name\":\"audit_trace_doc_id\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_trace_doc_id.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_trace_doc_id\"}}},{\"count\":0,\"name\":\"audit_trace_indices\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_trace_indices.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_trace_indices\"}}},{\"count\":0,\"name\":\"audit_trace_resolved_indices\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_trace_resolved_indices.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_trace_resolved_indices\"}}},{\"count\":0,\"name\":\"audit_trace_shard_id\",\"type\":\"number\",\"esTypes\":[\"long\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"count\":0,\"name\":\"audit_trace_task_id\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_trace_task_id.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_trace_task_id\"}}},{\"count\":0,\"name\":\"audit_transport_headers.X-Opaque-Id\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_transport_headers.X-Opaque-Id.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_transport_headers.X-Opaque-Id\"}}},{\"count\":0,\"name\":\"audit_transport_headers._opendistro_security_initial_action_class_header\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_transport_headers._opendistro_security_initial_action_class_header.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_transport_headers._opendistro_security_initial_action_class_header\"}}},{\"count\":0,\"name\":\"audit_transport_headers._opendistro_security_origin_header\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_transport_headers._opendistro_security_origin_header.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_transport_headers._opendistro_security_origin_header\"}}},{\"count\":0,\"name\":\"audit_transport_headers._opendistro_security_remotecn\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_transport_headers._opendistro_security_remotecn.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_transport_headers._opendistro_security_remotecn\"}}},{\"count\":0,\"name\":\"audit_transport_request_type\",\"type\":\"string\",\"esTypes\":[\"text\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"count\":0,\"name\":\"audit_transport_request_type.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"audit_transport_request_type\"}}}]"
      },
      "type": "index-pattern",
      "references": []
    }
  }
]
EOF
}

where I have used the sample docker-compose.yml to test it.

Regarding your bullet points

1. That's my guess too.
2. I don't think that's possible (not even in the web ui).
3. I faced the same issue in the past. OpenSearch Dashboard modifies the object if it does not comply to its internal rules. You really need to adapt your Terraform file.

The provider stores the dashboard objects in the .kibana index. So, you can fetch the object under DevTools with

GET .kibana/_doc/index-pattern:test_pattern

Regarding a dedicated terraform resource I think that would be really nice feature. However, one would have to deal with different OpenSearch versions and the dashboards objects are not well documented. Actually, there should be a terraform resource for every dashboard object. The current opensearch_dashboard_object solution allows to manage all resources but lets the user struggle with different versions and modifications by OpenSearch.

For Kibana there is a Saved Objects API that even states

Do not write documents directly to the .kibana index. When you write directly to the .kibana index, the data becomes corrupted and permanently breaks future Kibana versions.

In the future there might be a documented Saved Objects API in OpenSearch. The provider could communicate with the OpenSearch Dashboard API and not the OpenSearch API anymore.

[rblcoder]

Thank you @PhilippReinke

[jgournet]

Thank you @rblcoder and @PhilippReinke - that was really well explained and interesting

[rblcoder]

@jgournet Glad it was helpful

[prudhvigodithi]

Thanks @PhilippReinke @rblcoder, I agree on the following points.

• The provider should have a seperate terraform resource for each dashboards specific operation.
• The dashboards specific terraform resource should be using the dashboards API.
• Using dashboards API and a seperate terraform resource will not have any problems for the user in terms on JSON format specific to dashboard as the provider will be handling the CRUD operations based on the version.

To begin with we should move to dashboard saved_object API for opensearch_dashboard_object once the opensearch-project/OpenSearch-Dashboards#987, opensearch-project/OpenSearch-Dashboards#6310, opensearch-project/documentation-website#3799 are resolved.

But looks like saved_object API is being used in few places https://forum.opensearch.org/t/saved-objects-api-authentication/4576, adding @seraphjiang @kavilla to confirm if its ok for us to start using the saved_object API for modifying dashboard resources.

@bbarani @getsaurabh02 @dblock @gaiksaya @peterzhuamazon

[prudhvigodithi]

Here is an open issue for creating index_pattern with API opensearch-project/OpenSearch-Dashboards#910, but there is a comment here that says as possible with api/saved_objects/index-pattern to create index_pattern, if this is the case we can have a new terraform resource as opensearch_dashboard_index_pattern, WDYT @PhilippReinke @rblcoder @jgournet @arichtman-srt @alemairebe?

[arichtman-srt]

It's a question of how tightly we want to model the API. We can work with the generic dashboards object for now so it's no longer a blocking issue, just less ergonomic :). Thanks to all involved for your time too!