You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, users cannot customize the index mapping for the OpenSearch index that stores the Flint index data created via Spark SQL.
What solution would you like?
Similarly as index_settings option introduced previously, a new option index_mappings can be added to the create statement in Spark SQL. This would allow users fine grain control over the OpenSearch index mappings.
For example:
CREATE MATERIALIZED VIEW mv_test
AS
SELECT ...
WITH (
index_mappings: '{ "_source": { "enabled": false } }'
)
Note that for the implementation, the index schema generated by the Flint index must be merged with the schema provided in the index_mappings option.
What alternatives have you considered?
Manually modifying the OpenSearch index after the materialized view is created. However, this approach won’t work for configurations that need to be set before indexing begins.
Do you have any additional context?
This feature would provide users the flexibility to optimize the OpenSearch index according to their specific needs. For example, it would allow them to:
Disable the _source field to save space.
Disable doc_values for specific fields to save space.
Force certain field types (e.g., IP type) as a temporary workaround until SparkSQL supports them.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem?
Currently, users cannot customize the index mapping for the OpenSearch index that stores the Flint index data created via Spark SQL.
What solution would you like?
Similarly as
index_settings
option introduced previously, a new optionindex_mappings
can be added to the create statement in Spark SQL. This would allow users fine grain control over the OpenSearch index mappings.For example:
Note that for the implementation, the index schema generated by the Flint index must be merged with the schema provided in the
index_mappings
option.What alternatives have you considered?
Manually modifying the OpenSearch index after the materialized view is created. However, this approach won’t work for configurations that need to be set before indexing begins.
Do you have any additional context?
This feature would provide users the flexibility to optimize the OpenSearch index according to their specific needs. For example, it would allow them to:
The text was updated successfully, but these errors were encountered: