You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem?
When re-indexing, text_embedding processors and ml_inference processors will run the prediction again even when the inference fields are already existed in the document. text_embedding processors will over-write the the inference field, while ml_inference will not write to the inference field, then throw exception or skip writing to the document.
What solution would you like?
In the ingest processor that used ml inference, we should check the model output field name, for example, text_embedding field exists in the document before the prediction tasks run. If the field already exists, skip the predictions.
In this case, when re-index happens, it won't run the prediction tasks again if the field is existed.
What alternatives have you considered?
Welcome any other suggestions.
The text was updated successfully, but these errors were encountered:
Maybe it would be nice to add a parameter on ml ingest processors called overwrite: Option[Boolean] if true it overwrites current embeddings upon reindexing if false it does not overwrite
It is nice to still have the option to overwrite in case we change the ML model to one that outputs different embeddings
Is your feature request related to a problem?
When re-indexing,
text_embedding
processors andml_inference
processors will run the prediction again even when the inference fields are already existed in the document.text_embedding
processors will over-write the the inference field, whileml_inference
will not write to the inference field, then throw exception or skip writing to the document.What solution would you like?
In the ingest processor that used ml inference, we should check the model output field name, for example,
text_embedding
field exists in the document before the prediction tasks run. If the field already exists, skip the predictions.In this case, when re-index happens, it won't run the prediction tasks again if the field is existed.
What alternatives have you considered?
Welcome any other suggestions.
The text was updated successfully, but these errors were encountered: