insert/delete embeddings when content is published

[wezell]

there is something ... undone ... in the plugin. When you create an embeddings DB/index, we use the same database as dotCMS is using. We have workflow tasks that can automatically add/remove content from the embeddings db, but we do not currently have a way to push publish the embeddings db, or automatically run the indexing process on the reciever. (edited)

very much like we do not have a way to push site search indexes.

Short term, our answer is going to be to manually create indexes on prod. but we could solve longer term a few ways.

• we could add a jdbc connection string to our app and if that is set, use that to connect to the database that has the embeddings. this would allow us to separate out any ai db activity to another db. In the case of auth/prod, it would allow you to manage the prod embeddings db in auth.
• We could create a batch job that added/rebuilt the embeddings db every x interval
• We could update the EmbeddingContentListener to automatically add content to the embeddings when published (on the reciever) and remove it when unpublished. This would put all embeddings in a single db

none of these is too hard.

See discussion here:

https://dotcms.slack.com/archives/CQMHM7PNJ/p1704392333618319