You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, we're using the correctCommonEsqlMistakes (non-llm) function to fix common syntax mistakes, but it's fairly limited, and does not allow functional validation of the generated query.
The common pattern for LLM-based review is the review/correct chain prompting: after the generation, you ask the model to review the generation and list errors and possible improvements, when you run another prompt asking to correct the first generation based on the review of the second one.
The text was updated successfully, but these errors were encountered:
pgayvallet
changed the title
[Nl-to-ESQL] add review/correct steps
[NL-to-ESQL] add review/correct steps
Sep 18, 2024
Note that from various papers (e.g. this one), self-correctness doesn't work very well without external input.
The best approach here could be to perform programmatic validation of the query (ast validation), and then feed the result of the validation to the LLM for correction, instead of just asking it to review its own work without additional insight.
Part of https://github.com/elastic/kibana-team/issues/1264
At the moment, we're using the
correctCommonEsqlMistakes
(non-llm) function to fix common syntax mistakes, but it's fairly limited, and does not allow functional validation of the generated query.The common pattern for LLM-based review is the review/correct chain prompting: after the generation, you ask the model to review the generation and list errors and possible improvements, when you run another prompt asking to correct the first generation based on the review of the second one.
The text was updated successfully, but these errors were encountered: