[ES|QL] Client side validation enhancements #177699
Labels
enhancement
New value added to drive a business result
Feature:ES|QL
ES|QL related features in Kibana
impact:low
Addressing this issue will have a low level of impact on the quality/strength of our product.
Team:ESQL
ES|QL related features in Kibana
Comments
stratoula
added
the
bug
stratoula
added
Team:Visualizations
|
Pinging @elastic/kibana-visualizations (Team:Visualizations) |
stratoula
changed the title
1 task
dej611
added a commit
that referenced
this issue
Mar 7, 2024
## Summary Related issue #177699 Fix variables logic for expressions at `stats by ...` level. Fix validation logic for agg functions within `eval` or `where` scope. Fix validation and autocomplete logic for nested quoted expressions * i.e. ``` from index | eval round(numberField) + 1 | eval `round(numberField) + 1` + 1 | eval ```round(numberField) + 1`` + 1` + 1 | eval ```````round(numberField) + 1```` + 1`` + 1` + 1 | eval ```````````````round(numberField) + 1```````` + 1```` + 1`` + 1` + 1 | keep ```````````````````````````````round(numberField) + 1```````````````` + 1```````` + 1```` + 1`` + 1` ``` * updated `count_distinct` agg definition to have the `precision` second optional param. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Stratoula Kalafateli <[email protected]>
drewdaemon
pushed a commit
to drewdaemon/kibana
that referenced
this issue
Mar 7, 2024
## Summary Related issue elastic#177699 Fix variables logic for expressions at `stats by ...` level. Fix validation logic for agg functions within `eval` or `where` scope. Fix validation and autocomplete logic for nested quoted expressions * i.e. ``` from index | eval round(numberField) + 1 | eval `round(numberField) + 1` + 1 | eval ```round(numberField) + 1`` + 1` + 1 | eval ```````round(numberField) + 1```` + 1`` + 1` + 1 | eval ```````````````round(numberField) + 1```````` + 1```` + 1`` + 1` + 1 | keep ```````````````````````````````round(numberField) + 1```````````````` + 1```````` + 1```` + 1`` + 1` ``` * updated `count_distinct` agg definition to have the `precision` second optional param. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Stratoula Kalafateli <[email protected]> (cherry picked from commit cad276f) # Conflicts: # packages/kbn-monaco/src/esql/lib/ast/validation/esql_validation_meta_tests.json
drewdaemon
added a commit
that referenced
this issue
Mar 21, 2024
## Summary Part of #177699 We had `case` marked as if it required three parameters when in reality it only requires two. <img width="600" alt="Screenshot 2024-03-19 at 4 23 29 PM" src="https://github.com/elastic/kibana/assets/315764/45f7578a-e6ad-4ba9-b71a-05bb1978a384"> Note: we could consider testing these n-1 cases to prevent this kind of bug in the future. Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Marta Bondyra <[email protected]>
drewdaemon
added a commit
to drewdaemon/kibana
that referenced
this issue
Mar 21, 2024
## Summary Part of elastic#177699 We had `case` marked as if it required three parameters when in reality it only requires two. <img width="600" alt="Screenshot 2024-03-19 at 4 23 29 PM" src="https://github.com/elastic/kibana/assets/315764/45f7578a-e6ad-4ba9-b71a-05bb1978a384"> Note: we could consider testing these n-1 cases to prevent this kind of bug in the future. Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Marta Bondyra <[email protected]> (cherry picked from commit 28277c2) # Conflicts: # packages/kbn-monaco/src/esql/lib/ast/validation/esql_validation_meta_tests.json
drewdaemon
referenced
this issue
Mar 21, 2024
# Backport This will backport the following commits from `main` to `8.13`: - [[ES|QL] case only requires two parameters (#179011)](#179011) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Drew Tate","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-03-21T14:34:38Z","message":"[ES|QL] case only requires two parameters (#179011)\n\n## Summary\r\n\r\nPart of https://github.com/elastic/kibana/issues/177699\r\n\r\nWe had `case` marked as if it required three parameters when in reality\r\nit only requires two.\r\n\r\n<img width=\"600\" alt=\"Screenshot 2024-03-19 at 4 23 29 PM\"\r\nsrc=\"https://github.com/elastic/kibana/assets/315764/45f7578a-e6ad-4ba9-b71a-05bb1978a384\">\r\n\r\nNote: we could consider testing these n-1 cases to prevent this kind of\r\nbug in the future.\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>\r\nCo-authored-by: Marta Bondyra <[email protected]>","sha":"28277c25df26796a8aa51cb4b8e82b0483c9cf83","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","backport:prev-minor","v8.14.0"],"number":179011,"url":"https://github.com/elastic/kibana/pull/179011","mergeCommit":{"message":"[ES|QL] case only requires two parameters (#179011)\n\n## Summary\r\n\r\nPart of https://github.com/elastic/kibana/issues/177699\r\n\r\nWe had `case` marked as if it required three parameters when in reality\r\nit only requires two.\r\n\r\n<img width=\"600\" alt=\"Screenshot 2024-03-19 at 4 23 29 PM\"\r\nsrc=\"https://github.com/elastic/kibana/assets/315764/45f7578a-e6ad-4ba9-b71a-05bb1978a384\">\r\n\r\nNote: we could consider testing these n-1 cases to prevent this kind of\r\nbug in the future.\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>\r\nCo-authored-by: Marta Bondyra <[email protected]>","sha":"28277c25df26796a8aa51cb4b8e82b0483c9cf83"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.14.0","labelRegex":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/179011","number":179011,"mergeCommit":{"message":"[ES|QL] case only requires two parameters (#179011)\n\n## Summary\r\n\r\nPart of https://github.com/elastic/kibana/issues/177699\r\n\r\nWe had `case` marked as if it required three parameters when in reality\r\nit only requires two.\r\n\r\n<img width=\"600\" alt=\"Screenshot 2024-03-19 at 4 23 29 PM\"\r\nsrc=\"https://github.com/elastic/kibana/assets/315764/45f7578a-e6ad-4ba9-b71a-05bb1978a384\">\r\n\r\nNote: we could consider testing these n-1 cases to prevent this kind of\r\nbug in the future.\r\n\r\nCo-authored-by: Kibana Machine <[email protected]>\r\nCo-authored-by: Marta Bondyra <[email protected]>","sha":"28277c25df26796a8aa51cb4b8e82b0483c9cf83"}}]}] BACKPORT-->
drewdaemon
added a commit
that referenced
this issue
Mar 26, 2024
# Backport This will backport the following commits from `main` to `8.13`: - [[ES|QL] Fix some validation misconfiguration (#177783)](#177783) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Marco Liberati","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-03-07T12:57:27Z","message":"[ES|QL] Fix some validation misconfiguration (#177783)\n\n## Summary\r\n\r\nRelated issue #177699\r\n\r\nFix variables logic for expressions at `stats by ...` level.\r\nFix validation logic for agg functions within `eval` or `where` scope.\r\nFix validation and autocomplete logic for nested quoted expressions\r\n * i.e. \r\n ```\r\nfrom index | eval round(numberField) + 1 | eval `round(numberField) + 1`\r\n+ 1 | eval ```round(numberField) + 1`` + 1` + 1 | eval\r\n```````round(numberField) + 1```` + 1`` + 1` + 1 | eval\r\n```````````````round(numberField) + 1```````` + 1```` + 1`` + 1` + 1 |\r\nkeep ```````````````````````````````round(numberField) +\r\n1```````````````` + 1```````` + 1```` + 1`` + 1`\r\n ```\r\n* updated `count_distinct` agg definition to have the `precision` second\r\noptional param.\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Stratoula Kalafateli <[email protected]>","sha":"cad276fcbd9fe5b13c18f08fd01bcf0c802b7ec9","branchLabelMapping":{"^v8.14.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Visualizations","release_note:skip","backport:prev-minor","Feature:ES|QL","v8.13.0","v8.14.0"],"number":177783,"url":"https://github.com/elastic/kibana/pull/177783","mergeCommit":{"message":"[ES|QL] Fix some validation misconfiguration (#177783)\n\n## Summary\r\n\r\nRelated issue #177699\r\n\r\nFix variables logic for expressions at `stats by ...` level.\r\nFix validation logic for agg functions within `eval` or `where` scope.\r\nFix validation and autocomplete logic for nested quoted expressions\r\n * i.e. \r\n ```\r\nfrom index | eval round(numberField) + 1 | eval `round(numberField) + 1`\r\n+ 1 | eval ```round(numberField) + 1`` + 1` + 1 | eval\r\n```````round(numberField) + 1```` + 1`` + 1` + 1 | eval\r\n```````````````round(numberField) + 1```````` + 1```` + 1`` + 1` + 1 |\r\nkeep ```````````````````````````````round(numberField) +\r\n1```````````````` + 1```````` + 1```` + 1`` + 1`\r\n ```\r\n* updated `count_distinct` agg definition to have the `precision` second\r\noptional param.\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Stratoula Kalafateli <[email protected]>","sha":"cad276fcbd9fe5b13c18f08fd01bcf0c802b7ec9"}},"sourceBranch":"main","suggestedTargetBranches":["8.13"],"targetPullRequestStates":[{"branch":"8.13","label":"v8.13.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.14.0","labelRegex":"^v8.14.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/177783","number":177783,"mergeCommit":{"message":"[ES|QL] Fix some validation misconfiguration (#177783)\n\n## Summary\r\n\r\nRelated issue #177699\r\n\r\nFix variables logic for expressions at `stats by ...` level.\r\nFix validation logic for agg functions within `eval` or `where` scope.\r\nFix validation and autocomplete logic for nested quoted expressions\r\n * i.e. \r\n ```\r\nfrom index | eval round(numberField) + 1 | eval `round(numberField) + 1`\r\n+ 1 | eval ```round(numberField) + 1`` + 1` + 1 | eval\r\n```````round(numberField) + 1```` + 1`` + 1` + 1 | eval\r\n```````````````round(numberField) + 1```````` + 1```` + 1`` + 1` + 1 |\r\nkeep ```````````````````````````````round(numberField) +\r\n1```````````````` + 1```````` + 1```` + 1`` + 1`\r\n ```\r\n* updated `count_distinct` agg definition to have the `precision` second\r\noptional param.\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Stratoula Kalafateli <[email protected]>","sha":"cad276fcbd9fe5b13c18f08fd01bcf0c802b7ec9"}}]}] BACKPORT--> Co-authored-by: Marco Liberati <[email protected]> Co-authored-by: Kibana Machine <[email protected]>
stratoula
added
Team:ESQL
|
Pinging @elastic/kibana-esql (Team:ESQL) |
1 task
drewdaemon
added a commit
that referenced
this issue
Apr 2, 2024
…79707) ## Summary Part of #177699 `AND` and `OR` were not accepting `NULL` as an argument when they should have. The most basic test case is `ROW var = TRUE or NULL`. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Stratoula Kalafateli <[email protected]>
|
Added problem case: |
|
Nice catch! |
This is a legacy problem. ES has been accepting them casting any non-boolean values as |
|
I think the latter addition |
|
I went checking the ES annotations for Max but it's not mention any Also the documentation seems to not mention Given the work on auto sync the signatures from ES, it is probably worth notifying this thing to them as well, to avoid regressions. |
|
There are no date tests for the |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
May 9, 2024
## Summary This is a follow-on from elastic/elasticsearch#107859. Two main changes to our client-side validation code. 1. comparison operators like `==` and `>=` now support implicit casting from strings for `version`, `ip`, and `boolean` types 2. `in` and `not in` operators support implicit casting from strings for `version`, `ip`, `boolean`, and `date` constants. To address this quickly, I have disabled type-checking for array values (e.g. `in ( version_field, "1.2.3", "2.3.1" )`) because our parameter typing system does not currently support arrays of mixed types which it will need to in order to re-enable checking while allowing string literals. I have added this to elastic#177699 ### A note on testing These implicit casting changes may not be on the latest Elasticsearch snapshot. Instead, check out the `8.14` branch in a local Elasticsearch repo and run `yarn es source --source-path='path/to/elasticsearch'` See [these tests](https://github.com/elastic/kibana/pull/182989/files#diff-89c4af0faedcf80d51cfb19ae397a5898c7293055b19284a94cb3f6a7cd4d071R1727-R1763) for a set of good use cases to try. `to_<type>` functions can be used if the sample data doesn't contain a field of the type you want to test. ### A note on string to date casting In elastic#182856 we started accepting string literals for any function arguments that are dates. By the same logic, `"2022" - 1 day` would work, so our validator doesn't complain about it. However, it currently fails at the Elasticsearch level. In a discussion with Fang, we determined that this is a valid use case, so I have created elastic/elasticsearch#108432 and determined not to tighten the client-side validation since this seems more like a casting "hole" that will soon be filled in on the ES side (though not in 8.14). ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or (cherry picked from commit c3e1027)
kibanamachine
referenced
this issue
May 9, 2024
# Backport This will backport the following commits from `main` to `8.14`: - [[ES|QL] implicit casting changes (#182989)](#182989) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Drew Tate","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-05-09T05:51:18Z","message":"[ES|QL] implicit casting changes (#182989)\n\n## Summary\r\n\r\nThis is a follow-on from\r\nhttps://github.com/elastic/elasticsearch/pull/107859.\r\n\r\nTwo main changes to our client-side validation code.\r\n1. comparison operators like `==` and `>=` now support implicit casting\r\nfrom strings for `version`, `ip`, and `boolean` types\r\n\r\n2. `in` and `not in` operators support implicit casting from strings for\r\n`version`, `ip`, `boolean`, and `date` constants. To address this\r\nquickly, I have disabled type-checking for array values (e.g. `in (\r\nversion_field, \"1.2.3\", \"2.3.1\" )`) because our parameter typing system\r\ndoes not currently support arrays of mixed types which it will need to\r\nin order to re-enable checking while allowing string literals. I have\r\nadded this to https://github.com/elastic/kibana/issues/177699\r\n\r\n### A note on testing\r\n\r\nThese implicit casting changes may not be on the latest Elasticsearch\r\nsnapshot. Instead, check out the `8.14` branch in a local Elasticsearch\r\nrepo and run `yarn es source --source-path='path/to/elasticsearch'`\r\n\r\nSee [these\r\ntests](https://github.com/elastic/kibana/pull/182989/files#diff-89c4af0faedcf80d51cfb19ae397a5898c7293055b19284a94cb3f6a7cd4d071R1727-R1763)\r\nfor a set of good use cases to try. `to_<type>` functions can be used if\r\nthe sample data doesn't contain a field of the type you want to test.\r\n\r\n### A note on string to date casting\r\n\r\nIn #182856 we started accepting\r\nstring literals for any function arguments that are dates.\r\n\r\nBy the same logic, `\"2022\" - 1 day` would work, so our validator doesn't\r\ncomplain about it. However, it currently fails at the Elasticsearch\r\nlevel.\r\n\r\nIn a discussion with Fang, we determined that this is a\r\nvalid use case, so I have created\r\nhttps://github.com/elastic/elasticsearch/issues/108432 and determined\r\nnot to tighten the client-side validation since this seems more like a\r\ncasting \"hole\" that will soon be filled in on the ES side (though not in\r\n8.14).\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or","sha":"c3e1027b2d5da9361251e3af3304098717193ded","branchLabelMapping":{"^v8.15.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport:prev-minor","Feature:ES|QL","v8.14.0","Team:ESQL","v8.15.0"],"title":"[ES|QL] implicit casting changes","number":182989,"url":"https://github.com/elastic/kibana/pull/182989","mergeCommit":{"message":"[ES|QL] implicit casting changes (#182989)\n\n## Summary\r\n\r\nThis is a follow-on from\r\nhttps://github.com/elastic/elasticsearch/pull/107859.\r\n\r\nTwo main changes to our client-side validation code.\r\n1. comparison operators like `==` and `>=` now support implicit casting\r\nfrom strings for `version`, `ip`, and `boolean` types\r\n\r\n2. `in` and `not in` operators support implicit casting from strings for\r\n`version`, `ip`, `boolean`, and `date` constants. To address this\r\nquickly, I have disabled type-checking for array values (e.g. `in (\r\nversion_field, \"1.2.3\", \"2.3.1\" )`) because our parameter typing system\r\ndoes not currently support arrays of mixed types which it will need to\r\nin order to re-enable checking while allowing string literals. I have\r\nadded this to https://github.com/elastic/kibana/issues/177699\r\n\r\n### A note on testing\r\n\r\nThese implicit casting changes may not be on the latest Elasticsearch\r\nsnapshot. Instead, check out the `8.14` branch in a local Elasticsearch\r\nrepo and run `yarn es source --source-path='path/to/elasticsearch'`\r\n\r\nSee [these\r\ntests](https://github.com/elastic/kibana/pull/182989/files#diff-89c4af0faedcf80d51cfb19ae397a5898c7293055b19284a94cb3f6a7cd4d071R1727-R1763)\r\nfor a set of good use cases to try. `to_<type>` functions can be used if\r\nthe sample data doesn't contain a field of the type you want to test.\r\n\r\n### A note on string to date casting\r\n\r\nIn #182856 we started accepting\r\nstring literals for any function arguments that are dates.\r\n\r\nBy the same logic, `\"2022\" - 1 day` would work, so our validator doesn't\r\ncomplain about it. However, it currently fails at the Elasticsearch\r\nlevel.\r\n\r\nIn a discussion with Fang, we determined that this is a\r\nvalid use case, so I have created\r\nhttps://github.com/elastic/elasticsearch/issues/108432 and determined\r\nnot to tighten the client-side validation since this seems more like a\r\ncasting \"hole\" that will soon be filled in on the ES side (though not in\r\n8.14).\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or","sha":"c3e1027b2d5da9361251e3af3304098717193ded"}},"sourceBranch":"main","suggestedTargetBranches":["8.14"],"targetPullRequestStates":[{"branch":"8.14","label":"v8.14.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.15.0","branchLabelMappingKey":"^v8.15.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/182989","number":182989,"mergeCommit":{"message":"[ES|QL] implicit casting changes (#182989)\n\n## Summary\r\n\r\nThis is a follow-on from\r\nhttps://github.com/elastic/elasticsearch/pull/107859.\r\n\r\nTwo main changes to our client-side validation code.\r\n1. comparison operators like `==` and `>=` now support implicit casting\r\nfrom strings for `version`, `ip`, and `boolean` types\r\n\r\n2. `in` and `not in` operators support implicit casting from strings for\r\n`version`, `ip`, `boolean`, and `date` constants. To address this\r\nquickly, I have disabled type-checking for array values (e.g. `in (\r\nversion_field, \"1.2.3\", \"2.3.1\" )`) because our parameter typing system\r\ndoes not currently support arrays of mixed types which it will need to\r\nin order to re-enable checking while allowing string literals. I have\r\nadded this to https://github.com/elastic/kibana/issues/177699\r\n\r\n### A note on testing\r\n\r\nThese implicit casting changes may not be on the latest Elasticsearch\r\nsnapshot. Instead, check out the `8.14` branch in a local Elasticsearch\r\nrepo and run `yarn es source --source-path='path/to/elasticsearch'`\r\n\r\nSee [these\r\ntests](https://github.com/elastic/kibana/pull/182989/files#diff-89c4af0faedcf80d51cfb19ae397a5898c7293055b19284a94cb3f6a7cd4d071R1727-R1763)\r\nfor a set of good use cases to try. `to_<type>` functions can be used if\r\nthe sample data doesn't contain a field of the type you want to test.\r\n\r\n### A note on string to date casting\r\n\r\nIn #182856 we started accepting\r\nstring literals for any function arguments that are dates.\r\n\r\nBy the same logic, `\"2022\" - 1 day` would work, so our validator doesn't\r\ncomplain about it. However, it currently fails at the Elasticsearch\r\nlevel.\r\n\r\nIn a discussion with Fang, we determined that this is a\r\nvalid use case, so I have created\r\nhttps://github.com/elastic/elasticsearch/issues/108432 and determined\r\nnot to tighten the client-side validation since this seems more like a\r\ncasting \"hole\" that will soon be filled in on the ES side (though not in\r\n8.14).\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or","sha":"c3e1027b2d5da9361251e3af3304098717193ded"}}]}] BACKPORT--> Co-authored-by: Drew Tate <[email protected]>
|
Index patterns with a negative produce a syntax error, despite working perfectly: (this pattern should return all indices except those that start with a |
|
@sphilipse thanks for the report |
|
Pretty much all functions and operators can accept |
|
We are tracking valid time units as wrong
|
|
We aren't catching nested aggregation functions (e.g. |
2 tasks
stratoula
added
impact:medium
|
I looked through all the open issues here and except maybe from this all the others are mostly enhancements. So I am lowering the impact from high to medium. |
|
Using field names that are also field types cause the validation to complain wrongly 
|
The |
|
Sorry MArco what do you mean? If I use |
|
Yeah, you are right. I guess that is a type conflict that the validation logic won't fix by default. |
|
cc @stratoula |
This was referenced Aug 22, 2024
drewdaemon
added a commit
that referenced
this issue
Aug 23, 2024
## Summary Part of #189662 ## Suggests comma and pipe https://github.com/user-attachments/assets/c09bc6fd-20a6-4f42-a871-c70e68e6d81a ## Doesn't suggest comma when there are no more fields https://github.com/user-attachments/assets/29fce13e-e58b-4d93-bce5-6b1f913b4d92 ## Doesn't work for escaped columns :( https://github.com/user-attachments/assets/3d65f3b9-923d-4c0e-9c50-51dd83115c8b As part of this effort I discovered #191100 and #191105, as well as a problem with column name validation (see #177699) I think we can revisit column escaping and probably resolve all of these issues (issue [here](#191111)). ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
1 task
stratoula
added
enhancement
stratoula
changed the title
|
I cleaned it up and kept here only the enhancements (also updated the description) We are going to gather the bugs here #192255 Validation bugs are things we are marking as errors while they are not in ES side |
drewdaemon
added a commit
that referenced
this issue
Sep 16, 2024
…191312) ## Summary Part of #189662 Also, a follow-on to #187184 with certain source names (e.g. `foo$bar`, `my-policy`) and fields in the `ENRICH` command. During this effort I discovered #191321 and a bug with the validation of field names in the "ENRICH ... WITH" list (documented [here](#177699)), both of which will be addressed separately. ## Sources ### General flow https://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf ### Works with wild-card matches https://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c ### `METADATA` field list https://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a ## ENRICH Autosuggest now helps you along https://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f Also, fixed this bug (follow on to #187184 ) https://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Elastic Machine <[email protected]> Co-authored-by: Stratoula Kalafateli <[email protected]>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Sep 16, 2024
…lastic#191312) ## Summary Part of elastic#189662 Also, a follow-on to elastic#187184 with certain source names (e.g. `foo$bar`, `my-policy`) and fields in the `ENRICH` command. During this effort I discovered elastic#191321 and a bug with the validation of field names in the "ENRICH ... WITH" list (documented [here](elastic#177699)), both of which will be addressed separately. ## Sources ### General flow https://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf ### Works with wild-card matches https://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c ### `METADATA` field list https://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a ## ENRICH Autosuggest now helps you along https://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f Also, fixed this bug (follow on to elastic#187184 ) https://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Elastic Machine <[email protected]> Co-authored-by: Stratoula Kalafateli <[email protected]> (cherry picked from commit e404a39)
kibanamachine
added a commit
that referenced
this issue
Sep 16, 2024
…60;ENRICH` (#191312) (#193013) # Backport This will backport the following commits from `main` to `8.x`: - [[ES|QL] open suggestions automatically in sources lists and `ENRICH` (#191312)](#191312) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Drew Tate","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-16T13:41:38Z","message":"[ES|QL] open suggestions automatically in sources lists and `ENRICH` (#191312)\n\n## Summary\r\n\r\nPart of https://github.com/elastic/kibana/issues/189662\r\n\r\nAlso, a follow-on to https://github.com/elastic/kibana/issues/187184\r\nwith certain source names (e.g. `foo$bar`, `my-policy`) and fields in\r\nthe `ENRICH` command.\r\n\r\nDuring this effort I discovered\r\nhttps://github.com//issues/191321 and a bug with the\r\nvalidation of field names in the \"ENRICH ... WITH\" list (documented\r\n[here](#177699)), both of which\r\nwill be addressed separately.\r\n\r\n## Sources\r\n\r\n### General flow\r\n\r\n\r\nhttps://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf\r\n\r\n### Works with wild-card matches\r\n\r\n\r\nhttps://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c\r\n\r\n### `METADATA` field list\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a\r\n\r\n## ENRICH\r\n\r\nAutosuggest now helps you along\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f\r\n\r\nAlso, fixed this bug (follow on to\r\nhttps://github.com//issues/187184 )\r\n\r\n\r\n\r\nhttps://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>\r\nCo-authored-by: Stratoula Kalafateli <[email protected]>","sha":"e404a3992e220735ae51918c532a1a032e7f7993","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","v9.0.0","backport:prev-minor","Feature:ES|QL","Team:ESQL"],"title":"[ES|QL] open suggestions automatically in sources lists and `ENRICH`","number":191312,"url":"https://github.com/elastic/kibana/pull/191312","mergeCommit":{"message":"[ES|QL] open suggestions automatically in sources lists and `ENRICH` (#191312)\n\n## Summary\r\n\r\nPart of https://github.com/elastic/kibana/issues/189662\r\n\r\nAlso, a follow-on to https://github.com/elastic/kibana/issues/187184\r\nwith certain source names (e.g. `foo$bar`, `my-policy`) and fields in\r\nthe `ENRICH` command.\r\n\r\nDuring this effort I discovered\r\nhttps://github.com//issues/191321 and a bug with the\r\nvalidation of field names in the \"ENRICH ... WITH\" list (documented\r\n[here](#177699)), both of which\r\nwill be addressed separately.\r\n\r\n## Sources\r\n\r\n### General flow\r\n\r\n\r\nhttps://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf\r\n\r\n### Works with wild-card matches\r\n\r\n\r\nhttps://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c\r\n\r\n### `METADATA` field list\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a\r\n\r\n## ENRICH\r\n\r\nAutosuggest now helps you along\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f\r\n\r\nAlso, fixed this bug (follow on to\r\nhttps://github.com//issues/187184 )\r\n\r\n\r\n\r\nhttps://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>\r\nCo-authored-by: Stratoula Kalafateli <[email protected]>","sha":"e404a3992e220735ae51918c532a1a032e7f7993"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191312","number":191312,"mergeCommit":{"message":"[ES|QL] open suggestions automatically in sources lists and `ENRICH` (#191312)\n\n## Summary\r\n\r\nPart of https://github.com/elastic/kibana/issues/189662\r\n\r\nAlso, a follow-on to https://github.com/elastic/kibana/issues/187184\r\nwith certain source names (e.g. `foo$bar`, `my-policy`) and fields in\r\nthe `ENRICH` command.\r\n\r\nDuring this effort I discovered\r\nhttps://github.com//issues/191321 and a bug with the\r\nvalidation of field names in the \"ENRICH ... WITH\" list (documented\r\n[here](#177699)), both of which\r\nwill be addressed separately.\r\n\r\n## Sources\r\n\r\n### General flow\r\n\r\n\r\nhttps://github.com/user-attachments/assets/4b103621-0e66-4c36-807f-4932f0cb8faf\r\n\r\n### Works with wild-card matches\r\n\r\n\r\nhttps://github.com/user-attachments/assets/6b47fffc-e922-4e2d-b6aa-3d9a2fc2236c\r\n\r\n### `METADATA` field list\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d3bdf4dc-1d0c-4d56-81d7-af6bc4e25a4a\r\n\r\n## ENRICH\r\n\r\nAutosuggest now helps you along\r\n\r\n\r\nhttps://github.com/user-attachments/assets/d627484c-e729-4dc7-9e7b-795395a31d4f\r\n\r\nAlso, fixed this bug (follow on to\r\nhttps://github.com//issues/187184 )\r\n\r\n\r\n\r\nhttps://github.com/user-attachments/assets/aa62a0c3-6db5-434a-829a-59f14c5c4c85\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>\r\nCo-authored-by: Stratoula Kalafateli <[email protected]>","sha":"e404a3992e220735ae51918c532a1a032e7f7993"}}]}] BACKPORT--> Co-authored-by: Drew Tate <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Kibana version:
Here we are gathering enhancements of the client side validation which means things that ES is marking as errors but we allow them. The impact is low because the users will get the errors when run the query on the server side.
from kibana_sample_data_logs | where 23—whereshouldn't accept non-boolean expressionsWe could validate if the user is trying to use a field which is not available in a pipe such as
Here b is not valid as stats is not using it
[ES|QL] better inline casting validation with ES metadata #186409
... | STATS AVG(false OR false)should be an error.AVGdoes not support boolean. The validator works correctly with a literal:| STATS AVG(false), just not a function (OR) return type.We could consider validating named variables like
?t_startand?t_endThe text was updated successfully, but these errors were encountered: