Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Automatic Import] Improve error logging when hitting a recursion limit #192916

Closed
ebeahan opened this issue Sep 13, 2024 · 5 comments
Closed

[Automatic Import] Improve error logging when hitting a recursion limit #192916

ebeahan opened this issue Sep 13, 2024 · 5 comments
Assignees
@bhapas
Labels
enhancement New value added to drive a business result Feature:AutomaticImport Team:Security-Scalability Team label for Security Integrations Scalability Team

Comments

@ebeahan
Copy link
Member

ebeahan commented Sep 13, 2024

Summary

The improvements made in #190008 help limit reaching recursion limits in the categorization and related graph outputs. However, certain conditions will still present recursion errors returned by the underlying LLM connector:

Image

The current exception returned isn't specific or actionable. Error handling could be improved to offer more detail or guidance to resolve.
@ebeahan ebeahan added enhancement New value added to drive a business result Feature:AutomaticImport Team:Security-Scalability Team label for Security Integrations Scalability Team labels Sep 13, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-scalability (Team:Security-Scalability)

@bhapas bhapas mentioned this issue Sep 20, 2024
Merged
2 tasks
@bhapas
Copy link
Contributor

bhapas commented Sep 23, 2024

@jamiehynds What would be an ideal expression to the end user if we encounter this issue? We don't have any docs currently that could be used for self service , but that would be something for future.

@jamiehynds
Copy link

@bhapas do we have a sense of typical scenarios that trigger the recursion errors? Is there anything a user can do to prevent against it (like checking for correct JSON in samples, etc)?

@bhapas
Copy link
Contributor

bhapas commented Sep 23, 2024

@bhapas do we have a sense of typical scenarios that trigger the recursion errors? Is there anything a user can do to prevent against it (like checking for correct JSON in samples, etc)?

@jamiehynds One of the major reasons I have seen lately is the correctness of samples. Like wrong formatted logs in the sample file. We have fixed some scenarios that typically caused this issue in 8.15.0 and are fixing the ones that are identified from time to time. But double checking the log samples / shinking the size of samples. might help

bhapas added a commit that referenced this issue Sep 25, 2024
@bhapas
[Automatic Import] Add Error handling framework (#193577)
9fb6f55 
## Release Note

Adds error handling framework that provides error message with more
context to user.

## Summary

Relates - [192916](#192916)

This PR adds an error handling framework.
- Add Error classes for specific error scenarios.
- If the error caught is of the predefined Error type the `message` and
`errorCode` is sent back to UI from server.
- The original error message is used to track telemetry and the
errorCode can be translated into a User visible error.
- If there is any non-predefined error server still throws a
`badRequest` with the error message.

This PR also adds/updates the graph images for different langgraphs

## Screenshots for error messages
<img width="690" alt="image"
src="https://github.com/user-attachments/assets/bb848ce7-e474-4e4e-8d07-59b534c543ea">

<img width="691" alt="image"
src="https://github.com/user-attachments/assets/fbf4cf46-9bbe-4c37-aaaa-0ede1cdcba7c">



### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Sep 25, 2024
@bhapas
[Automatic Import] Add Error handling framework (elastic#193577)
d9145da 
## Release Note

Adds error handling framework that provides error message with more
context to user.

## Summary

Relates - [192916](elastic#192916)

This PR adds an error handling framework.
- Add Error classes for specific error scenarios.
- If the error caught is of the predefined Error type the `message` and
`errorCode` is sent back to UI from server.
- The original error message is used to track telemetry and the
errorCode can be translated into a User visible error.
- If there is any non-predefined error server still throws a
`badRequest` with the error message.

This PR also adds/updates the graph images for different langgraphs

## Screenshots for error messages
<img width="690" alt="image"
src="https://github.com/user-attachments/assets/bb848ce7-e474-4e4e-8d07-59b534c543ea">

<img width="691" alt="image"
src="https://github.com/user-attachments/assets/fbf4cf46-9bbe-4c37-aaaa-0ede1cdcba7c">

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 9fb6f55)
@bhapas
Copy link
Contributor

bhapas commented Sep 26, 2024

#193577 closes this

@bhapas bhapas closed this as completed Sep 26, 2024
kibanamachine referenced this issue Sep 26, 2024
@kibanamachine @bhapas @elasticmachine
[8.x] [Automatic Import] Add Error handling framework (#193577) (#194017
a84d8cb 
)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Automatic Import] Add Error handling framework
(#193577)](#193577)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Bharat
Pasupula","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-25T14:24:01Z","message":"[Automatic
Import] Add Error handling framework (#193577)\n\n## Release
Note\r\n\r\nAdds error handling framework that provides error message
with more\r\ncontext to user.\r\n\r\n## Summary\r\n\r\nRelates -
[192916](https://github.com/elastic/kibana/issues/192916)\r\n\r\nThis PR
adds an error handling framework.\r\n- Add Error classes for specific
error scenarios.\r\n- If the error caught is of the predefined Error
type the `message` and\r\n`errorCode` is sent back to UI from
server.\r\n- The original error message is used to track telemetry and
the\r\nerrorCode can be translated into a User visible error.\r\n- If
there is any non-predefined error server still throws a\r\n`badRequest`
with the error message.\r\n\r\nThis PR also adds/updates the graph
images for different langgraphs\r\n\r\n## Screenshots for error
messages\r\n<img width=\"690\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/bb848ce7-e474-4e4e-8d07-59b534c543ea\">\r\n\r\n<img
width=\"691\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/fbf4cf46-9bbe-4c37-aaaa-0ede1cdcba7c\">\r\n\r\n\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"9fb6f5582220148820fb6b83880539f8df0b0841","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["enhancement","v9.0.0","release_note:feature","backport:prev-minor","Team:Security-Scalability","Feature:AutomaticImport"],"title":"[Automatic
Import] Add Error handling
framework","number":193577,"url":"https://github.com/elastic/kibana/pull/193577","mergeCommit":{"message":"[Automatic
Import] Add Error handling framework (#193577)\n\n## Release
Note\r\n\r\nAdds error handling framework that provides error message
with more\r\ncontext to user.\r\n\r\n## Summary\r\n\r\nRelates -
[192916](https://github.com/elastic/kibana/issues/192916)\r\n\r\nThis PR
adds an error handling framework.\r\n- Add Error classes for specific
error scenarios.\r\n- If the error caught is of the predefined Error
type the `message` and\r\n`errorCode` is sent back to UI from
server.\r\n- The original error message is used to track telemetry and
the\r\nerrorCode can be translated into a User visible error.\r\n- If
there is any non-predefined error server still throws a\r\n`badRequest`
with the error message.\r\n\r\nThis PR also adds/updates the graph
images for different langgraphs\r\n\r\n## Screenshots for error
messages\r\n<img width=\"690\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/bb848ce7-e474-4e4e-8d07-59b534c543ea\">\r\n\r\n<img
width=\"691\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/fbf4cf46-9bbe-4c37-aaaa-0ede1cdcba7c\">\r\n\r\n\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"9fb6f5582220148820fb6b83880539f8df0b0841"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193577","number":193577,"mergeCommit":{"message":"[Automatic
Import] Add Error handling framework (#193577)\n\n## Release
Note\r\n\r\nAdds error handling framework that provides error message
with more\r\ncontext to user.\r\n\r\n## Summary\r\n\r\nRelates -
[192916](https://github.com/elastic/kibana/issues/192916)\r\n\r\nThis PR
adds an error handling framework.\r\n- Add Error classes for specific
error scenarios.\r\n- If the error caught is of the predefined Error
type the `message` and\r\n`errorCode` is sent back to UI from
server.\r\n- The original error message is used to track telemetry and
the\r\nerrorCode can be translated into a User visible error.\r\n- If
there is any non-predefined error server still throws a\r\n`badRequest`
with the error message.\r\n\r\nThis PR also adds/updates the graph
images for different langgraphs\r\n\r\n## Screenshots for error
messages\r\n<img width=\"690\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/bb848ce7-e474-4e4e-8d07-59b534c543ea\">\r\n\r\n<img
width=\"691\"
alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/fbf4cf46-9bbe-4c37-aaaa-0ede1cdcba7c\">\r\n\r\n\r\n\r\n###
Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n\r\n### For
maintainers\r\n\r\n- [ ] This was checked for breaking API changes and
was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"9fb6f5582220148820fb6b83880539f8df0b0841"}}]}]
BACKPORT-->

Co-authored-by: Bharat Pasupula <[email protected]>
Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bhapas bhapas

Labels
enhancement New value added to drive a business result Feature:AutomaticImport Team:Security-Scalability Team label for Security Integrations Scalability Team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ebeahan @elasticmachine @jamiehynds @bhapas