Fs 104/Fix-Webagent #44
Conversation
| @@ -86,7 +86,7 @@ async def web_general_search_core(search_query, llm, model) -> str: | |||
| continue # Skip if the summarization is not valid | |||
| response = { | |||
| "content": summary, | |||
| "ignore_validation": "false" | |||
| "ignore_validation": "true" # This is to ignore the validation of the answer again by the supervisor | |||
There was a problem hiding this comment.
why are we disabling validation for the web agent?
There was a problem hiding this comment.
@evpearce - Because we have already validated it on the line 84.
|
|
||
| Guidelines: | ||
| - If the user has asked to check online, then each question in the questions array should also specify that. |
There was a problem hiding this comment.
Why do we want this?
If this is a beneficial change, then I would expect more tests to be added to intent_config.yaml to prove that this is working as expected.
There was a problem hiding this comment.
I saw that Intent was not coming right as it for ESG related tasks, it was going to Datastore agent when there were more than 1 question.
Sorry, I had no clue about intent_config.yaml, will have a look at it and add more tests related to it there.
There was a problem hiding this comment.
Do you have an example of when it was wrong and how, I'm still not quite getting the problem
| @@ -2,7 +2,7 @@ You are an expert validator. You can help with validating the answers to the tas | |||
|
|
|||
| Your entire purpose is to return a "true" or "false" value to indicate if the answer has fulfilled the task, along with a reasoning to explain your decision. | |||
|
|
|||
| You will be passed a task and an answer. You need to determine if the answer is correct or not. | |||
| You will be passed a task and an answer. You need to determine if the answer is correct or not, ensuring that the task's specific requirements are addressed. | |||
There was a problem hiding this comment.
Why are there changes to the validator template if we are disabling it for the webAgent? Again promptfoo tests should be added for these changes.
There was a problem hiding this comment.
I disabled the second validation, I will add promptfoo tests.
There was a problem hiding this comment.
I see, thanks for explaining
| @@ -1,4 +1,4 @@ | |||
| description: "Intent" | |||
| description: 'Intent' | |||
There was a problem hiding this comment.
the other yaml files use ", I assume your IDE linting flagged this, could you update your IDE linting config instead of this change?
| @@ -13,7 +13,7 @@ | |||
| engine = PromptEngine() | |||
|
|
|||
|
|
|||
| async def search_urls(search_query, num_results=10) -> str: | |||
| async def search_urls(search_query, num_results=30) -> str: | |||
There was a problem hiding this comment.
Does this change slow down the web agent significantly? We are looking to improve the web agent search in general with https://scottlogic.atlassian.net/browse/FS-46
Description
This is to fix the webagent to return proper results
Changelog