EvoResearcher Tool: prediction_with_research_report

[namesty]

Overview

This PR aims to add a new tool that uses: https://github.com/polywrap/evo.researcher in order to generate a research report which is then injected as additional_information to the already existing prediction prompt from the prediction_sentence_embedding tool.

This has shown to consistently impact the scores of predictions for questions such as: "Will Twitter implement a new misinformation policy before the 2024 elections?", "Will the goal of not surpassing 1.5 degrees Celsius set by the Paris Agreement be reached by 2030?", among others taken from Omen's subgraph. It makes prediction scores more nuanced and information_utility scores higher.

Evo researcher's repo contains the implementation of the research function.

Costs & Time

Average times in seconds (s):

Using gpt-3.5-turbo-1106: 58
Using gpt-4-1106-preview: 120

Average costs in USD ($):

Using gpt-3.5-turbo-1106: 0,0099
Using gpt-4-1106-preview: 0,1012

Notes

• Sometimes behavior is not consistent when using gpt-3.5-turbo-1106
• Prediction prompt and generation are untouched - taken directly from prediction_sentence_embedding

[Adamantios]

Please update the docstring.

[namesty]

Should it be any different? I see it's the same in all tools

[Adamantios]

I see 😞
Opened #157.

Ideally, it should include a description of the corresponding module. Something like:

Suggested change

	"""This module contains the bet amount per threshold strategy."""
	"""This module contains the evo researcher tool."""

[0xArdi]

We currently don't utilize this service. It seems like a good addition, we will take a look.

[0xArdi]

The mech specifies ^3.10 for the allowed python versions. But this package (or one of its deps) requires sqlite3>3.35.0, which doesnt come with all ^3.10 versions.

[namesty]

You're right. This is due to ChromaDB: https://docs.trychroma.com/troubleshooting#sqlite

I went with their second proposed approach (the first being just upgrading to a later Python 3.10 version): https://gist.github.com/defulmere/8b9695e415a44271061cc8e272f3c300

But do let me know if this should be handled differently

[0xArdi]

@namesty we are aiming to include this next week on the mech. There are two things required to do that:

• Deps introduced here should be satisfied in the mech agent.
• Dynamic pricing of requests is introduced in the mech.

[0xArdi]

Dynamic pricing was added on #166

[namesty]

@0xArdi I've updated the deps, added token counting for dynamic pricing as per the other example tools, and re-pinned everything to IPFS

[dOrgJelli]

I believe in this PR there should have been updates to the mech's api_keys, as it adds a new dependency on tavily:

mech/packages/polywrap/customs/prediction_with_research_report/prediction_with_research_report.py

Line 519 in 5cb0383

tavily_api_key = kwargs["api_keys"]["tavily"]

Not sure what needs to be updated though, would appreciate some guidance @0xArdi @Adamantios @kongzii
https://github.com/search?q=repo%3Avalory-xyz%2Fmech%20api_keys&type=code