Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Sim] Integrate subgraph predictions data into simulations #1234

Open
4 tasks
KatunaNorbert opened this issue Jun 18, 2024 · 3 comments
Open
4 tasks

[Sim] Integrate subgraph predictions data into simulations #1234

KatunaNorbert opened this issue Jun 18, 2024 · 3 comments
Labels

Comments

@KatunaNorbert
Copy link
Member

KatunaNorbert commented Jun 18, 2024

Target:
Make simulation profit results as close to reality as possible

Issue:
On high level, simulation profit using backtracking is not accurate and hard to trust.

  • Currently when calculating the predictoor bot profit using simulations, the profit depends on a few variables which currently are mocked, which means they don't follow the other predictoors changes(stake, accuracy).
  • Calibrating this variables is difficult because we don't have enough visibility in the current predictoors data.

Solution:
Use the new subgraph data provided through LAKE ETL to follow the real prediction data from the past inside simulations.

With the following changes we can achieve a more accurate, up to date predictoor profit metric:

  • Update simulation flow to fetch the required prediction data
  • Get and prepare the data to be used by the simulation flow
  • For each simulation step( epoch ) calculate the predictoor profit using the calculated predicted value agains other predictoors predictions
  • Update the UI with useful data that could help to calibrate the predictoor config towards making more money ( ex: display avg stake per epoch, display average accuray among predictoors etc)
@KatunaNorbert KatunaNorbert added the Type: Enhancement New feature or request label Jun 18, 2024
@trentmc trentmc changed the title Integrate subgraph predictions data into simulations [Sim] Integrate subgraph predictions data into simulations Jun 18, 2024
@trentmc
Copy link
Member

trentmc commented Jun 20, 2024

I recommend to close this issue, marked as duplicate. Why: it's mostly superseded by #1257

And it's that issue that gets to the heart of what we want

@KatunaNorbert
Copy link
Member Author

I still think this issue has value, can we keep it in backlog and maybe come back to it latter?

@trentmc
Copy link
Member

trentmc commented Jun 20, 2024

I still think this issue has value, can we keep it in backlog and maybe come back to it latter?

OK. Then please (a) update it where #1257 is a pre-requisite (b) mark as low priority

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants