Fix #1325: Create basic predictoor dashboard (#1326)

* created a working dash app structure

* displayed feeds data inside a table

* get and display predictoors data as table

* added predictoors accuracy chart

* added costs table

* added profit chart

* update layout so plots take up more space

* added some typings and descriptions

* handle db connection errors and display them

* added tests for utils

* add search inputs for tables

* added tests for plots

* added readmes and documentation

* increase stake bar width

---------

Co-authored-by: Mustafa Tuncay <[email protected]>
Co-authored-by: Călina Cenan <[email protected]>

READMEs/predictoor-dashboard.md

@@ -0,0 +1,54 @@
+# Predictoor Dashboard
+
+**The Predictoor dashboard provides insights into blockchain data through interactive plots and tables.**
+
+## **How it works**
+
+Live data from a specified supported blockchain is fetched and stored in a local database. By running a Dash dashboard, this data is processed and displayed in various formats to provide valuable information and control.
+
+## **How to setup and run**
+
+To set up and run the Predictoor dashboard, follow these steps:
+
+1. **Fetch chain data into the lake**
+
+The first step is to fetch the data from the blockchain using the lake's ETL, by this command:
+
+```console
+pdr lake etl update ./my_ppss.yaml sapphire-mainnet
+```
+
+For more information on how the lake works and how to configure it, refer to [`this readme`](./lake-and-etl.md).
+
+2. **Run the dash app from command line**
+
+After fetching the chain data locally into the lake, the next step is to read, process, and display the data by running the dashboard with the following command:
+
+```console
+pdr predictoor_dashboard ./my_ppss.yaml sapphire-mainnet
+```
+
+This command will open a browser window where you can select predictors and feeds to visualize their performance.
+
+## **How to use**
+
+By default, the plots will be empty because no predictors or feeds are selected.
+
+After **selecting one or more predictors and feeds combinations** where predictions has been submitted, and the payout method called, the plots will be populated with the relevant data.
+
+You can then observe how your predictor bots or others have been performing on different feeds and even compare them.
+
+**IMPORTANT: Only predictions data where the payout method has been called are visible.**
+
+The displayed charts are:
+
+1. **Accuracy** - how predictoor accuracy has evolved durring the selected period of time.
+2. **Profit** - what is the profit in OCEAN over the period. Here you can clearly see if you are profitable or not.
+3. **Costs** - how much predictoor has staked for each epoch.
+
+## **How to contribute**
+
+We are constantly looking for ways to improve the Predictoor dashboard to help predictors become more profitable and are open to suggestions and ideas.
+
+If you are a predictor and want to help us improve this tool, [join our Discord channel](https://discord.gg/Tvqd7Z648H) and drop a message, or open a GitHub issue.
+

READMEs/predictoor.md

@@ -14,6 +14,7 @@ This README shows how to earn $ by running a predictoor bot on mainnet.
 1. **[Run bot on testnet](#3-run-predictoor-bot-on-sapphire-testnet)**
 1. **[Run bot on mainnet](#4-run-predictoor-bot-on-sapphire-mainnet)**
 1. **[Claim payout](#5-claim-payout)**
+1. **[Run dashboard to monitor and analyze performance](#6-run-dashboard)**
 
 Once you're done the main flow, you can go beyond, with any of:
 
@@ -217,6 +218,14 @@ When running predictoors on mainnet, you have the potential to earn $.
 
 Congrats! You've gone through all the essential steps to earn $ by running a predictoor bot on mainnet.
 
+## 6. Run Dashboard
+
+After running your predictor bot, you may **monitor and analyze its performance**.
+
+To assist with this, pdr_backend provides a dashboard that utilizes **live blockchain data** to visualize key performance metrics such as **accuracy, profit, and costs over a specific period**.
+
+Please refer to this **[setup guide](predictoor-dashboard.md)** for detailed instructions on how to set up, configure, and **run the dashboard** to visualize real-time data from the blockchain.
+
 The next sections describe how to go beyond, by optimizing the model and more.
 
 # Optimize model

pdr_backend/analytics/predictoor_dashboard/dash_components/callbacks.py

@@ -0,0 +1,180 @@
+from dash import Input, Output, State
+import dash
+from pdr_backend.analytics.predictoor_dashboard.dash_components.util import (
+    get_feeds_data_from_db,
+    get_predictoors_data_from_db,
+    get_payouts_from_db,
+    filter_objects_by_field,
+)
+from pdr_backend.analytics.predictoor_dashboard.dash_components.view_elements import (
+    get_graph,
+)
+from pdr_backend.analytics.predictoor_dashboard.dash_components.plots import (
+    get_figures,
+)
+
+
+# pylint: disable=too-many-statements
+def get_callbacks(app):
+    @app.callback(
+        Output("feeds-data", "data"),
+        Output("predictoors-data", "data"),
+        Output("error-message", "children"),
+        Input("data-folder", "data"),
+    )
+    def get_input_data_from_db(files_dir):
+        try:
+            feeds_data = get_feeds_data_from_db(files_dir)
+            predictoors_data = get_predictoors_data_from_db(files_dir)
+            return feeds_data, predictoors_data, None
+        except Exception as e:
+            return None, None, dash.html.H3(str(e))
+
+    @app.callback(
+        Output("payouts-data", "data"),
+        [
+            Input("feeds_table", "selected_rows"),
+            Input("predictoors_table", "selected_rows"),
+            Input("search-input-Feeds", "value"),
+            Input("search-input-Predictoors", "value"),
+        ],
+        State("feeds-data", "data"),
+        State("predictoors-data", "data"),
+        State("data-folder", "data"),
+    )
+    def get_display_data_from_db(
+        feeds_table_selected_rows,
+        predictoors_table_selected_rows,
+        search_value_feeds,
+        search_value_predictoors,
+        feeds_data,
+        predictoors_data,
+        lake_dir,
+    ):
+        feeds_addrs = []
+        predictoors_addrs = []
+        if (
+            len(feeds_table_selected_rows) == 0
+            or len(predictoors_table_selected_rows) == 0
+        ):
+            return dash.no_update
+
+        current_feeds_table_data = update_feeds_table_on_search(
+            search_value_feeds, feeds_data
+        )
+        for i in feeds_table_selected_rows:
+            feeds_addrs.append(current_feeds_table_data[i]["contract"])
+
+        current_predictoors_table_data = update_predictoors_table_on_search(
+            search_value_predictoors, predictoors_data
+        )
+        for i in predictoors_table_selected_rows:
+            predictoors_addrs.append(current_predictoors_table_data[i]["user"])
+
+        payouts = get_payouts_from_db(feeds_addrs, predictoors_addrs, lake_dir)
+
+        return payouts
+
+    @app.callback(
+        Output("feeds_table", "columns"),
+        Input("feeds-data", "data"),
+    )
+    def create_feeds_table(feeds_data):
+        if not feeds_data:
+            return dash.no_update
+
+        for feed in feeds_data:
+            del feed["contract"]
+
+        columns = [{"name": col, "id": col} for col in feeds_data[0].keys()]
+        return columns
+
+    @app.callback(
+        Output("predictoors_table", "columns"),
+        Input("predictoors-data", "data"),
+    )
+    def create_predictoors_table(predictoors_data):
+        if not predictoors_data:
+            return dash.no_update
+        columns = [{"name": col, "id": col} for col in predictoors_data[0].keys()]
+        return columns
+
+    @app.callback(
+        Output("accuracy_chart", "children"),
+        Output("profit_chart", "children"),
+        Output("stake_chart", "children"),
+        Input("payouts-data", "data"),
+        Input("feeds_table", "selected_rows"),
+        Input("predictoors_table", "selected_rows"),
+        Input("search-input-Feeds", "value"),
+        Input("search-input-Predictoors", "value"),
+        State("feeds-data", "data"),
+        State("predictoors-data", "data"),
+    )
+    def create_charts(
+        payouts_data,
+        feeds_table_selected_rows,
+        predictoors_table_selected_rows,
+        search_value_feeds,
+        search_value_predictoors,
+        feeds_data,
+        predictoors_data,
+    ):
+        feeds_addrs = []
+        predictoors_addrs = []
+
+        ## calculate selected feeds
+        current_feeds_table_data = update_feeds_table_on_search(
+            search_value_feeds, feeds_data
+        )
+        for i in feeds_table_selected_rows:
+            feeds_addrs.append(
+                {
+                    "contract": current_feeds_table_data[i]["contract"],
+                    "feed_name": f"{current_feeds_table_data[i]['pair']}-{current_feeds_table_data[i]['timeframe']}",  # pylint: disable=line-too-long
+                }
+            )
+
+        ## calculate selected predictoors addrs
+        current_predictoors_table_data = update_predictoors_table_on_search(
+            search_value_predictoors, predictoors_data
+        )
+        for i in predictoors_table_selected_rows:
+            predictoors_addrs.append(current_predictoors_table_data[i]["user"])
+
+        # get figures
+        accuracy_fig, profit_fig, stakes_fig = get_figures(
+            payouts_data, feeds_addrs, predictoors_addrs
+        )
+
+        return get_graph(accuracy_fig), get_graph(profit_fig), get_graph(stakes_fig)
+
+    @app.callback(
+        Output("predictoors_table", "data"),
+        [
+            Input("search-input-Predictoors", "value"),
+            Input("predictoors-data", "data"),
+        ],
+    )
+    def update_predictoors_table_on_search(search_value, predictoors_data):
+        if not search_value:
+            return predictoors_data
+
+        # filter predictoors by user address
+        filtered_data = filter_objects_by_field(predictoors_data, "user", search_value)
+        return filtered_data
+
+    @app.callback(
+        Output("feeds_table", "data"),
+        [
+            Input("search-input-Feeds", "value"),
+            Input("feeds-data", "data"),
+        ],
+    )
+    def update_feeds_table_on_search(search_value, feeds_data):
+        if not search_value:
+            return feeds_data
+
+        # filter feeds by pair address
+        filtered_data = filter_objects_by_field(feeds_data, "pair", search_value)
+        return filtered_data

pdr_backend/analytics/predictoor_dashboard/dash_components/plots.py

@@ -0,0 +1,133 @@
+from itertools import product
+from typing import Union, List, Tuple, Optional
+import plotly.graph_objects as go
+from enforce_typing import enforce_types
+from pdr_backend.util.time_types import UnixTimeS
+
+
+@enforce_types
+def process_payouts(payouts: List[dict], predictor: str, feed: str) -> tuple:
+    """
+    Process payouts data for a given predictor and feed.
+    Args:
+        payouts (list): List of payouts data.
+        predictor (str): Predictor address.
+        feed (str): Feed contract address.
+    Returns:
+        tuple: Tuple of slots, accuracies, profits, and stakes.
+    """
+    slots, accuracies, profits, stakes = [], [], [], []
+    profit = predictions = correct_predictions = 0
+
+    for p in payouts:
+        if not (predictor in p["ID"] and feed in p["ID"]):
+            continue
+
+        predictions += 1
+        profit_change = max(p["payout"], 0) - p["stake"]
+        profit += profit_change
+        correct_predictions += p["payout"] > 0
+
+        slots.append(p["slot"])
+        accuracies.append((correct_predictions / predictions) * 100)
+        profits.append(profit)
+        stakes.append(p["stake"])
+    slot_in_date_format = [
+        UnixTimeS(ts).to_milliseconds().to_dt().strftime("%m-%d %H:%M") for ts in slots
+    ]
+    return slot_in_date_format, accuracies, profits, stakes
+
+
+@enforce_types
+def create_figure(
+    data_traces: List[Union[go.Scatter, go.Bar]],
+    title: str,
+    yaxis_title: str,
+    show_legend: bool = True,
+):
+    """
+    Create a figure with the given data traces.
+    Args:
+        data_traces (list): List of data traces.
+        title (str): Figure title.
+        yaxis_title (str): Y-axis title.
+        show_legend (bool): Show legend. Default is True.
+    Returns:
+        go.Figure: Plotly figure.
+    """
+    fig = go.Figure(data_traces)
+    legend_config = (
+        {
+            "orientation": "h",
+            "yanchor": "bottom",
+            "y": 1.02,
+            "xanchor": "right",
+            "x": 1,
+        }
+        if show_legend
+        else {}
+    )
+
+    fig.update_layout(
+        title=title,
+        yaxis_title=yaxis_title,
+        margin={"l": 20, "r": 0, "t": 50, "b": 0},
+        showlegend=show_legend,
+        xaxis_nticks=4,
+        bargap=0.1,
+        barmode="stack",
+        legend=legend_config,
+    )
+    return fig
+
+
+@enforce_types
+def get_figures(
+    payouts: Optional[List], feeds: List, predictoors: List[str]
+) -> Tuple[go.Figure, go.Figure, go.Figure]:
+    """
+    Get figures for accuracy, profit, and costs.
+    Args:
+        payouts (list): List of payouts data.
+        feeds (list): List of feeds data.
+        predictoors (list): List of predictoors data.
+    Returns:
+        tuple: Tuple of accuracy, profit, and costs figures.
+    """
+    accuracy_scatters, profit_scatters, stakes_scatters = [], [], []
+
+    if payouts:
+        for predictor, feed in product(predictoors, feeds):
+            slots, accuracies, profits, stakes = process_payouts(
+                payouts, predictor, feed["contract"]
+            )
+            if not slots:
+                continue
+
+            short_name = f"{predictor[:5]} - {feed['feed_name']}"
+            accuracy_scatters.append(
+                go.Scatter(x=slots, y=accuracies, mode="lines", name=short_name)
+            )
+            profit_scatters.append(
+                go.Scatter(x=slots, y=profits, mode="lines", name=short_name)
+            )
+            stakes_scatters.append(go.Bar(x=slots, y=stakes, name=short_name, width=5))
+
+    if not accuracy_scatters:
+        accuracy_scatters.append(go.Scatter(x=[], y=[], mode="lines", name="accuracy"))
+    if not profit_scatters:
+        profit_scatters.append(go.Scatter(x=[], y=[], mode="lines", name="profit"))
+    if not stakes_scatters:
+        stakes_scatters.append(go.Bar(x=[], y=[], name="stakes", width=5))
+
+    fig_accuracy = create_figure(
+        accuracy_scatters, "Accuracy", "'%' accuracy over time"
+    )
+    fig_profit = create_figure(
+        profit_scatters, "Profit", "OCEAN profit over time", show_legend=False
+    )
+    fig_costs = create_figure(
+        stakes_scatters, "Costs", "Stake (OCEAN) at a time", show_legend=False
+    )
+
+    return fig_accuracy, fig_profit, fig_costs