Divide functionality of pv-api into 3 repos: HTTP interface, Backtesting Engine, Portfolio library

[jdfergason]

PV-API has grown such that it is time to split its functionality into 3 different repositories:

Flow-chart

graph LR
    S[Strategy] -->|Dynamically Loaded| B[Backtest Engine]
    B --> D[(Database)]
    A[API] --> B
    A <--> D
    F[Frontend] --> A
    U((User)) <--> F
    Sync --> A
    Brokerage <--> Sync

Loading

Components

HTTP Interface

The purpose of the HTTP interface is to implement the HTTP API used by the Frontend and Sync clients. It interfaces with the PVDB database and the Backtesting Engine to accomplish its functions.

Backtesting Engine

The Backtesting Engine executes strategies and saves results in the PVDB database Most of the existing functionality of pv-api will reside here. Notably, individual Strategies should use the golang plugin API for dynamically loading strategies.

Portfolio Library

Library with data structures necessary for maintaining a portfolio of financial instruments and contains useful functions for calculating performance and metrics related to those holdings.

Desired Outcomes

• Improve reliability of PV-API by isolating backtests that could potentially fail. Under this new model each backtest would be independent of every other backtest meaning the failure of one could not negatively impact others
• Improve our ability to scale-out. Backtests can scale horizontally - just start more as they are needed. AWS Lambda could be a good option for running backtests.
• A divided repository will hopefully be easier to test and see where test coverage needs to be improved
• By removing strategies from the main application we set ourselves up for better scalability and gain the ability to release additional strategies independently of the backtesting engine / HTTP api
• Make it easier to only grab the functionality desired (this may be a bit of a fantasy, it's not hard to get pv-api going today. One could argue that having to stand up more components actually makes getting going more difficult)

Potential Risks

• We want to ensure that pv-api is simple to install; multiple components usually make things more difficult to install even if they are more flexible
• While it gets easier to see relevant code coverage and related metrics with divided repositories the necessity for cross-repository integration tests becomes more important. There is currently no solution in place for cross-repo integration tests