Download, save and manipulate Binance data #128
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AlexTheLion123
changed the title
miohtama
requested changes
Nov 17, 2023
| logger = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def get_binance_candlestick_data( |
There was a problem hiding this comment.
Rename this to fetch_ so it is aligned with other function names in this package.
- get: read from memory
- fetch: read over HTTP
| return df | ||
|
|
||
|
|
||
| def get_binance_data_parquet( |
| @@ -34,7 +34,7 @@ def test_generate_candle_data(synthetic_candles): | |||
| assert universe.get_pair_count() == 1 | |||
| assert universe.get_candle_count() == 4 | |||
|
|
|||
There was a problem hiding this comment.
Can you add an integration test.
Test cases needed
- Read fresh data
- Read cached data
- Purge cache
- Fetch prices
- Fetch lending rates
- Use a specific folder under
/tmpfolder for tests.
| #pandas_time_delta = new_bucket.to_pandas_timedelta() | ||
| # https://stackoverflow.com/questions/21140630/resampling-trade-data-into-ohlcv-with-pandas | ||
| candles = df.resample(new_timedelta).mean(numeric_only=True) | ||
| candles = df.resample(new_timedelta).mean() |
There was a problem hiding this comment.
I am not sure if this a correct way to resample, and it might be that open/high/etc. gets incorrectly resampled. I am not sure if this code path was ever correctly tested.
We are using a different method here:
Can you confirm the resamping works by adding a simple test case that checks whether the given data was correctly resampled?
| [], | ||
| ) | ||
|
|
||
| for i in range(0, len(timestamps) - 1): |
There was a problem hiding this comment.
For user friendliness we can add a tqdm progress bar that gives a good time estimator in Jupyter Notebooks if data is being downloaded.
An example here:
miohtama
reviewed
Nov 17, 2023
miohtama
requested changes
Nov 22, 2023
| def test_read_fresh_candle_data(candle_downloader: BinanceDownloader): | ||
| """Test reading fresh candle data. Will be downloaded from Binance API.""" | ||
|
|
||
| if os.environ.get("GITHUB_ACTIONS", None) == "true": |
There was a problem hiding this comment.
Comment why this is done like this - there is no context
miohtama
pushed a commit
to tradingstrategy-ai/trade-executor
that referenced
this pull request
Dec 6, 2023
### Goal Primarily, allow the use of easily downloading and using Binance data in a strategy. This allows for backtesting much longer time periods; around 3 years in total for shorting, even longer for long only - Easily download candlestick and lending data from Binance API (no API keys needed) - convert lending data to supply data - Use Binance data in long and shorting strategies See related prerequisite PR: tradingstrategy-ai/trading-strategy#128 ### Summary - Adds notebook demonstrating how to use `create_binance_universe` - Bump tradingstrategy to 0.22.3 - make loading functions in `alternative_market_data.py` allow for multipair - Add utils file for `load_binance_dataset` and `create_binance_universe` #### Unrelated - remove incorrect warning in `position_manager.py` - Add `hover_text` bool argument for `visualise_single_pair` so user can hide hover_text if needs be. This is because the hover_text can sometimes be very large and cover much of the chart.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Goal
Primarily, allow the use of easily downloading and using Binance data in a strategy. This allows for backtesting much longer time periods; around 3 years in total for shorting, even longer for long only
get_candles_by_pairmethodresample_candles()methodresample_seriesmethodRelated to tradingstrategy-ai/trade-executor#664