avoid changing the original candles array

[movy]

Occasionally, especially when using warm_up_candles with distributed backtests (e.g via Ray.tune), backtests were crashing with assignment destination is read-only error. Copying array before mutating it help avoid this error.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[saleh-mir]

Sorry about the delay for my response. So I looked into this. Now I'm assuming you're trying to use Rey or similar frameworks with Jesse's research module's backtest function. And if so, you're the one passing the candles and you can pass in copies of candles then to make sure mutations don't happen. Am I missing something?

[movy]

Correct, the error is thrown when I run backtests from within Ray. Ray itself uses some kind of storage to distribute candles between many workers, and candles are passed around by reference, so I assumed Ray was not happy when original array is changed. I don't think there is a way to create a separate candles array copy for each ray worker, so the original candles array must be immutable:

backtest(config=ray.get(config_ref),  # starting balance, fees etc.
                            routes=ray.get(routes_refs[params["tf"]]),  # exchange, strategy, symbol, timeframe or 5m (if extra_timeframes)
                            extra_routes=ray.get(extra_routes_refs[anchor_timeframe(params["tf"])]),
                            candles=ray.get(candles_ref),  # candles within backtest date range
                            generate_csv=False,
                            generate_json=True,
                            hyperparameters=params)

The error was traced back to this call https://github.com/jesse-ai/jesse/pull/419/files#diff-678d1837e93b9e24db7260c519fdf374b46673e1e5855b936e1e8527a9cb0bc2L399, and copying the array instead of modifying it fixed the problem. I've patched it in my copy of Jesse long time ago (when this PR was created), and it's been working flawlessly since then.

[saleh-mir]

This example code you provided, is it being used inside a loop? Or is it a one-time call and Rey is creating some kind of loop of its own? I'm saying if it's inside a loop you can pass a copy of the candles in it so that it would be copied inside the initial call to do the backtest() function.

[movy]

It's not a loop per se, rather a testing function is distributed amongst many workers and each worker gets access to candles via immutable central storage.

def backtest_rungs(params):
    for candles_ref in candles_refs[overlap]:
        result = backtest(config=ray.get(config_ref),  
                        routes=ray.get(routes_refs[params["tf"]]), 
                         extra_routes=ray.get(extra_routes_refs[anchor_timeframe(params["tf"])]), 
                        candles=ray.get(candles_ref),  # candles within backtest date range
                        generate_csv=False,
                        generate_json=True,
                        hyperparameters=params)

tune.Tuner(tune.with_resources(backtest_rungs, tune.PlacementGroupFactory([{"CPU": 0.5}])), 
                        param_space={**prefix_hyperparams[strategy_prefix]}, 
                        tune_config=tune_config,
                        run_config=run_config
                        ).fit()

I cannot remember whether I tried copying candles received by reference, and running backtest() function with that copy, it would work I guess, but right now I don't have time to test this approach. I think this PR can be closed for now, and if someone else faces this problem, they can refer to this discussion for the solution.

[saleh-mir]

I agree.