Results can not be replicated

[yanxurui]

In the file ddpg_experiments.ipynb you have shown 2 plots of the results in testing_history, one for imitation, one for ddpg, which is very attractive. For example, ddpg with window 3, cnn, and batch norm achieves 3 fold return.

However, after training it myself using the same code, same model, same config and same data, I fail to replicate the results. The return is always below the market.

[yanxurui]

Your pretrained model,

After training it by myself

[vermouth1992]

It is one of the major issues known in many fields using deep learning models, especially RL. I suggest you try different random seeds and train for longer period.
Here is a reference How Many Random Seeds? Statistical Power
Analysis in Deep Reinforcement Learning
Experiments

[yanxurui]

Thanks for you reply.
I don't think seed and training period are to blame.
Since I am using the exactly same config including seed and training data as you did, the results should not be too different.

[NarendarK29]

Hi,
I like to know if we change our data, I am downloading for yf and I am getting exactly same as market portfolio. I have transferred the data into required shape, of around 4000 data points (Open,Close,low, high, Adj Close) and have tried config changes, window length, Epochs and batch size. @yanxurui and @vermouth1992, I will be very thankful if you could inform if the data need to be changed before application of DDPG (both LSTM and CNN is giving exactly equity weighted porfolio)
P.S : with 28 stocks with 15 year of data and model is taking approximately 5 minutes to build the graphs.
Thank you,
Best regards.

[yanxurui]

Hi, @NarendarK29
I'd like to help but I failed to get your point.
Could you please state it clearly what your goal is and what your problem is? Thanks.

[NarendarK29]

Hi, @yanxurui
Thank you for replying. Unfortunately, not all the folder were forked and it was not getting trained model parameters.
It is an excellent work and I like to contribute toward it. I like to get your point of view that if I am trying to incorporate other features, as the P/E and rolling variance to make the model prone to value-based indicators and volatility of stock, where should I begin. I will deeply appreciate if you can provide me any guidance over where I can incorporate these changes.
I am currently working on these models with intention to incorporate it into my master thesis project. I will deeply appreciate your help and please consider the request.
Regards.

[yanxurui]

I think you should make sure this model (the model in this repo) works and then apply your improvements.
If it does not work, you will not get satisfying results no matter what fancy features you add.

[NarendarK29]

Hello @yanxurui
Below are the results European stock:
Training

Testing