Some questions about the test model and test results

[yys-Polaris]

I followed the steps to set up the environment on the 3090 and put the dataset into the corresponding folder. The code works. But I found that the model saved by training was in the same folder as SAF-Net.ipynb, not in model_saver. For the testing step, the model used is the one in the model_saver folder. Therefore, I tested the model parameters obtained by training and obtained the following results

This result is not consistent with the result in SAF-Net.ipynb. Could you share the parameters related to the training？

[chenhyle]

Hi author, the training loss and validation loss calculated in the SAF-Net.ipynb file is the accumulation of the average loss of multiple batch, I calculated the average loss for the model, as well as, I also normalized y during the test set, after that I calculated the average loss, and I found that the model's test loss is very different from the training loss, is the model overfitted? Below is the code I have changed, I hope the author can respond, thanks a lot!!!!

[xuguangning1218]

Hi author, the training loss and validation loss calculated in the SAF-Net.ipynb file is the accumulation of the average loss of multiple batch, I calculated the average loss for the model, as well as, I also normalized y during the test set, after that I calculated the average loss, and I found that the model's test loss is very different from the training loss, is the model overfitted? Below is the code I have changed, I hope the author can respond, thanks a lot!!!!

Thank you for your interest in this work. The code is quite old, and my opinion may not be correct. I have reviewed my original code and the code provided by you. My code has normalized the training y, but the test y is read directly without normalization. During testing, it is necessary to invert the normalization of pred. Your current test results are intuitively a normalization issue. You can consider whether this can solve the problem from this perspective.

[chenhyle]

作者您好，SAF-Net.ipynb 文件中计算的训练损失和验证损失是多个批次的平均损失的累加，我计算了模型的平均损失，并且我还在测试集期间对 y 进行了归一化，之后我计算了平均损失，发现模型的测试损失和训练损失相差很大，模型是不是过拟合了？下面是我修改的代码，希望作者能回复，非常感谢！！！！

感谢您对这项工作的兴趣。代码已经很老了，我的观点可能不正确。我已经检查了我的原始代码和您提供的代码。我的代码已经对训练y进行了归一化，但是测试y是直接读取的，没有归一化。在测试过程中，有必要反转 pred 的标准化。您当前的测试结果直观上是一个标准化问题。你可以从这个角度考虑这样是否可以解决问题。

Thanks to the author for the reply! Since I was trying to visually compare the training loss with the test loss, I used the normalized y to calculate the loss function during the test set. Below are the test results I ran with the model saved in your code, the calculated y loss is around 0.06, but the average loss should be around 0.01 according to the source code training results, here is the code I changed and the results, I hope the author can respond, thank you very much!

[xuguangning1218]

作者您好，SAF-Net.ipynb 文件中计算的训练损失和验证损失是多个批次的平均损失的累加，我计算了模型的平均损失，并且我还在测试集期间对 y 进行了归一化，之后我计算了平均损失，发现模型的测试损失和训练损失相差很大，模型是不是过拟合了？下面是我修改的代码，希望作者能回复，非常感谢！！！！

感谢您对这项工作的兴趣。代码已经很老了，我的观点可能不正确。我已经检查了我的原始代码和您提供的代码。我的代码已经对训练y进行了归一化，但是测试y是直接读取的，没有归一化。在测试过程中，有必要反转 pred 的标准化。您当前的测试结果直观上是一个标准化问题。你可以从这个角度考虑这样是否可以解决问题。

Thanks to the author for the reply! Since I was trying to visually compare the training loss with the test loss, I used the normalized y to calculate the loss function during the test set. Below are the test results I ran with the model saved in your code, the calculated y loss is around 0.06, but the average loss should be around 0.01 according to the source code training results, here is the code I changed and the results, I hope the author can respond, thank you very much!

Oh, I see what you mean. But it's difficult to judge whether it's overfitting based on these two errors alone. I suggest you retrain using the open-source code on GitHub and plot the error curves for the training set, validation set, and test set during the training process. This will clearly show whether overfitting has occurred. In my paper, I didn't do this because I didn't notice this problem at that time. Thank you very much!