question about the performance

[imsotable]

Hello! I only used the simulated heatmap for experiments. The teacher model is resnet-50, and the student model is resnet-18. However, after 210 epochs, the accuracy of resnet-18 is only 0.2. After reading the log file, I found that the main reason is that the reg_loss is seriously oscillating. Can you give me some advice?

[Shelro]

I guess you use res-18+GAP+deeppose as the student, am i right?

If so, GAP will cause the loss of spatial information in the feature map, making it difficult for the model to learn beneficial spatial information by simulating the heat map, thus causing oscillations.