You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I understand that we for save of computing power measure accuracy and loss via moving model, but it may be the case that current implementation that just returns last loss and last acc in the last batch a bit incorrect.
I suggest replace it:
``` sum(losses)/len(losses), sum(accs)/len(accs)``
It's still approximate computation of loss and accuracy because we evaluate acc and loss in different points.
The text was updated successfully, but these errors were encountered:
To be honest, this design choice feels subjective. In the current implementation, we return a noisy estimate of the final training loss / final training accuracy of the episode. In your case, you propose to return the average training loss / average training accuracy. Since at the beginning of each episode the model has random performance, such measure would be polluted with bad scores received at the beginning of the training — especially if num_train_steps_per_episode is small (as in our case). For me, evaluating a model's performance based on its final performance feels more natural, but you can use whatever measure you are comfortable with
cs326-few-shot-classification/trainers/trainer.py
Line 46 in 7db7842
I understand that we for save of computing power measure accuracy and loss via moving model, but it may be the case that current implementation that just returns last loss and last acc in the last batch a bit incorrect.
I suggest replace it:
``` sum(losses)/len(losses), sum(accs)/len(accs)``
It's still approximate computation of loss and accuracy because we evaluate acc and loss in different points.
The text was updated successfully, but these errors were encountered: