Autoencoder evaluation seems to be incorrect

[hitvoice]

Line 242-244 function evaluate_autoencoder, train.py:

all_accuracies += \
    torch.mean(max_indices.eq(masked_target).float()).data[0]
bcnt += 1

Since masks in batches are different, in each batch there'll be a different number of indices being compared. The average of average is equal to the average of all if and only if all the chunks are of the same size, which is not the case here. The all_accuracies calculated here is inaccurate.

Besides, in line 233 in function evaluate_autoencoder, train.py:

output = autoencoder(source, lengths, noise=True)

During the evaluation, the autoencoder seems to be "cheating" here. The correct word vector of the last word is given, and this is not the case when the autoencoder is doing the real inference. The actual accuracy will be a lot lower. The sentence-level accuracy is way lower than word-level accuracies as well. When the decoder fails to generate "<eos>", the whole sentence is wrong, but in word-level accuracies, the sentence is (n-1)/n correct.