About julia files in examples - overfitting

[jskDr]

I found that 'test-wasserstein.jl' does not consider overfitting yet although it includes AccuracyLayer. If my understanding is right, I am suggesting to include a new extended example. In neural networks, early stop is one of the essential item to train the network.

The following is my executing results of 'test-wasserstein.jl' with increasing of max_iter to 20000:
Accuracy (avg over 1000) = 90.9000%
...
Accuracy (avg over 1000) = 94.3000%
...
Accuracy (avg over 1000) = 94.0000%
...
Accuracy (avg over 1000) = 93.7000%

[pluskid]

@jskDr Thanks! We have a DecayOnValidation learning rate policy that allows one to half the learning rate when the performance on validation set drops.

[jskDr]

@pluskid I mean not improving the performance of a test set but stopping of training after the performance of a test set starts to drop earlier than reaching the stopping criteria, so called early stopping. If early stopping is not yet implemented in add_coffee_break(), I might want to implement it.
http://www.mathworks.com/help/nnet/ug/improve-neural-network-generalization-and-avoid-overfitting.html