model overfitting

[ayyappa428]

Issue 1:

How to avoid overfitting?
The model is getting overfitted.

Issue 2:

I trained the model with 5000 epochs with 100 MB dataset. I am not getting the exact output.
For some sentences it is displaying correct output and for some sentences it is not displaying correct output.

For 5000 epochs

---

Hi how are you (English)
comment comment (France)

---

I went to office
Je suis allés à bureau

---

I am going to my home
Je vais chez moi

I trained again with 8000 epochs i am not getting correct output.

---

hai how are you
Avez-vous comment quelle

---

I went to office
Je suis allé à bureau

---

I am going to my home
Je vais chez moi chez moi

Please suggest me for how many epochs i will get the correct output?

suggest any other alternatives to avoid overfitting

[Nemzy]

Hi @ayyappa428 ,

Well just a small correction, you didn't train the model with 5k epochs. 1 epoch is when your model sees a whole dataset once, so NUM_STEPS * BATCH_SIZE / NUM_SAMPLES = EPOCHS_TRAINED.

Steps I would try to increase accuracy of the model are:

• Train the model with more epochs, 10 at least (takes a lot of time!).
• Try different hyperparameters:
  • Deeper encoder and decoder networks (more layers).
  • Wider encoder and decoder networks (more neurons in layers).
  • Smaller learning rates.
  • etc.
• Reorder words in input sentences in reverse ("How are you _ _ _ -> _ _ _ you are How").

For overfitting just use standard techniques like regularization, dropout, bigger dataset etc.