Regression with CNNs ?

[PonteIneptique]

Hey :)
Weird thing: as I actually had equivalent scores between CNN and RNN embeddings, my experiments with Optuna for now have yielded consistently lower result (by a big margin) on CNNs (I assume this change is in cause bca9516#diff-4853226f237bf1b0a17a302fbfa3e997 )

I cannot say if it's a regression before I actually retrain the same config as before but this might be something to look at in the next weeks.

[PonteIneptique]

Current scores:

| Id | Param | Type (1=RNN, 0=CNN) | Status |
| -- | -- | -- | -- | -- |
| 64 | cemb_type | 1.0 | | RUNNING |
| 18 | cemb_type | 0.0 | | RUNNING |
| 5 | cemb_type | 1.0 | 0.0 | COMPLETE |
| 9 | cemb_type | 1.0 | 0.0 | COMPLETE |
| 72 | cemb_type | 1.0 | 0.0 | COMPLETE |
| 7 | cemb_type | 0.0 | 0.7219 | COMPLETE |
| 9 | cemb_type | 0.0 | 0.842 | PRUNED |
| 50 | cemb_type | 0.0 | 0.8423 | PRUNED |
| 4 | cemb_type | 0.0 | 0.8585 | PRUNED |
| 13 | cemb_type | 0.0 | 0.8606 | PRUNED |
| 14 | cemb_type | 0.0 | 0.8635 | PRUNED |
| 15 | cemb_type | 0.0 | 0.8645 | PRUNED |
| 16 | cemb_type | 0.0 | 0.8668 | PRUNED |
| 10 | cemb_type | 0.0 | 0.8677 | PRUNED |
| 6 | cemb_type | 0.0 | 0.8698 | PRUNED |
| 6 | cemb_type | 0.0 | 0.881 | PRUNED |
| 38 | cemb_type | 0.0 | 0.8815 | PRUNED |
| 26 | cemb_type | 1.0 | 0.8883 | PRUNED |
| 8 | cemb_type | 0.0 | 0.8899 | PRUNED |
| 35 | cemb_type | 0.0 | 0.901 | PRUNED |
| 40 | cemb_type | 0.0 | 0.9029 | PRUNED |
| 81 | cemb_type | 0.0 | 0.9032 | PRUNED |
| 0 | cemb_type | 0.0 | 0.9127 | COMPLETE |
| 0 | cemb_type | 0.0 | 0.9193 | COMPLETE |
| 1 | cemb_type | 0.0 | 0.9196 | COMPLETE |
| 36 | cemb_type | 1.0 | 0.9343 | PRUNED |
| 1 | cemb_type | 0.0 | 0.9372 | COMPLETE |
| 34 | cemb_type | 1.0 | 0.947 | PRUNED |
| 3 | cemb_type | 0.0 | 0.9496 | COMPLETE |
| 12 | cemb_type | 1.0 | 0.9555 | PRUNED |
| 39 | cemb_type | 1.0 | 0.959 | PRUNED |
| 79 | cemb_type | 1.0 | 0.959 | PRUNED |
| 47 | cemb_type | 1.0 | 0.9606 | PRUNED |
| 30 | cemb_type | 1.0 | 0.9609 | PRUNED |
| 20 | cemb_type | 1.0 | 0.9619 | PRUNED |
| 29 | cemb_type | 1.0 | 0.9636 | PRUNED |
| 12 | cemb_type | 1.0 | 0.9642 | PRUNED |
| 22 | cemb_type | 1.0 | 0.9644 | PRUNED |
| 49 | cemb_type | 1.0 | 0.9646 | PRUNED |
| 63 | cemb_type | 1.0 | 0.9647 | PRUNED |
| 59 | cemb_type | 1.0 | 0.9652 | PRUNED |
| 66 | cemb_type | 1.0 | 0.9652 | PRUNED |
| 97 | cemb_type | 1.0 | 0.9653 | PRUNED |
| 37 | cemb_type | 1.0 | 0.9656 | PRUNED |
| 94 | cemb_type | 1.0 | 0.9656 | PRUNED |
| 48 | cemb_type | 1.0 | 0.966 | PRUNED |
| 75 | cemb_type | 1.0 | 0.9661 | PRUNED |
| 84 | cemb_type | 1.0 | 0.9661 | PRUNED |
| 33 | cemb_type | 1.0 | 0.9664 | PRUNED |
| 46 | cemb_type | 1.0 | 0.9665 | PRUNED |
| 82 | cemb_type | 1.0 | 0.967 | PRUNED |
| 78 | cemb_type | 1.0 | 0.9671 | PRUNED |
| 83 | cemb_type | 1.0 | 0.9671 | PRUNED |
| 98 | cemb_type | 1.0 | 0.9674 | PRUNED |
| 85 | cemb_type | 1.0 | 0.9676 | PRUNED |
| 44 | cemb_type | 1.0 | 0.9678 | PRUNED |
| 74 | cemb_type | 1.0 | 0.9678 | PRUNED |
| 62 | cemb_type | 1.0 | 0.968 | PRUNED |
| 68 | cemb_type | 1.0 | 0.9683 | PRUNED |
| 71 | cemb_type | 1.0 | 0.9683 | PRUNED |
| 92 | cemb_type | 1.0 | 0.9683 | PRUNED |
| 27 | cemb_type | 1.0 | 0.9684 | PRUNED |
| 99 | cemb_type | 1.0 | 0.9696 | PRUNED |
| 56 | cemb_type | 1.0 | 0.9706 | PRUNED |
| 3 | cemb_type | 1.0 | 0.9719 | COMPLETE |
| 16 | cemb_type | 1.0 | 0.973 | COMPLETE |
| 80 | cemb_type | 1.0 | 0.9733 | PRUNED |
| 2 | cemb_type | 1.0 | 0.9739 | COMPLETE |
| 13 | cemb_type | 1.0 | 0.9742 | COMPLETE |
| 19 | cemb_type | 1.0 | 0.9743 | PRUNED |
| 73 | cemb_type | 1.0 | 0.9744 | COMPLETE |
| 11 | cemb_type | 1.0 | 0.9745 | COMPLETE |
| 10 | cemb_type | 1.0 | 0.9747 | COMPLETE |
| 7 | cemb_type | 1.0 | 0.9748 | COMPLETE |
| 89 | cemb_type | 1.0 | 0.9751 | PRUNED |
| 96 | cemb_type | 1.0 | 0.9751 | PRUNED |
| 70 | cemb_type | 1.0 | 0.9752 | COMPLETE |
| 2 | cemb_type | 1.0 | 0.9752 | COMPLETE |
| 5 | cemb_type | 1.0 | 0.9753 | COMPLETE |
| 17 | cemb_type | 1.0 | 0.9754 | COMPLETE |
| 95 | cemb_type | 1.0 | 0.9756 | PRUNED |
| 11 | cemb_type | 1.0 | 0.9757 | COMPLETE |
| 54 | cemb_type | 1.0 | 0.9757 | COMPLETE |
| 8 | cemb_type | 1.0 | 0.9757 | COMPLETE |
| 45 | cemb_type | 1.0 | 0.9758 | COMPLETE |
| 25 | cemb_type | 1.0 | 0.976 | COMPLETE |
| 51 | cemb_type | 1.0 | 0.976 | COMPLETE |
| 17 | cemb_type | 1.0 | 0.976 | COMPLETE |
| 41 | cemb_type | 1.0 | 0.9761 | COMPLETE |
| 87 | cemb_type | 1.0 | 0.9769 | COMPLETE |
| 60 | cemb_type | 1.0 | 0.977 | COMPLETE |
| 18 | cemb_type | 1.0 | 0.9772 | COMPLETE |
| 21 | cemb_type | 1.0 | 0.9772 | COMPLETE |
| 28 | cemb_type | 1.0 | 0.9773 | COMPLETE |
| 42 | cemb_type | 1.0 | 0.9773 | COMPLETE |
| 77 | cemb_type | 1.0 | 0.9774 | COMPLETE |
| 86 | cemb_type | 1.0 | 0.9775 | COMPLETE |
| 32 | cemb_type | 1.0 | 0.9776 | COMPLETE |
| 91 | cemb_type | 1.0 | 0.9776 | COMPLETE |
| 61 | cemb_type | 1.0 | 0.9778 | COMPLETE |
| 23 | cemb_type | 1.0 | 0.9779 | COMPLETE |
| 24 | cemb_type | 1.0 | 0.978 | COMPLETE |
| 14 | cemb_type | 1.0 | 0.9781 | COMPLETE |
| 31 | cemb_type | 1.0 | 0.9781 | COMPLETE |
| 4 | cemb_type | 1.0 | 0.9782 | COMPLETE |
| 76 | cemb_type | 1.0 | 0.9783 | COMPLETE |
| 43 | cemb_type | 1.0 | 0.9784 | COMPLETE |
| 58 | cemb_type | 1.0 | 0.9784 | COMPLETE |
| 53 | cemb_type | 1.0 | 0.9785 | COMPLETE |
| 67 | cemb_type | 1.0 | 0.9785 | COMPLETE |
| 88 | cemb_type | 1.0 | 0.9785 | COMPLETE |
| 52 | cemb_type | 1.0 | 0.9787 | COMPLETE |
| 69 | cemb_type | 1.0 | 0.9787 | COMPLETE |
| 90 | cemb_type | 1.0 | 0.9787 | COMPLETE |
| 15 | cemb_type | 1.0 | 0.9788 | COMPLETE |
| 55 | cemb_type | 1.0 | 0.9788 | COMPLETE |
| 57 | cemb_type | 1.0 | 0.979 | COMPLETE |
| 93 | cemb_type | 1.0 | 0.9794 | COMPLETE |
| 65 | cemb_type | 1.0 | 0.9796 | COMPLETE |

[PonteIneptique]

It seems it has been a while that I did not train on CNN. I leave that open for the future, when comparing with the models we have for Old French.

[emanjavacas]

Hey, I believe bca9516 fixed some previous hacks with the CNNs, it's an old commit I had lying around. I am surprised I hadn't pushed it before. I haven't used cnn embeddings very extensively, so I am cannot rule out a bug for sure. It would need some experimenting to make clear what's going on. Same architecture trained with previous and current implementation and comparison.