Do openKiwi have confident score?

[baoloc97]

I use openNMT for translate but its dont have confident score when translate so they suggest me use openkiwi. But i dont see confident score in openKiwi too ? Confident score it mean how many percent that my model think that sentence is predict right

[captainvera]

Hello @baoloc97,

Sorry I took a while to respond.

Openkiwi does not produce a confidence score in that sense. When you are talking about a confidence score from a MT model, it can be thought of as the uncertainty of the model on inference time. It can be obtained with methods such as monte carlo dropout and similar.

On the other hand, what OpenKiwi does it Quality Estimation, it provides models that specialise in looking at parallel corpora and producing "quality scores" for this data. Either at word-level (with Ok/BAD tags) or at sentence level (by predicting HTER), this functions as a proxy of MT quality when you don't have a reference, with results from wmt20 metrics shared task showing a high correlation between QE and metrics.