Ambiguous compound words and hyphenation

[dr-m]

I think that Voikko is a great project. Some 30 years ago I was thinking of developing something similar, but in the end I specialized in other type of software.

https://oikofix.com/analysis is a great service. Is anything similar also available via the command line, say, on Debian GNU/Linux?

I was curious how it would interpret this sentence with 3 ambiguous compound words, playing on the ambiguity between "ruis-kui-tu" and "ruis-ku-i-tu":

Ruiskuoppia levittänyt ruiskuituhippi sai ruiskusta.

For the 3-part compound word, the alternative "ruisku-itu-hippi" is not being recognized.

How are compound words handled? Is there some built-in database?

Also, I wonder if there is any hyphenation interface with (say) LibreOffice, to avoid hyphenations that would mislead the reader (say, avoiding both "ruisku-itu" and "ruiskui-tu" to play it safe).

[hatapitk]

https://oikofix.com/analysis is a great service. Is anything similar also available via the command line, say, on Debian GNU/Linux?

Yes there is. voikkospell -m does the same, one word at a time. If you have paragraphs of text to process you can use voikkogc --tokenize to extract words and feed them to voikkospell -m.

Ruiskuoppia levittänyt ruiskuituhippi sai ruiskusta.

For the 3-part compound word, the alternative "ruisku-itu-hippi" is not being recognized.

How are compound words handled? Is there some built-in database?

As you may already guess handling compounds in Voikko is a bit complicated. In fact we would likely not even have Voikko if 17 years ago it would have been possible to use any open source tool that existed at that time to build a reasonably working Finnish morphology for spell checkers. Handling inflection, while also complicated, was not the problem. It was compounds.

So Voikko has compounding rules that work well for spell checking. That involves a lot of compromises. One of them is that we have largely blocked the use of short and rare words in automatic compounding. "itu" is one such words and thus we need to add real world compounds containing "itu" as separate entries in the vocabulary. Sadly we are missing "ituhippi" which I will add once our vocabulary management application https://joukahainen.puimula.org/ is restored (currently undergoing technology migration from mod_python to Flask).

Also, I wonder if there is any hyphenation interface with (say) LibreOffice, to avoid hyphenations that would mislead the reader (say, avoiding both "ruisku-itu" and "ruiskui-tu" to play it safe).

Yes, libreoffice-voikko does that. But with you example it will hyphenate "ruis-kui-tu" which I guess is correct as the ambiguity here is mostly theoretical.

[dr-m]

Thank you for the explanation. By my limited understanding, the current architecture is "good enough", and developing a "perfect" one may not be worth the effort. It could indeed be better to add the exceptional compound words to the vocabulary.

[hatapitk]

As for example where both ambigous hyphenations are avoided I looked up this from our test suite:

"pöytähienosto" can be interpreted either as "pöytä-hienosto" or "pöytä-hien-osto" so we only hyphenate it "pöy-tä-hienos-to".

[dr-m]

That as well as "hienostoalueen autonostaja" are correctly being recognized as ambiguous.