Implement Kincaid-Fleisch test on comments

[mgajda]

Kincaid-Fleisch readability test on comments is easy to compute metric that allows to estimate how hard it is to read the comment.

[GildedHonour]

Since the comments are independent from each other, each peace of a comment accross all the code files should be extracted and tested manually for readability. Correct?

[mgajda]

Actually homplexity has a different method:

• first we gather all comments (or metrics) per unit of code (function or module)
• then compute statistics for entire unit

[mgajda]

I corrected this on Gitter: we should first assign comments to individual functions or modules, and then extract readability statistics per code unit (module, function, or other top level declaration ).

[GildedHonour]

Okay.

I'm getting familiar with the code. It's unclear still how it works -- the algorithms for different metircs, parsing...

[workshub]

This issue is now published on WorksHub. If you would like to work on this issue you can
start work on the WorksHub Issue Details page.

[workshub]

@willnwhite started working on this issue via WorksHub.

[ghost]

How accurate must the Kincaid-Fleisch test be? The test depends on the number of syllables, and I think the only way to accurately get the number of syllables in a word is by using a list of all words and how many syllables each one has... no algorithm is more accurate than that. However, this would add to program size/network requests compared to using an algorithm.

[mgajda]

I bet that syllabification can be super-simple, with only checking consecutive stretches of vowels and consonants. This is statistics, small errors should not change much.

So you can have:

isVowel = (`elem` "aeoiu")
countSyllables = length . filter (not . null) . break isVowel

Same for sentences:

isPunct = (`elem` ",:.!?")
countSentences = length . filter (not . null) . break isPunct

[mgajda]

Please note that the code will be probably too short to justify Kincaid-Fleisch package on its own.
Unless you plan to maintain it for the other project.

[ghost]

The test is very sensitive to the syllable count. I ran your algorithm on a comment from this project and the score was -66, versus the real score of 30. Could that still be useful, e.g. for relative scores, or not?

[mgajda]

Did you split words before running syllable count?

[ghost]

Yes.

[mgajda]

Tell me your branch, and I will try to help you find the bug.
https://gitter.im/