[BUG]: Code coverage calculation discrepancy related to code formatting and partial coverage reporting

[Rd4dev]

Describe the bug

Oppia Android Code Coverage tool is as of now limited to Line coverage analysis. Line coverage, as calculated by JaCoCo and Bazel, is a measure of how many lines of code are executed during tests. In general, if a line of code is executed at least once, it is considered covered. However, line coverage does not account for the complexity within a single line of code or how different parts of the code are executed.

When code is formatted into multiple lines, it often more accurately reflects the different logical parts of the code, making it easier to see which branches and statements are covered. On the other hand, when code is formatted into a single line, the coverage report may not differentiate between the different logical branches within that line. This can lead to discrepancies in the coverage data, particularly with partial coverage.

• This issue is a tracking item of the Code Coverage Enforcement feature - [Feature Request]: Introduce support for code coverage & enforcement (GSoC'24 4.1) #5343.

Steps To Reproduce

Taking the specific case from ComputeChangedFiles.kt,

Formatted as multiple logically separated lines:

When the code is formatted with logic separated into distinct lines as below:

companion object {
  private val EXTRACT_BUCKET_REGEX = "^([^/]+)".toRegex()

  /** Returns the [FileBucket] that corresponds to the specific [changedFiles]. */
  fun retrieveCorrespondingFileBucket(filePath: String): FileBucket {
    return EXTRACT_BUCKET_REGEX.find(filePath)
      ?.groupValues
      ?.get(1)
      ?.let { bucket ->
        values().find { it.cacheBucketName == bucket }
          ?: error("Invalid bucket name: $bucket")
      } ?: error("Invalid file path: $filePath")
  }
}

And generating the coverage report using the command:

bazel run //scripts:run_coverage -- $(pwd) scripts/src/java/org/oppia/android/scripts/ci/ComputeChangedFiles.kt

The generated coverage report marks the line coverage as:

Formatted in single line:

But when the same is formatted to fit in a single line as below:

companion object {
  private val EXTRACT_BUCKET_REGEX = "^([^/]+)".toRegex()

  /** Returns the [FileBucket] that corresponds to the specific [changedFiles]. */
  fun retrieveCorrespondingFileBucket(filePath: String): FileBucket { return EXTRACT_BUCKET_REGEX.find(filePath) ?.groupValues ?.get(1) ?.let { bucket -> values().find { it.cacheBucketName == bucket } ?: error("Invalid bucket name: $bucket") } ?: error("Invalid file path: $filePath") }
}

The generated coverage report marks the line coverage as:

The coverage analysis treats it as one continuous line. As the code is executed and has hit at least one condition, it is reported as fully covered (marked as green highlighted line) for line coverage because the line is considered as a single unit.

This can result in misleading coverage reports, where partial coverage or specific logical branches may not be accurately reflected.

Expected Behavior

Accurate Line Coverage Reporting: Code coverage tools should reflect the actual execution coverage based on the logical segments of code. Lines of code that are partially executed should not be reported as fully covered.

Clear Reporting: Coverage reports should clearly differentiate between lines that are fully covered and those that are not, and should provide detailed insights into which specific conditions and branches within those lines have been executed.

Additional Context

To address this issue, we may focus on two main areas:

Formatting: Establish best practices for formatting code to ensure that coverage reports accurately reflect the code's logical structure.

Branch Coverage: Implement and utilize detailed branch coverage information to more precisely identify which parts of the code are covered, beyond just line coverage.

Attention to these areas will help in obtaining more accurate and meaningful coverage reports.

Bazel version

6.5.0

What device/emulator are you using?

No response

Which Android version is your device/emulator running?

No response

Which version of the Oppia Android app are you using?

No response