Skip to content

Commit

Permalink
Executing bazel command in 2 different ways for tests with shard conf…
Browse files Browse the repository at this point in the history
…iguration and tests that don't have shard configuration

The reason for this is -with tests that have shard configuration, to use the same shards we need to use the bazel test --collect code coverage flag to run them with the same parallelization but the downside is they also provide the coverage results in separate shards which could easily turn into a misrepresentation if the coverage.dat file has missing information since they only represent data of one shard. This was noticed with RunCoverageTest where on alternate runs in ci it represented percentages of 50% and 70% with the exact same test data. To overcome this we need to combine these coverages into one. While bazel does provide --combined report flag, it doesnot provide the ability to provide a desired path. Which is again a next downside. So far the provided combine report flag saves the report to the _coverage/_coverage_report.dat file but but but the catch is, it is the same dir for all the coverage (test) runs. Which means that while running asynchronously if a report is overriden there is a change of discrepency in the data at times. This might be a rare case but it is definitely not fool proof atleast to the extend I can think of. -And also this method of collect coverage flag might not even be required for non sharded tests, since this might introduce extra effort and even overriden reports too as now they too will use the same common report folder. So now there are 2 ways of executing coverage 1. with bazel coverage for non sharded tests 2. with bazel test --collect_code_coverage for sharded tests.
  • Loading branch information
Rd4dev committed Jul 21, 2024
1 parent 8946b06 commit 3d78a86
Showing 1 changed file with 39 additions and 3 deletions.
42 changes: 39 additions & 3 deletions scripts/src/java/org/oppia/android/scripts/common/BazelClient.kt
Original file line number Diff line number Diff line change
Expand Up @@ -144,22 +144,58 @@ class BazelClient(private val rootDirectory: File, private val commandExecutor:
fun runCoverageForTestTarget(bazelTestTarget: String): List<String>? {
val instrumentation = bazelTestTarget.split(":")[0]
val computeInstrumentation = instrumentation.split("/").let { "//${it[2]}/..." }
// val coverageCommandOutputLines = executeBazelCommand(
executeBazelCommand(
/*val coverageCommandOutputLines = executeBazelCommand(
// executeBazelCommand(
"test",
"--collect_code_coverage",
"--combined_report=lcov",
bazelTestTarget,
"--instrumentation_filter=$computeInstrumentation"
)*/

val targetParts = bazelTestTarget.split(":")
val targetPath = "${targetParts[0]}:*"
val targetName = targetParts[1]

val buildRule = executeBazelCommand(
"query",
"attr(name, $targetName, $targetPath)",
"--output=build",
)
println("BUILD: $buildRule")
println("BUILD has shard count: ${buildRule.any { "shard_count" in it }}")

println("Printing for the bazel test target: $bazelTestTarget")
// println(File(rootDirectory,"/bazel-out/_coverage/_coverage_report.dat").exists())
// println(File(rootDirectory,"/bazel-out/_coverage/_coverage_report.dat").readText())
/*return parseCoverageDataFilePath(coverageCommandOutputLines)?.let { path ->
File(path).readLines()
}*/

return File(rootDirectory,"/bazel-out/_coverage/_coverage_report.dat").readLines()
val hasShardCount = buildRule.any { "shard_count" in it }
if (hasShardCount) {
println("In has Shard count")
// val coverageCommandOutputLines = executeBazelCommand(
executeBazelCommand(
"test",
"--collect_code_coverage",
"--combined_report=lcov",
bazelTestTarget,
"--instrumentation_filter=$computeInstrumentation"
)
return File(rootDirectory, "/bazel-out/_coverage/_coverage_report.dat").readLines()
} else {
println("In does not have Shard count")
val coverageCommandOutputLines = executeBazelCommand(
// executeBazelCommand(
"coverage",
bazelTestTarget,
"--instrumentation_filter=$computeInstrumentation"
)
return parseCoverageDataFilePath(coverageCommandOutputLines)?.let { path ->
File(path).readLines()
}
}
}

private fun parseCoverageDataFilePath(coverageCommandOutputLines: List<String>): String? {
Expand Down

0 comments on commit 3d78a86

Please sign in to comment.