Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Executing bazel command in 2 different ways for tests with shard conf…
…iguration and tests that don't have shard configuration The reason for this is -with tests that have shard configuration, to use the same shards we need to use the bazel test --collect code coverage flag to run them with the same parallelization but the downside is they also provide the coverage results in separate shards which could easily turn into a misrepresentation if the coverage.dat file has missing information since they only represent data of one shard. This was noticed with RunCoverageTest where on alternate runs in ci it represented percentages of 50% and 70% with the exact same test data. To overcome this we need to combine these coverages into one. While bazel does provide --combined report flag, it doesnot provide the ability to provide a desired path. Which is again a next downside. So far the provided combine report flag saves the report to the _coverage/_coverage_report.dat file but but but the catch is, it is the same dir for all the coverage (test) runs. Which means that while running asynchronously if a report is overriden there is a change of discrepency in the data at times. This might be a rare case but it is definitely not fool proof atleast to the extend I can think of. -And also this method of collect coverage flag might not even be required for non sharded tests, since this might introduce extra effort and even overriden reports too as now they too will use the same common report folder. So now there are 2 ways of executing coverage 1. with bazel coverage for non sharded tests 2. with bazel test --collect_code_coverage for sharded tests.
- Loading branch information