Different scores at each run in Angular 17 project

[rdechiara]

Summary

Almost each time I run Stryker against my Angular v17 project I get different score without changing the code. I have noticed this issue because after getting a low score for a component I added the needed tests and the score went up. When I run the tests again, without any modification, the score dropped by about 20%.
Stryker config

// @ts-check
/** @type {import('@stryker-mutator/api/core').PartialStrykerOptions} */
const config = {
  _comment:
    "This config was generated using 'stryker init'. Please see the guide for more information: https://stryker-mutator.io/docs/stryker-js/guides/angular",
  mutate: [
    "src/**/*.component.ts",
    "src/**/*.service.ts",
    "src/**/*.interceptor.ts",
    "!src/**/*.spec.ts",
    "!src/test.ts",
    "!src/environments/*.ts",
  ],
  testRunner: "karma",
  karma: {
    configFile: "karma.conf.js",
    projectType: "angular-cli",
    config: {
      browsers: ["ChromeHeadless"],
    },
  },
  reporters: ["progress", "clear-text", "html"],
  concurrency: 4,
  concurrency_comment:
    "Recommended to use about half of your available cores when running stryker with angular",
  coverageAnalysis: "perTest",
};
export default config;

Test runner config

Stryker environment

+-- @stryker-mutator/[email protected]
+-- @stryker-mutator/[email protected]

+-- [email protected]

Test runner environment

npx stryker run

Didn't configure karma so it has all defaults.

Your Environment

software	version(s)
node	18.19.0
npm	npm 10.8
Operating System	Windows 11

Add stryker.log

[nicojs]

Could you provide (screenshots of the) HTML report? I'm curious to see where the differences are coming from.

StrykerJS should be deterministic, but some variables can still change the score. For example, computer use during mutation testing might cause some test runner processes to starve, resulting in timeout mutants (false positive).

[arielBarZoomInfo]

Hi nicojs,

I’m using StrykerJS in my NestJS project, but I’m facing an issue where the mutation scores differ with every run, even when the computer is not being actively used.

I tried attaching the HTML reports here, but GitHub blocks that file type. Instead, I’ve sent the reports to the email associated with your GitHub profile.

I’d really love to use Stryker in my projects, but the inconsistent results are making it challenging. Any guidance would be greatly appreciated.

Thanks!