diff --git a/docs/configuration.md b/docs/configuration.md index b1965eec73..5abc56e65d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -208,7 +208,7 @@ Config file: `"mutator": { "plugins": ["classProperties"], "excludedMutations": * `plugins`: allows you to override the default [babel plugins](https://babeljs.io/docs/en/plugins) to use for JavaScript files. By default, Stryker uses [a default list of babel plugins to parse your JS file](https://github.com/stryker-mutator/stryker-js/blob/master/packages/instrumenter/src/parsers/js-parser.ts#L8-L32). It also loads any plugins or presets you might have configured yourself with `.babelrc` or `babel.config.js` files. In the rare situation where the plugins Stryker loads conflict with your own local plugins (for example, when using the decorators and decorators-legacy plugins together), you can override the `plugins` here to `[]`. -* `excludedMutations`: allow you to specify a [list of mutator names](https://github.com/stryker-mutator/stryker-handbook/blob/master/mutator-types.md#supported-mutators) to be excluded (`ignored`) from the test run. +* `excludedMutations`: allow you to specify a [list of mutator names](https://github.com/stryker-mutator/stryker-handbook/blob/master/mutator-types.md#supported-mutators) to be excluded (`ignored`) from the test run. See [Disable mutants](./disable-mutants) for more options of how to disable specific mutants. _Note: prior to Stryker version 4, the mutator also needed a `name` (or be defined as `string`). This is removed in version 4. Stryker now supports mutating of JavaScript and friend files out of the box, without the need for a mutator plugin._ diff --git a/docs/disable-mutants.md b/docs/disable-mutants.md new file mode 100644 index 0000000000..329006e819 --- /dev/null +++ b/docs/disable-mutants.md @@ -0,0 +1,181 @@ +--- +title: Disable mutants +custom_edit_url: https://github.com/stryker-mutator/stryker-js/edit/master/docs/disable-mutants.md +--- + +During mutation testing, you might run into [equivalent mutants](../mutation-testing-elements/equivalent-mutants) or simply mutants that you are not interested in. + +## An example + +Given this code: + +```js +function max(a, b) { + return a < b ? b : a; +} +``` + +And these tests: + +```js +describe('math', () => { + it('should return 4 for max(4, 3)', () => { + expect(max(4, 3)).eq(4); + }); + it('should return 4 for max(3, 4)', () => { + expect(max(3, 4)).eq(4); + }); +}); +``` + +Stryker will generate (amongst others) these mutants: + +```diff +function max(a, b) { +- return a < b ? b : a; ++ return true ? b : a; // 👽 1 ++ return false ? b : a; // 👽 2 ++ return a >= b ? b : a; // 👽 3 +} +``` + +Mutant 1 and 2 are killed by the tests. However, mutant 3 isn't killed. In fact, mutant 3 _cannot be killed_ because the mutated code is equivalent to the original. It is therefore called _equivalent mutant_. + +![equivalent mutant](./images/disable-mutants-equivalent-mutant.png) + +## Disable mutants + +StrykerJS supports 2 ways to disable mutants. + +1. [Exclude the mutator](#exclude-the-mutator). +2. [Using a `// Stryker disable` comment](#using-a--stryker-disable-comment). + +Disabled mutants will still end up in your report, but will get the `ignored` status. This means that they don't influence your mutation score, but are still visible if you want to look for them. This has no impact on the performance of mutation testing. + +## Exclude the mutator + +You can simply disable the mutator entirely. This is done by stating the mutator name in the `mutator.excludedMutations` array in your stryker configuration file: + +```json +{ + "mutator": { + "excludedMutations": ["EqualityOperator"] + } +} +``` + +The mutator name can be found in the clear-text or html report. + +If you've enabled the clear-text reporter (enabled by default), you can find the mutator name in your console: + +``` +#3. [Survived] EqualityOperator +src/math.js:3:12 +- return a < b ? b : a; ++ return a <= b ? b : a; +Tests ran: + math should return 4 for max(4, 3) + math should return 4 for max(3, 4) +``` + +In the html report, you will need to select the mutant you want to ignore, the drawer at the bottom has the mutator name in its title. + +However, disable the mutator for all your files is kind of a shotgun approach. Sure it works, but the mutator is now also disabled for other files and places. You probably want to use a comment instead. + +## Using a `// Stryker disable` comment. + +_Available since Stryker 5.4_ + +You can disable Stryker for a specific line of code using a comment. + + +```js +function max(a, b) { + // Stryker disable next-line all + return a < b ? b : a; +} +``` + +After running Stryker again, the report looks like this: + +![disable all](./images/disable-mutants-disable-all.png) + +This works, but is not exactly what we want. As you can see, all mutants on line 4 are not disabled. + +We can do better by specifying which mutator we want to ignore: + +```js +function max(a, b) { + // Stryker disable next-line EqualityOperator + return a < b ? b : a; +} +``` + +We can even provide a custom reason for disabling this mutator behind a colon (`:`). This reason will also end up in your report (drawer below) + +```js +function max(a, b) { + // Stryker disable next-line EqualityOperator: The <= mutant results in an equivalent mutant + return a < b ? b : a; +} +``` + +After running Stryker again, the report looks like this: + +![disable equality operator](./images/disable-mutants-disable-equality-operator.png) + +## Disable comment syntax + +_Available since Stryker 5.4_ + +The disabled comment is pretty powerful. Some more examples: + +Disable an entire file: + +```js +// Stryker disable all +function max(a, b) { + return a < b ? b : a; +} +``` + +Disable parts of a file: + +```js +// Stryker disable all +function max(a, b) { + return a < b ? b : a; +} +// Stryker restore all +function min(a, b) { + return a < b ? b : a; +} +``` + +Disable 2 mutators for an entire file with a custom reason: + +```js +// Stryker disable EqualityOperator,ObjectLiteral: We'll implement tests for these next sprint +function max(a, b) { + return a < b ? b : a; +} +``` + +Disable all mutators for an entire file, but restore the EqualityOperator for 1 line: + +```js +// Stryker disable all +function max(a, b) { + // Stryker restore EqualityOperator + return a < b ? b : a; +} +``` + +The syntax looks like this: + +``` +// Stryker [disable|restore] [next-line] *mutatorList*[: custom reason] +``` + +The comment always starts with `// Stryker`, followed by either `disable` or `restore`. Next, you can specify whether or not this comment targets the `next-line`, or all lines from this point on. The next part is the mutator list. This is either a comma separated list of mutators, or the "all" text signaling this comment targets all mutators. Last is an optional custom reason text, which follows the colon. + diff --git a/docs/images/disable-mutants-disable-all.png b/docs/images/disable-mutants-disable-all.png new file mode 100644 index 0000000000..7d22dcda3e Binary files /dev/null and b/docs/images/disable-mutants-disable-all.png differ diff --git a/docs/images/disable-mutants-disable-equality-operator.png b/docs/images/disable-mutants-disable-equality-operator.png new file mode 100644 index 0000000000..2dab34a13d Binary files /dev/null and b/docs/images/disable-mutants-disable-equality-operator.png differ diff --git a/docs/images/disable-mutants-equivalent-mutant.png b/docs/images/disable-mutants-equivalent-mutant.png new file mode 100644 index 0000000000..2773b8ad59 Binary files /dev/null and b/docs/images/disable-mutants-equivalent-mutant.png differ diff --git a/e2e/helpers.ts b/e2e/helpers.ts index ee56b40086..e7885cd645 100644 --- a/e2e/helpers.ts +++ b/e2e/helpers.ts @@ -54,7 +54,7 @@ export async function readMutationTestResult(eventResultDirectory = path.resolve return metricsResult; } -async function readMutationTestingJsonResult(jsonReportFile = path.resolve('reports', 'mutation', 'mutation.json')) { +export async function readMutationTestingJsonResult(jsonReportFile = path.resolve('reports', 'mutation', 'mutation.json')) { const mutationTestReportContent = await fsPromises.readFile(jsonReportFile, 'utf8'); const report = JSON.parse(mutationTestReportContent) as mutationTestReportSchema.MutationTestResult; const metricsResult = calculateMetrics(report.files); @@ -99,7 +99,7 @@ export async function expectMetrics(expectedMetrics: Partial) { expectActualMetrics(expectedMetrics, actualMetricsResult); } -function expectActualMetrics(expectedMetrics: Partial, actualMetricsResult: MetricsResult) { +export function expectActualMetrics(expectedMetrics: Partial, actualMetricsResult: MetricsResult) { const actualMetrics: Partial = {}; Object.entries(expectedMetrics).forEach(([key]) => { if (key === 'mutationScore' || key === 'mutationScoreBasedOnCoveredCode') { diff --git a/e2e/test/ignore-project/src/Add.js b/e2e/test/ignore-project/src/Add.js index 0d6e64385c..26d104ade0 100644 --- a/e2e/test/ignore-project/src/Add.js +++ b/e2e/test/ignore-project/src/Add.js @@ -1,26 +1,41 @@ -module.exports.add = function(num1, num2) { +module.exports.add = function (num1, num2) { return num1 + num2; }; -module.exports.addOne = function(number) { +module.exports.addOne = function (number) { number++; return number; }; -module.exports.negate = function(number) { +module.exports.negate = function (number) { return -number; }; -module.exports.notCovered = function(number) { +module.exports.notCovered = function (number) { return number > 10; }; -module.exports.isNegativeNumber = function(number) { +module.exports.userNextLineIgnored = function (number) { + // Stryker disable next-line all: Ignoring this on purpose + return number > 10; +}; + +// Stryker disable all +module.exports.blockUserIgnored = function (number) { + return number > 10; +}; +// Stryker restore all + +module.exports.userNextLineSpecificMutator = function (number) { + // Stryker disable next-line BooleanLiteral, ConditionalExpression: Ignore boolean and conditions + return true && number > 10; +}; + + +module.exports.isNegativeNumber = function (number) { var isNegative = false; - if(number < 0){ + if (number < 0) { isNegative = true; } return isNegative; }; - - diff --git a/e2e/test/ignore-project/stryker.conf.json b/e2e/test/ignore-project/stryker.conf.json index 90ba8aaf1c..9007679fbd 100644 --- a/e2e/test/ignore-project/stryker.conf.json +++ b/e2e/test/ignore-project/stryker.conf.json @@ -9,6 +9,7 @@ "reporters": [ "clear-text", "html", + "json", "event-recorder" ], "plugins": [ diff --git a/e2e/test/ignore-project/verify/verify.ts b/e2e/test/ignore-project/verify/verify.ts index 43b3ee5a4a..8c9374f904 100644 --- a/e2e/test/ignore-project/verify/verify.ts +++ b/e2e/test/ignore-project/verify/verify.ts @@ -1,16 +1,52 @@ -import { expectMetrics } from '../../../helpers'; +import { expect } from 'chai'; +import { MutantStatus } from 'mutation-testing-report-schema'; +import { expectMetricsJson, readMutationTestingJsonResult } from '../../../helpers'; describe('After running stryker on jest-react project', () => { it('should report expected scores', async () => { - await expectMetrics({ + await expectMetricsJson({ killed: 8, - ignored: 13, - mutationScore: 66.67, + ignored: 29, + mutationScore: 53.33, + }); + }); + + /* + -----------|---------|----------|-----------|------------|----------|---------| + File | % score | # killed | # timeout | # survived | # no cov | # error | + -----------|---------|----------|-----------|------------|----------|---------| + All files | 53.33 | 8 | 0 | 0 | 7 | 0 |*/ + + + it('should report mutants that are disabled by a comment with correct ignore reason', async () => { + const actualMetricsResult = await readMutationTestingJsonResult(); + const addResult = actualMetricsResult.childResults.find(file => file.name.endsWith('Add.js')).file!; + const mutantsAtLine31 = addResult.mutants.filter(({ location }) => location.start.line === 31) + const booleanLiteralMutants = mutantsAtLine31.filter(({mutatorName}) => mutatorName === 'BooleanLiteral'); + const conditionalExpressionMutants = mutantsAtLine31.filter(({mutatorName}) => mutatorName === 'ConditionalExpression'); + const equalityOperatorMutants = mutantsAtLine31.filter(({mutatorName}) => mutatorName === 'EqualityOperator'); + booleanLiteralMutants.forEach((booleanMutant) => { + expect(booleanMutant.status).eq(MutantStatus.Ignored); + expect(booleanMutant.statusReason).eq('Ignore boolean and conditions'); + }); + conditionalExpressionMutants.forEach((conditionalMutant) => { + expect(conditionalMutant.status).eq(MutantStatus.Ignored); + expect(conditionalMutant.statusReason).eq('Ignore boolean and conditions'); + }); + + equalityOperatorMutants.forEach((equalityMutant) => { + expect(equalityMutant.status).eq(MutantStatus.NoCoverage); + }); + }); + + it('should report mutants that result from excluded mutators with the correct ignore reason', async () => { + const actualMetricsResult = await readMutationTestingJsonResult(); + const circleResult = actualMetricsResult.childResults.find(file => file.name.endsWith('Circle.js')).file!; + const mutantsAtLine3 = circleResult.mutants.filter(({ location }) => location.start.line === 3) + + mutantsAtLine3.forEach((mutant) => { + expect(mutant.status).eq(MutantStatus.Ignored); + expect(mutant.statusReason).eq('Ignored because of excluded mutation "ArithmeticOperator"'); }); - /* ------------|---------|----------|-----------|------------|----------|---------| -File | % score | # killed | # timeout | # survived | # no cov | # error | ------------|---------|----------|-----------|------------|----------|---------| -All files | 66.67 | 8 | 0 | 0 | 4 | 0 |*/ }); }); diff --git a/packages/instrumenter/.vscode/launch.json b/packages/instrumenter/.vscode/launch.json index 7919d57ff5..98aae41cda 100644 --- a/packages/instrumenter/.vscode/launch.json +++ b/packages/instrumenter/.vscode/launch.json @@ -4,7 +4,7 @@ { "type": "node", "request": "launch", - "name": "Unit / Integration tests", + "name": "🎻 Unit / Integration tests", "program": "${workspaceRoot}/../../node_modules/mocha/bin/_mocha", "internalConsoleOptions": "openOnSessionStart", "outFiles": [ diff --git a/packages/instrumenter/src/transformers/babel-transformer.ts b/packages/instrumenter/src/transformers/babel-transformer.ts index 1c8c45007c..0ec16bd95c 100644 --- a/packages/instrumenter/src/transformers/babel-transformer.ts +++ b/packages/instrumenter/src/transformers/babel-transformer.ts @@ -11,6 +11,8 @@ import { ScriptFormat } from '../syntax'; import { allMutantPlacers, MutantPlacer, throwPlacementError } from '../mutant-placers'; import { Mutable, Mutant } from '../mutant'; +import { DirectiveBookkeeper } from './directive-bookkeeper'; + import { AstTransformer } from '.'; interface MutantsPlacement { @@ -37,6 +39,9 @@ export const transformBabel: AstTransformer = ( // Create a placementMap for the mutation switching bookkeeping const placementMap: PlacementMap = new Map(); + // Create the bookkeeper responsible for the // Stryker ... directives + const directiveBookkeeper = new DirectiveBookkeeper(); + // Now start the actual traversing of the AST // // On the way down: @@ -52,6 +57,8 @@ export const transformBabel: AstTransformer = ( // eslint-disable-next-line @typescript-eslint/no-unsafe-argument traverse(file.ast, { enter(path) { + directiveBookkeeper.processStrykerDirectives(path.node); + if (shouldSkip(path)) { path.skip(); } else { @@ -149,7 +156,11 @@ export const transformBabel: AstTransformer = ( function* mutate(node: NodePath): Iterable { for (const mutator of mutators) { for (const replacement of mutator.mutate(node)) { - yield { replacement, mutatorName: mutator.name, ignoreReason: formatIgnoreReason(mutator.name) }; + yield { + replacement, + mutatorName: mutator.name, + ignoreReason: directiveBookkeeper.findIgnoreReason(node.node.loc!.start.line, mutator.name) ?? formatIgnoreReason(mutator.name), + }; } } diff --git a/packages/instrumenter/src/transformers/directive-bookkeeper.ts b/packages/instrumenter/src/transformers/directive-bookkeeper.ts new file mode 100644 index 0000000000..c62af41f06 --- /dev/null +++ b/packages/instrumenter/src/transformers/directive-bookkeeper.ts @@ -0,0 +1,92 @@ +import { types } from '@babel/core'; +import { notEmpty } from '@stryker-mutator/util'; + +const WILDCARD = 'all'; +const DEFAULT_REASON = 'Ignored using a comment'; + +type IgnoreReason = string | undefined; + +interface Rule { + findIgnoreReason(mutatorName: string, line: number): IgnoreReason; +} + +class IgnoreRule implements Rule { + constructor(public mutatorNames: string[], public line: number | undefined, public ignoreReason: IgnoreReason, public previousRule: Rule) {} + + private matches(mutatorName: string, line: number): boolean { + const lineMatches = () => this.line === undefined || this.line === line; + const mutatorMatches = () => this.mutatorNames.includes(mutatorName) || this.mutatorNames.includes(WILDCARD); + return lineMatches() && mutatorMatches(); + } + + public findIgnoreReason(mutatorName: string, line: number): IgnoreReason { + if (this.matches(mutatorName, line)) { + return this.ignoreReason; + } + return this.previousRule.findIgnoreReason(mutatorName, line); + } +} + +class RestoreRule extends IgnoreRule { + constructor(mutatorNames: string[], line: number | undefined, previousRule: Rule) { + super(mutatorNames, line, undefined, previousRule); + } +} + +const rootRule: Rule = { + findIgnoreReason() { + return undefined; + }, +}; + +/** + * Responsible for the bookkeeping of "// Stryker" directives like "disable" and "restore". + */ +export class DirectiveBookkeeper { + // https://regex101.com/r/nWLLLm/1 + private readonly strykerCommentDirectiveRegex = /^\s?Stryker (disable|restore)(?: (next-line))? ([a-zA-Z, ]+)(?::(.+)?)?/; + + private currentIgnoreRule = rootRule; + + public processStrykerDirectives({ loc, leadingComments }: types.Node): void { + leadingComments + ?.map( + (comment) => + this.strykerCommentDirectiveRegex.exec(comment.value) as + | [fullMatch: string, directiveType: string, scope: string | undefined, mutators: string, reason: string | undefined] + | null + ) + .filter(notEmpty) + .forEach(([, directiveType, scope, mutators, optionalReason]) => { + const mutatorNames = mutators.split(',').map((mutator) => mutator.trim().toLowerCase()); + const reason = (optionalReason ?? DEFAULT_REASON).trim(); + switch (directiveType) { + case 'disable': + switch (scope) { + case 'next-line': + this.currentIgnoreRule = new IgnoreRule(mutatorNames, loc!.start.line, reason, this.currentIgnoreRule); + break; + default: + this.currentIgnoreRule = new IgnoreRule(mutatorNames, undefined, reason, this.currentIgnoreRule); + break; + } + break; + case 'restore': + switch (scope) { + case 'next-line': + this.currentIgnoreRule = new RestoreRule(mutatorNames, loc!.start.line, this.currentIgnoreRule); + break; + default: + this.currentIgnoreRule = new RestoreRule(mutatorNames, undefined, this.currentIgnoreRule); + break; + } + break; + } + }); + } + + public findIgnoreReason(line: number, mutatorName: string): string | undefined { + mutatorName = mutatorName.toLowerCase(); + return this.currentIgnoreRule.findIgnoreReason(mutatorName, line); + } +} diff --git a/packages/instrumenter/test/integration/instrumenter.it.spec.ts b/packages/instrumenter/test/integration/instrumenter.it.spec.ts index b2ed9383e3..05396bb5a7 100644 --- a/packages/instrumenter/test/integration/instrumenter.it.spec.ts +++ b/packages/instrumenter/test/integration/instrumenter.it.spec.ts @@ -45,8 +45,11 @@ describe('instrumenter integration', () => { it('should be able to instrument js files with a shebang in them', async () => { await arrangeAndActAssert('shebang.js'); }); - it('should not place ignored mutants', async () => { - await arrangeAndActAssert('ignore.js', createInstrumenterOptions({ excludedMutations: ['ArithmeticOperator'] })); + it('should not place excluded mutations', async () => { + await arrangeAndActAssert('excluded-mutations.js', createInstrumenterOptions({ excludedMutations: ['ArithmeticOperator'] })); + }); + it('should not place disabled mutants', async () => { + await arrangeAndActAssert('disabled.js'); }); it('should be able to instrument switch case statements (using the switchCaseMutantPlacer)', async () => { await arrangeAndActAssert('switch-case.js'); diff --git a/packages/instrumenter/test/unit/transformers/babel-transformer.spec.ts b/packages/instrumenter/test/unit/transformers/babel-transformer.spec.ts index 59cb5a2bf6..bac0a56aa8 100644 --- a/packages/instrumenter/test/unit/transformers/babel-transformer.spec.ts +++ b/packages/instrumenter/test/unit/transformers/babel-transformer.spec.ts @@ -27,7 +27,7 @@ describe('babel-transformer', () => { let mutantCollector: MutantCollector; const fooMutator: NodeMutator = { - name: 'foo', + name: 'Foo', *mutate(path) { if (path.isIdentifier() && path.node.name === 'foo') { yield types.identifier('bar'); @@ -35,7 +35,7 @@ describe('babel-transformer', () => { }, }; const plusMutator: NodeMutator = { - name: 'plus', + name: 'Plus', *mutate(path) { if (path.isBinaryExpression() && path.node.operator === '+') { yield types.binaryExpression('-', types.cloneNode(path.node.left, true), types.cloneNode(path.node.right, true)); @@ -68,9 +68,9 @@ describe('babel-transformer', () => { act(ast); expect(mutantCollector.mutants).lengthOf(2); expect(mutantCollector.mutants[0].replacementCode).eq('bar'); - expect(mutantCollector.mutants[0].mutatorName).eq('foo'); + expect(mutantCollector.mutants[0].mutatorName).eq('Foo'); expect(mutantCollector.mutants[1].replacementCode).eq('bar - baz'); - expect(mutantCollector.mutants[1].mutatorName).eq('plus'); + expect(mutantCollector.mutants[1].mutatorName).eq('Plus'); expect(normalizeWhitespaces(generator(ast.root).code)).contains('{ bar = bar + baz; foo = bar - baz; foo = bar + baz; }'); }); @@ -99,31 +99,31 @@ describe('babel-transformer', () => { const ast = createJSAst({ rawContent: 'foo("bar")' }); // Act - expect(() => act(ast)).throws('example.js:1:0 brokenPlacer could not place mutants with type(s): "foo".'); + expect(() => act(ast)).throws('example.js:1:0 brokenPlacer could not place mutants with type(s): "Foo".'); }); }); describe('excluded mutations', () => { it('should not place mutants that are ignored', () => { const ast = createJSAst({ rawContent: 'foo = bar + baz;' }); - context.options.excludedMutations = ['foo']; + context.options.excludedMutations = ['Foo']; act(ast); const result = normalizeWhitespaces(generator(ast.root).code); expect(result).not.include('bar = bar + baz;'); }); it('should still place other mutants', () => { const ast = createJSAst({ rawContent: 'foo = bar + baz;' }); - context.options.excludedMutations = ['foo']; + context.options.excludedMutations = ['Foo']; act(ast); const result = normalizeWhitespaces(generator(ast.root).code); expect(result).include('foo = bar - baz'); }); it('should collect ignored mutants with correct ignore message', () => { const ast = createJSAst({ rawContent: 'foo' }); - context.options.excludedMutations = ['foo']; + context.options.excludedMutations = ['Foo']; act(ast); expect(mutantCollector.mutants).lengthOf(1); - expect(mutantCollector.mutants[0].ignoreReason).eq('Ignored because of excluded mutation "foo"'); + expect(mutantCollector.mutants[0].ignoreReason).eq('Ignored because of excluded mutation "Foo"'); }); }); @@ -171,6 +171,375 @@ describe('babel-transformer', () => { }); }); + describe('with directive', () => { + function notIgnoredMutants() { + return mutantCollector.mutants.filter((mutant) => !mutant.ignoreReason); + } + function ignoredMutants() { + return mutantCollector.mutants.filter((mutant) => Boolean(mutant.ignoreReason)); + } + + describe('"Stryker disable next-line"', () => { + it('should ignore all mutants with the leading comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line all + const foo = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(0); + }); + + it('should ignore mutants that spawn multiple lines', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line all + const foo = 1 + + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(0); + }); + + it('should be supported in the middle of a function call', () => { + const ast = createTSAst({ + rawContent: ` + console.log( + // Stryker disable next-line plus + 1 + 1 + ); + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(0); + }); + + it('should only ignore a single line', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line all + let foo = 1 + 1; + foo = 1 + 1; + `, + }); + act(ast); + expect(ignoredMutants()).lengthOf(2); + expect(ignoredMutants().map((mutant) => mutant.original.loc!.start.line)).deep.eq([3, 3]); + expect(notIgnoredMutants()).lengthOf(2); + expect(notIgnoredMutants().map((mutant) => mutant.original.loc!.start.line)).deep.eq([4, 4]); + }); + + it('should ignore a mutant when lead with a "Stryker disable next-line mutator" comment targeting that mutant', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line plus + const foo = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(1); + expect(ignoredMutants()).lengthOf(1); + const ignoredMutant = ignoredMutants()[0]; + expect(ignoredMutant.mutatorName).eq('Plus'); + }); + + it('should ignore mutants when lead with a "Stryker disable next-line mutator" comment targeting with multiple mutators', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line plus,foo + const foo = 1 + 1; + `, + }); + act(ast); + expect(ignoredMutants()).lengthOf(2); + }); + + it('should ignore mutants when lead with multiple "Stryker disable next-line mutator" comments spread over multiple lines', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line plus + // Stryker disable next-line foo + const foo = 1 + 1; + `, + }); + act(ast); + expect(ignoredMutants()).lengthOf(2); + }); + + it('should ignore mutants when lead with a "Stryker disable next-line all" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line all + const foo = 1 + 1; + `, + }); + act(ast); + expect(ignoredMutants()).lengthOf(2); + }); + + it('should allow users to add an ignore reasons', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line foo: I don't like foo + const foo = "bar"; + `, + }); + act(ast); + expect(mutantCollector.mutants[0].ignoreReason).to.equal("I don't like foo"); + }); + + it('should allow multiple user comments for one line', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable next-line foo: I don't like foo + // Stryker disable next-line plus: I also don't like plus + const foo = 1 + 1; + `, + }); + act(ast); + expect(mutantCollector.mutants.find((mutant) => mutant.mutatorName === 'Foo')?.ignoreReason).to.equal("I don't like foo"); + expect(mutantCollector.mutants.find((mutant) => mutant.mutatorName === 'Plus')?.ignoreReason).to.equal("I also don't like plus"); + }); + }); + + describe('"Stryker disable"', () => { + it('should ignore all following mutants', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(0); + expect(ignoredMutants()).lengthOf(3); + }); + + it('should not ignore all mutants following a "Stryker restore" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + // Stryker restore all + + const foo = 'a'; + `, + }); + act(ast); + expect(ignoredMutants()).lengthOf(3); + expect(notIgnoredMutants()).lengthOf(1); + const notIgnoredMutant = notIgnoredMutants()[0]; + expect(notIgnoredMutant.mutatorName).eq('Foo'); + }); + + it('should ignore all mutants, even if some where explicitly disabled with a "Stryker disable next-line" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + a = 1 + 1; + // Stryker disable next-line Foo: with a custom reason + foo = 1 + 1; + c = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(0); + expect(ignoredMutants()).lengthOf(4); + }); + + it('should allow an ignore reason', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all: Disable everything + // Stryker disable foo: But have a reason for disabling foo + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + const foo = 'a'; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(0); + expect( + mutantCollector.mutants.filter((mutant) => mutant.mutatorName === 'Plus').every((mutant) => mutant.ignoreReason === 'Disable everything') + ).to.be.true; + expect(mutantCollector.mutants.find((mutant) => mutant.mutatorName === 'Foo')!.ignoreReason).to.equal('But have a reason for disabling foo'); + }); + + it('should be able to restore a specific mutator that was previously explicitly disabled', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable foo,plus + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + // Stryker restore foo + const foo = 'a'; + const d = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(1); + expect(notIgnoredMutants()[0].mutatorName).eq('Foo'); + }); + + it('should be able to restore a specific mutator after all mutators were disabled', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + // Stryker restore foo + const foo = 'a'; + const d = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(1); + expect(notIgnoredMutants()[0].mutatorName).eq('Foo'); + }); + + it('should restore all mutators following a "Stryker restore" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable foo,plus + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + // Stryker restore all + const foo = 'a'; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(1); + expect(notIgnoredMutants()[0].original.loc!.start.line).eq(7); + }); + + it('should restore a specific mutators when using a "Stryker restore mutant" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + const a = 1 + 1; + const b = 1 + 1; + const c = 1 + 1; + // Stryker restore foo + const foo = 'a'; + const d = 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(1); + }); + + it('should allow to restore for next-line using a specific "Stryker restore next-line mutator" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + 1 + 1; + // Stryker restore next-line plus + 1 + foo; + 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(1); + expect(ignoredMutants()).lengthOf(3); + const actualNotIgnoredMutant = notIgnoredMutants()[0]; + expect(actualNotIgnoredMutant.mutatorName).eq('Plus'); + expect(actualNotIgnoredMutant.original.loc!.start.line).eq(5); + }); + + it('should allow multiple restore for next-line using a specific "Stryker restore next-line mutator" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + 1 + 1; + // Stryker restore next-line plus + // Stryker restore next-line foo + 1 + foo; + 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(2); + expect(ignoredMutants()).lengthOf(2); + const [actualRestoredMutantPlus, actualRestoredMutantFoo] = notIgnoredMutants(); + expect(actualRestoredMutantPlus.mutatorName).eq('Plus'); + expect(actualRestoredMutantPlus.original.loc!.start.line).eq(6); + expect(actualRestoredMutantFoo.mutatorName).eq('Foo'); + expect(actualRestoredMutantFoo.original.loc!.start.line).eq(6); + }); + + it('should allow to restore for next-line using a "Stryker restore next-line all" comment', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + 1 + 1; + // Stryker restore next-line all + 1 + foo; + 1 + 1; + `, + }); + act(ast); + expect(notIgnoredMutants()).lengthOf(2); + expect(ignoredMutants()).lengthOf(2); + const actualNotIgnoredPlusMutant = notIgnoredMutants()[0]; + const actualNotIgnoredFooMutant = notIgnoredMutants()[1]; + expect(actualNotIgnoredPlusMutant.mutatorName).eq('Plus'); + expect(actualNotIgnoredPlusMutant.original.loc!.start.line).eq(5); + expect(actualNotIgnoredFooMutant.mutatorName).eq('Foo'); + expect(actualNotIgnoredFooMutant.original.loc!.start.line).eq(5); + }); + + it('should allow disable, restore mutator, disable all', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable all + 1 + 1; + // Stryker restore plus + 1 + 1; + // Stryker disable all + 1 + 1; + `, + }); + act(ast); + + expect(notIgnoredMutants()).lengthOf(1); + expect(ignoredMutants()).lengthOf(2); + const actualNotIgnoredFooMutant = notIgnoredMutants()[0]; + expect(actualNotIgnoredFooMutant.mutatorName).eq('Plus'); + expect(actualNotIgnoredFooMutant.original.loc!.start.line).eq(5); + }); + + it('should allow disable mutator, restore all, disable mutator', () => { + const ast = createTSAst({ + rawContent: ` + // Stryker disable plus + 1 + 1; + // Stryker restore all + 1 + 1; + // Stryker disable plus + 1 + 1; + `, + }); + act(ast); + + expect(notIgnoredMutants()).lengthOf(1); + expect(ignoredMutants()).lengthOf(2); + const actualNotIgnoredFooMutant = notIgnoredMutants()[0]; + expect(actualNotIgnoredFooMutant.mutatorName).eq('Plus'); + expect(actualNotIgnoredFooMutant.original.loc!.start.line).eq(5); + }); + }); + }); + describe('with mutationRanges', () => { let ast: ScriptAst; diff --git a/packages/instrumenter/testResources/instrumenter/disabled.js b/packages/instrumenter/testResources/instrumenter/disabled.js new file mode 100644 index 0000000000..f073fa4193 --- /dev/null +++ b/packages/instrumenter/testResources/instrumenter/disabled.js @@ -0,0 +1,14 @@ +function factorial (num) { + if (typeof (num) !== 'number') throw new Error("Input must be a number."); + if (num < 0) throw new Error("Input must not be negative."); + var i = 2, + o = 1; + + // Stryker disable next-line BlockStatement: Infinite loop + while (i <= num) { + // Stryker disable next-line UpdateOperator: Infinite loop + o *= i++; + } + + return o; +}; diff --git a/packages/instrumenter/testResources/instrumenter/disabled.js.out.snap b/packages/instrumenter/testResources/instrumenter/disabled.js.out.snap new file mode 100644 index 0000000000..273389f4d3 --- /dev/null +++ b/packages/instrumenter/testResources/instrumenter/disabled.js.out.snap @@ -0,0 +1,86 @@ +// Jest Snapshot v1, https://goo.gl/fbAQLP + +exports[`instrumenter integration should not place disabled mutants 1`] = ` +"function stryNS_9fa48() { + var g = new Function(\\"return this\\")(); + var ns = g.__stryker__ || (g.__stryker__ = {}); + + if (ns.activeMutant === undefined && g.process && g.process.env && g.process.env.__STRYKER_ACTIVE_MUTANT__) { + ns.activeMutant = g.process.env.__STRYKER_ACTIVE_MUTANT__; + } + + function retrieveNS() { + return ns; + } + + stryNS_9fa48 = retrieveNS; + return retrieveNS(); +} + +stryNS_9fa48(); + +function stryCov_9fa48() { + var ns = stryNS_9fa48(); + var cov = ns.mutantCoverage || (ns.mutantCoverage = { + static: {}, + perTest: {} + }); + + function cover() { + var c = cov.static; + + if (ns.currentTestId) { + c = cov.perTest[ns.currentTestId] = cov.perTest[ns.currentTestId] || {}; + } + + var a = arguments; + + for (var i = 0; i < a.length; i++) { + c[a[i]] = (c[a[i]] || 0) + 1; + } + } + + stryCov_9fa48 = cover; + cover.apply(null, arguments); +} + +function stryMutAct_9fa48(id) { + var ns = stryNS_9fa48(); + + function isActive(id) { + if (ns.activeMutant === id) { + if (ns.hitCount !== void 0 && ++ns.hitCount > ns.hitLimit) { + throw new Error('Stryker: Hit count limit reached (' + ns.hitCount + ')'); + } + + return true; + } + + return false; + } + + stryMutAct_9fa48 = isActive; + return isActive(id); +} + +function factorial(num) { + if (stryMutAct_9fa48(\\"0\\")) { + {} + } else { + stryCov_9fa48(\\"0\\"); + if (stryMutAct_9fa48(\\"3\\") ? typeof num === 'number' : stryMutAct_9fa48(\\"2\\") ? false : stryMutAct_9fa48(\\"1\\") ? true : (stryCov_9fa48(\\"1\\", \\"2\\", \\"3\\"), typeof num !== (stryMutAct_9fa48(\\"4\\") ? \\"\\" : (stryCov_9fa48(\\"4\\"), 'number')))) throw new Error(stryMutAct_9fa48(\\"5\\") ? \\"\\" : (stryCov_9fa48(\\"5\\"), \\"Input must be a number.\\")); + if (stryMutAct_9fa48(\\"9\\") ? num >= 0 : stryMutAct_9fa48(\\"8\\") ? num <= 0 : stryMutAct_9fa48(\\"7\\") ? false : stryMutAct_9fa48(\\"6\\") ? true : (stryCov_9fa48(\\"6\\", \\"7\\", \\"8\\", \\"9\\"), num < 0)) throw new Error(stryMutAct_9fa48(\\"10\\") ? \\"\\" : (stryCov_9fa48(\\"10\\"), \\"Input must not be negative.\\")); + var i = 2, + o = 1; // Stryker disable next-line BlockStatement: Infinite loop + + while (stryMutAct_9fa48(\\"13\\") ? i > num : stryMutAct_9fa48(\\"12\\") ? i < num : stryMutAct_9fa48(\\"11\\") ? false : (stryCov_9fa48(\\"11\\", \\"12\\", \\"13\\"), i <= num)) { + // Stryker disable next-line UpdateOperator: Infinite loop + o *= i++; + } + + return o; + } +} + +;" +`; diff --git a/packages/instrumenter/testResources/instrumenter/ignore.js b/packages/instrumenter/testResources/instrumenter/excluded-mutations.js similarity index 100% rename from packages/instrumenter/testResources/instrumenter/ignore.js rename to packages/instrumenter/testResources/instrumenter/excluded-mutations.js diff --git a/packages/instrumenter/testResources/instrumenter/ignore.js.out.snap b/packages/instrumenter/testResources/instrumenter/excluded-mutations.js.out.snap similarity index 94% rename from packages/instrumenter/testResources/instrumenter/ignore.js.out.snap rename to packages/instrumenter/testResources/instrumenter/excluded-mutations.js.out.snap index 3867747dae..015222ace2 100644 --- a/packages/instrumenter/testResources/instrumenter/ignore.js.out.snap +++ b/packages/instrumenter/testResources/instrumenter/excluded-mutations.js.out.snap @@ -1,6 +1,6 @@ // Jest Snapshot v1, https://goo.gl/fbAQLP -exports[`instrumenter integration should not place ignored mutants 1`] = ` +exports[`instrumenter integration should not place excluded mutations 1`] = ` "function stryNS_9fa48() { var g = new Function(\\"return this\\")(); var ns = g.__stryker__ || (g.__stryker__ = {});