From 3bd7e74319d040b7492185302ab83c1eba66756c Mon Sep 17 00:00:00 2001 From: Telyn Z <175827+telyn@users.noreply.github.com> Date: Tue, 8 Dec 2020 20:41:44 +0000 Subject: [PATCH 1/2] First-pass through most of the documentation. Still to-do: * Big rework of docs/mutant-rspec.md * More jokes (or fewer jokes - depending on the main authors' preferences) * Get rid of the fancy quote marks & trailing whitespace * Continue reworking docs/nomenclature, starting with the TODO under Mutation Operators, and then carrying on from Insertion until the end of the file. --- README.md | 21 +++++---- docs/commercial-support.md | 2 +- docs/limitations.md | 21 +++++++-- docs/mutant-rspec.md | 2 +- docs/nomenclature.md | 89 +++++++++++++++++++++++++++++++++----- docs/reading-reports.md | 2 +- meta/README.md | 50 +++++++++++++++++++++ 7 files changed, 158 insertions(+), 29 deletions(-) create mode 100644 meta/README.md diff --git a/README.md b/README.md index 232a0f1be..21c27bff5 100644 --- a/README.md +++ b/README.md @@ -7,16 +7,17 @@ mutant ## What is Mutant? -Mutant is a mutation testing tool for Ruby. Mutation testing is a technique to verify semantic coverage of your code. +Mutant is a mutation testing tool for Ruby. + Mutation testing is the practice of systematically applying small changes one at a time to a codebase then re-running the (relevant) tests for each change. + ## Why do I want it? -Mutant adds to your toolbox: Detection of uncovered semantics in your code. -Coverage becomes a meaningful metric! +Mutation testing is an extra tool in your toolbox which detects uncovered semantics in your code. Coverage becomes a meaningful metric! -On each detection of uncovered semantics you have the opportunity to: +With each detection of uncovered semantics you have the opportunity to: -* Delete dead code, as you do not want the extra semantics not specified by the tests +* Delete dead code * Add (or improve a test) to cover the unwanted semantics. * Learn something new about the semantics of Ruby and your direct and indirect dependencies. @@ -47,7 +48,7 @@ per developer. Opensource projects have to acquire their free license per repository. -The license distribution happens through the `mutant-license` gem in mutants dependencies. +The license distribution happens through the `mutant-license` gem in mutant’s dependencies. This gem is dynamically generated per licensee and comes with a unique license gem source URL. @@ -62,8 +63,7 @@ end The mutant license gem contains metadata that allows mutant to verify licensed use. -For commercial licenses mutant checks the git commit author or the configured git email -to be in the set of licensed developers. +For commercial licenses mutant checks the git commit author or the configured git email to be in the set of licensed developers. For opensource licenses mutant checks the git remotes against the license whitelist. This allows the project maintainer to sign up and not bother collaborators with the details. @@ -71,7 +71,7 @@ This allows the project maintainer to sign up and not bother collaborators with There are, apart from initial license gem installation, no remote interactions for license validation. -To inquire for license please contact [Markus Schirp](mailto:mbj@schirp-dso.com?subject=Mutant%20License). +To inquire for a license please contact [Markus Schirp](mailto:mbj@schirp-dso.com?subject=Mutant%20License). ### Pricing @@ -109,8 +109,7 @@ of their private time. Additionally, the following features where sponsored by organizations: * The `mutant-minitest` integration was sponsored by [Arkency](https://arkency.com/) -* Mutant's initial concurrency support was sponsored by an undisclosed company that does - currently not wish to be listed here. +* Mutant's initial concurrency support was sponsored by an undisclosed company that does currently not wish to be listed here. ### Legal diff --git a/docs/commercial-support.md b/docs/commercial-support.md index 851e17344..aaf5a8d7e 100644 --- a/docs/commercial-support.md +++ b/docs/commercial-support.md @@ -10,5 +10,5 @@ Covers 1 incident per quarter, with a max response time of 7 days. Scope is limited to mutant not the application or infrastructure. For support email [Markus Schirp](mailto:mbj@schirp-dso.com?subject=Mutant%20Support). -Please email using the same domain as the roiginal license email or explain +Please email using the same domain as the original license email or explain your connection to the license. diff --git a/docs/limitations.md b/docs/limitations.md index 181177061..e19463db4 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -6,8 +6,8 @@ Subject Mutant cannot emit mutations for some subjects. -* methods defined within a closure. For example, methods defined using `module_eval`, `class_eval`, - `define_method`, or `define_singleton_method`: +* methods defined within a closure. For example, methods defined using + `module_eval`, `class_eval`, `define_method`, or `define_singleton_method`: ```ruby class Example @@ -29,7 +29,7 @@ Mutant cannot emit mutations for some subjects. end ``` -* singleton methods not defined on a constant or `self` +* inline-style singleton methods not defined on a constant or on `self` ```ruby class Foo @@ -41,6 +41,21 @@ Mutant cannot emit mutations for some subjects. end ``` +* multiline-style singleton methods not defined on `self` + + ```ruby + class Foo + class << self + def bar; end # ok + end + end + + foo = "Hello" + class << foo + def greet!; end # cannot mutate + end + ``` + * methods defined with eval: ```ruby diff --git a/docs/mutant-rspec.md b/docs/mutant-rspec.md index a0a4a9b54..3311e8e1b 100644 --- a/docs/mutant-rspec.md +++ b/docs/mutant-rspec.md @@ -21,7 +21,7 @@ To add mutant to your rspec code base you need to: ## Run through example -This uses [mbj/auom](https://github.com/mbj/auom) a small library that +This uses [mbj/auom](https://github.com/mbj/auom), a small library that has 100% mutation coverage. Its tests execute very fast and do not have any IO so its a good playground example to interact with. diff --git a/docs/nomenclature.md b/docs/nomenclature.md index a92393269..3db9430d1 100644 --- a/docs/nomenclature.md +++ b/docs/nomenclature.md @@ -1,13 +1,22 @@ Nomenclature ============ -The following explains several nouns you may experience in mutant's documentation. +This document explains several nouns you may experience in mutant's documentation. It's a good idea to familiarize yourself before moving on. +## Mutation Testing +The practice of systematically applying small changes one at a time to a codebase then re-running the (relevant) tests for each change. If the tests fail, then they cover the semantics of those changes. If the tests continue to pass with the changes in place, the tests do not cover the complete semantics of the changed parts of the codebase. + +Each type of change (for example, “change `a && b` to `a`”) is known as a [Mutation Operator](#mutation-operator), and is applied to the pristine codebase - that is, the changes are not stacked up, but applied in isolation. The places in which the Mutation Operators can make their changes are called [Subjects](#subject). The changed Subject is then referred to as a [Mutation](#mutation). + +Mutation testing can be useful for suggesting semantic simplifications to your code, as well as highlighting gaps in your tests as you are writing them. + ## AST Acronym for [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) and the level of abstraction mutant operates on. +In short, this is a representation of the structure and content of your code +stored in memory which mutant alters. ## Subject @@ -18,25 +27,81 @@ Mutant currently supports the following subjects: * Instance methods * Singleton (class) methods -Other subjects (constants, class bodies for DSLs, ...) are possible but aren't -implemented in the OSS version. +Other subjects are possible (even project-specific subjects) but aren't implemented in the OSS version. Please get in touch with the authors. Some examples are: +* Constants +* Class bodies for DSLs + +The more subjects that mutant can alter in your project, the more mutations it can create, and so the higher your confidence you can have that your tests cover the semantics of your application. Please get in touch if you require subjects beyond those implemented in Mutant already - support may be available in the commercial version. + +## Mutation Operator + +A transformation applied to the AST of a subject. Mutant knows the following high level operator classes (you will not be tested on your knowledge of these terms) + +Please be aware that there is no reason to learn these terms (there’s no quiz at the end of this semester!) - we’re just providing them to give you some ideas of what high-level classes of changes mutant makes to your code. For a full list of all the Mutation Operators in mutant, see the code in the [meta directory](https://github.com/mbj/mutant/tree/master/meta). + +* **Semantic Reduction** + + This type of transformation replaces a piece of code which has somewhat complex semantics with one that has simpler semantics. To aid understanding, here are a couple of different sub-categories you could put them into. + + * **Method call replacement** - for example, `#==` -> `#eql?` -> `#equal?` + + `#==` commonly performs conversion between types in addition to checking equality, while `#eql?` tends to check only that the class and instance variables are equal. Therefore we would say that #== is semantically simpler. + + You could also think of a semantic reduction in these cases as an increase in “strictness” of the code. `#equal?` is a stricter equality test than `#eql?`, and so on + + * **Code removal** - For example, `def my_method; do_something; end` -> `def my_method; end` + + We could argue[^1] that this category includes all of the following: + + * Removing a particular expression, from the AST entirely + * Replacing an expression with `nil`, or `true`, or `false`, or other simple literals. + + * **Interface reduction** + + TODO + -## Mutation operator +[^1]: After all; these are not official terms handed down from an authority on mutation testing. We’ve invented them for the purposes of introducing the concepts here. -A transformation applied to the AST of a subject. Mutant knows the following high level operator -classes: +* **Orthogonal Replacement** - for example `>` -> `<` -* Semantic Reduction -* Orthogonal Replacement -* [Noop](#neutral-noop-tests) + Unlike semantic reduction, where the result is a “simpler” version of the input, an orthogonal replacement transforms some code with a given function into code with a similar semantic complexity, but which does something different (usually opposite). This category is probably better understood by examples: + * `true` -> `false` + * `#>` -> `#<` + + These transformations are less common, +* **No-Op** - This type of operator does nothing. It is needed in order to ensure that mutant’s presence, and prior mutations’ side effects, do not cause the test suite to fail. -An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta) +An exhaustive list can be found in the subdirectory of the source. ## Mutation -The result of applying a mutation operator to the AST of a subject. A mutation represents a -hypothesis that ideally gets falsified by the tests. +The result of applying a mutation operator to the AST of a subject. A mutation +represents a hypothesis that ideally gets falsified by the tests. Some example hypotheses: + +```ruby +# before application of mutation operator +def equal_to_5?(number) + number == 5 +end + +# after application of `#==` -> `#eql?` +# hypothesis: the conversion semantics of #== are not necessary for this method +def equal_to_5? + number.eql?(5) +end + +# after removal of `number == 5` +# hypothesis: that line of the method is dead code +def equal_to_5?; end + +# after replacement of `number == 5` with `true` +# hypothesis: the tests only check the happy-path +def equal_to_5?(number) + true +end +``` ## Insertion diff --git a/docs/reading-reports.md b/docs/reading-reports.md index fcb817fcc..789edfb27 100644 --- a/docs/reading-reports.md +++ b/docs/reading-reports.md @@ -21,7 +21,7 @@ Mutation output is grouped by selection groups. Each group contains three sectio **Format**: - ```text +, ```text - [INTEGRATION]:0:[SPEC LOCATION]:[SPEC DESCRIPTION] - [INTEGRATION]:1:[SPEC LOCATION]:[SPEC DESCRIPTION] ``` diff --git a/meta/README.md b/meta/README.md new file mode 100644 index 000000000..40c188858 --- /dev/null +++ b/meta/README.md @@ -0,0 +1,50 @@ +Mutant’s Mutation Operators +=== + +This directory contains all the [Mutation Operators](../docs/nomenclature.md#mutation-operator)[^mutation-operator] that Mutant can apply. + +The filenames mostly correspond to the names of code elements as used by the [parser](https://github.com/whitequark/parser) and [unparser](https://github.com/mbj/unparser) libraries. For the most part they are the same as keywords within Ruby, but here are some explanations of the less obvious ones: + +* `str` and `sym` are string and symbol literals that do not contain dynamic elements (`"I'm a dstring #{i_am_the_dynamic_element}"`) + * `dst` and `dsym` are the equivalents for strings and symbols which do +* `regexp` is a regular expression without any options (`/like this/`) + * `regopt` is a regular expression with options (`/like this/i`) +* a few come in regular and `-asgn` variants - the latter are the assignment versions. E.g., `||=` is the assignment version (`or_asgn.rb`) of `||` (`or.rb`) +* `gvar`, `cvar`, `ivar` and `lvar` refer to global (`$VERBOSE`), class (`@@debug`), instance (`@amount`) and local (`response`) variables, respectively. +* `csend` is the conditional send operator (`&.`) +* `cbase` is a top-level constant (`::Kernel` or `::Errno::ENOENT`), whereas `const` is a "normal" constant (`Kernel` or `Errno::ENOENT`) +* `casgn` is an assignment to a constant (`VERSION = "0.1.0"`) + +All the files in this directory use a fairly simple DSL. To explain it, let's run through (a slightly modified version of) `return.rb`: + +```ruby +# Adds an example matching expressions like `return foo`, where foo +# can be anything. In parser's s-expression DSL, this would match +# `s(:return, s(...))`, but not the simpler `s(:return)`. +Mutant::Meta::Example.add :return do + # Each Example can only have one source, which provides a pattern to match + source 'return foo' + + # multiple `mutation`s are provided. Each is simply a ruby + # source-code representation of the code after the change, using + # the same placeholders as the source + mutation 'foo' # replaces `return foo` with `foo` + mutation 'return nil' # replaces `return nil` with `nil` + mutation 'return self' # replaces `return foo` with `return self` +end + +# This Example matches the simpler `s(:return)` case (or simply, +# `return` expressions) +Mutant::Meta::Example.add :return do + source 'return' + + # singleton_mutations is a convenience method for the following: + #  mutation 'nil' + #  mutation 'self' + # This just helps ensure consistency throughout - a very large + # number of expressions can be usefully mutated to `nil` or `self` + singleton_mutations +end +``` + +[^mutation-operator]: Remember, Mutation Operator just means “a change that can be applied to the code” \ No newline at end of file From 44e481be04b91531ad98ae6dbb89427154475998 Mon Sep 17 00:00:00 2001 From: Telyn Z <175827+telyn@users.noreply.github.com> Date: Mon, 14 Dec 2020 21:08:09 +0000 Subject: [PATCH 2/2] remove all the humongous lines --- docs/limitations.md | 10 +- docs/mutant-rspec.md | 2 + docs/nomenclature.md | 231 ++++++++++++++++++++++++++++------------ docs/reading-reports.md | 2 +- 4 files changed, 169 insertions(+), 76 deletions(-) diff --git a/docs/limitations.md b/docs/limitations.md index e19463db4..42bcc48c7 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -29,7 +29,7 @@ Mutant cannot emit mutations for some subjects. end ``` -* inline-style singleton methods not defined on a constant or on `self` +* singleton methods not defined on a constant or on `self` ```ruby class Foo @@ -38,15 +38,9 @@ Mutant cannot emit mutations for some subjects. myself = self def myself.qux; end # cannot mutate - end - ``` - -* multiline-style singleton methods not defined on `self` - ```ruby - class Foo class << self - def bar; end # ok + def corge; end # ok end end diff --git a/docs/mutant-rspec.md b/docs/mutant-rspec.md index fe9d1a75c..482ca8aca 100644 --- a/docs/mutant-rspec.md +++ b/docs/mutant-rspec.md @@ -128,3 +128,5 @@ these example groups *must* kill the mutation. ```sh RAILS_ENV=test bundle exec mutant run -r ./config/environment --use rspec User ``` + +TODO something about using `mutant_expression` metadata here. \ No newline at end of file diff --git a/docs/nomenclature.md b/docs/nomenclature.md index 3db9430d1..550680c87 100644 --- a/docs/nomenclature.md +++ b/docs/nomenclature.md @@ -1,22 +1,33 @@ Nomenclature ============ -This document explains several nouns you may experience in mutant's documentation. -It's a good idea to familiarize yourself before moving on. +This document explains several nouns you may experience in mutant's +documentation. It's a good idea to familiarize yourself before moving on. ## Mutation Testing -The practice of systematically applying small changes one at a time to a codebase then re-running the (relevant) tests for each change. If the tests fail, then they cover the semantics of those changes. If the tests continue to pass with the changes in place, the tests do not cover the complete semantics of the changed parts of the codebase. - -Each type of change (for example, “change `a && b` to `a`”) is known as a [Mutation Operator](#mutation-operator), and is applied to the pristine codebase - that is, the changes are not stacked up, but applied in isolation. The places in which the Mutation Operators can make their changes are called [Subjects](#subject). The changed Subject is then referred to as a [Mutation](#mutation). - -Mutation testing can be useful for suggesting semantic simplifications to your code, as well as highlighting gaps in your tests as you are writing them. +The practice of systematically applying small changes one at a time to a +codebase then re-running the (relevant) tests for each change. If the tests +fail, then they cover the semantics of those changes. If the tests continue to +pass with the changes in place, the tests do not cover the complete semantics of +the changed parts of the codebase. + +Each type of change (for example, `a && b` to `a`) is known as a +[Mutation](#mutation), and is applied to the pristine codebase - that is, the +changes are not stacked up, but applied in isolation. The places in the code +where the changes can be made are called [Subjects](#subject). The changed +Subject is then referred to (potentially confusingly) as a Mutant, or sometimes +a Mutation. + +Mutation testing can be useful for suggesting semantic simplifications to your +code, as well as highlighting gaps in your tests as you are writing them. ## AST -Acronym for [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) -and the level of abstraction mutant operates on. -In short, this is a representation of the structure and content of your code -stored in memory which mutant alters. +Acronym for [Abstract Syntax Tree][AST] and the level of abstraction mutant +operates on. In short, this is a representation of the structure and content of +your code stored in memory which mutant alters. + +[AST]: https://en.wikipedia.org/wiki/Abstract_syntax_tree ## Subject @@ -27,58 +38,22 @@ Mutant currently supports the following subjects: * Instance methods * Singleton (class) methods -Other subjects are possible (even project-specific subjects) but aren't implemented in the OSS version. Please get in touch with the authors. Some examples are: +Other subjects are possible (even project-specific subjects) but aren't +implemented in the OSS version. Some examples are: * Constants * Class bodies for DSLs -The more subjects that mutant can alter in your project, the more mutations it can create, and so the higher your confidence you can have that your tests cover the semantics of your application. Please get in touch if you require subjects beyond those implemented in Mutant already - support may be available in the commercial version. - -## Mutation Operator - -A transformation applied to the AST of a subject. Mutant knows the following high level operator classes (you will not be tested on your knowledge of these terms) - -Please be aware that there is no reason to learn these terms (there’s no quiz at the end of this semester!) - we’re just providing them to give you some ideas of what high-level classes of changes mutant makes to your code. For a full list of all the Mutation Operators in mutant, see the code in the [meta directory](https://github.com/mbj/mutant/tree/master/meta). - -* **Semantic Reduction** - - This type of transformation replaces a piece of code which has somewhat complex semantics with one that has simpler semantics. To aid understanding, here are a couple of different sub-categories you could put them into. - - * **Method call replacement** - for example, `#==` -> `#eql?` -> `#equal?` - - `#==` commonly performs conversion between types in addition to checking equality, while `#eql?` tends to check only that the class and instance variables are equal. Therefore we would say that #== is semantically simpler. - - You could also think of a semantic reduction in these cases as an increase in “strictness” of the code. `#equal?` is a stricter equality test than `#eql?`, and so on - - * **Code removal** - For example, `def my_method; do_something; end` -> `def my_method; end` - - We could argue[^1] that this category includes all of the following: - - * Removing a particular expression, from the AST entirely - * Replacing an expression with `nil`, or `true`, or `false`, or other simple literals. - - * **Interface reduction** - - TODO - - -[^1]: After all; these are not official terms handed down from an authority on mutation testing. We’ve invented them for the purposes of introducing the concepts here. - -* **Orthogonal Replacement** - for example `>` -> `<` - - Unlike semantic reduction, where the result is a “simpler” version of the input, an orthogonal replacement transforms some code with a given function into code with a similar semantic complexity, but which does something different (usually opposite). This category is probably better understood by examples: - * `true` -> `false` - * `#>` -> `#<` - - These transformations are less common, -* **No-Op** - This type of operator does nothing. It is needed in order to ensure that mutant’s presence, and prior mutations’ side effects, do not cause the test suite to fail. - -An exhaustive list can be found in the -subdirectory of the source. +The more subjects that mutant can alter in your project, the more mutations it +can create, and so the higher the confidence you can have that your tests cover +the semantics of your application. Please get in touch if you require subjects +beyond those implemented in Mutant already - support may be available in the +commercial version. ## Mutation -The result of applying a mutation operator to the AST of a subject. A mutation -represents a hypothesis that ideally gets falsified by the tests. Some example hypotheses: +An alteration to your codebase. Each mutation represents a hypothesis that +ideally gets falsified by the tests. Some examples of mutations and the +hypotheses they represent: ```ruby # before application of mutation operator @@ -103,18 +78,132 @@ def equal_to_5?(number) end ``` +### Categories of mutation +Mutations broadly fall into a couple of categories. There is no reason to +deliberately seek to learn these categories (there’s no quiz at the end of this +semester!) - we’re just providing them to give you some ideas of what high-level +classes of changes mutant makes to your code. For a full list of all the types +of mutation that mutant can perform, see the code in the [meta directory][meta] + +**Evil** mutations are mutations which should cause the test suite to fail, +while **neutral** mutations are expected not to break the test suite. +[No-ops](#no-op) are the only form of neutral mutation in mutant - all others +are evil. + +[meta]: https://github.com/mbj/mutant/tree/master/meta + +#### Semantic Reduction + +This type of transformation replaces a piece of code which has somewhat complex +semantics with one that has simpler semantics. To aid understanding, here are a +couple of different sub-categories you could put them into. + +* **Method call replacement** + + Example: + + ```ruby + num == 1 + # gets replaced by + num.eql?(1) + ``` + + `#==` commonly performs conversion between types in addition to checking + equality, while `#eql?` tends to check only that the class and instance + variables are equal. (sticking with numbers, `1 == 1.0`, but `!1.eql?(1.0)`) + Therefore, `#== -> #eql?` is a semantic reduction. + + You could also think of a semantic reduction in these cases as an increase + in “strictness” of the code. `#equal?` (object identity) is a stricter + equality test than `#eql?`, which is stricter than `#==`. + +* **Code removal** + + Example: + + ```ruby + def my_method + if some_condition? + do_something + end + end + + # replaced by + + def my_method + end + ``` + + It's arguable whether or not this is its own category - we include it only + to show that entire chunks of code are removed in some mutations performed + by mutant. These mutations make mutant the ultimate ruby dead code finder! + +* **Interface reduction** + + Example: + + ```ruby + def join2(one, two, options) + one + two + end + + # replaced by + + def join2(one two) + one + two + end + ``` + +#### Orthogonal Replacement +Example: + +```ruby +def true? + a.equal?(true) +end + +def true? + a.equal?(false) +end +``` + +Unlike semantic reduction, where the result is a simpler or stricter version of +the input, an orthogonal replacement changes code with a given function into +code which does something quite different (usually the opposite, or a different +value of the same type). This category is probably better understood by +examples: + * `true` -> `false` + * `#>` -> `#<` + +These mutations are less frequent, owing to their relative lack of use as +suggestions for dead code removal or code simplification. The category really +only exists to include constant replacements such as `true` -> `false` - which +is needed to ensure that you have tested that `true` is indeed the required +value in that circumstance. + +#### **No-Op** + +This type of mutation makes no changes to the code, but performs the test +execution in exactly the same way as an "evil" mutation It is needed in order +to ensure that mutant’s presence, and prior mutations’ side effects, do not +cause the test suite to fail. + +A no-op mutation is added at the beginning of the mutation test run (to ensure +that using mutant alone doesn't cause the test suite to fail) + ## Insertion The process of inserting a mutation into the runtime environment. Mutant currently supports insertion via dynamically created monkeypatches. -Other insertion strategies (such as "boot time") are possible but aren't implemented -in the OSS version. +Other insertion strategies (such as "boot time") are possible but aren't +implemented in the OSS version. ## Isolation -The attempt to isolate the (side) effects of killing a mutation via an integration -to prevent a mutation leaking into adjacent concurrent, or future mutations. +The attempt to isolate the (side) effects of killing a mutation via an +integration to prevent a mutation leaking into adjacent concurrent, or future +mutations. Examples of sources for leaks are @@ -123,18 +212,23 @@ Examples of sources for leaks are * DB State * File system -Natively, mutant offers fork isolation. This works for any state within the executing -Ruby process. For all state reachable via IO, it's the test author's responsibility to -provide proper isolation. +Natively, mutant offers fork isolation. This works for any state within the +executing Ruby process. For all state reachable via IO, it's the test author's +responsibility to provide proper isolation (for example, by wrapping tests which +touch the database in a transaction) ## Integration -The method used to determine if a specific inserted mutation is covered by tests. +The method used to determine if a specific inserted mutation is covered by +tests. Currently mutant supports integrations for: -* [mutant-rspec](/docs/mutant-rspec.md) for [rspec](https://rspec.info) -* [mutant-minitest](/docs/mutant-minitest.md) for [minitest](https://github.com/seattlerb/minitest) +* [mutant-rspec](/docs/mutant-rspec.md) for [rspec][rspec] +* [mutant-minitest](/docs/mutant-minitest.md) for [minitest][minitest] + +[rspec]: https://rspec.info +[minitest]: https://github.com/seattlerb/minitest ## Report @@ -145,3 +239,6 @@ Mutant currently provides two different reporters: A reporter producing a machine readable report does not exist in the OSS version at the time of writing this documentation. + +See the [reading-reports.md](./reading-reports.md) file for documentation on the +information provided by those reports. diff --git a/docs/reading-reports.md b/docs/reading-reports.md index 789edfb27..fcb817fcc 100644 --- a/docs/reading-reports.md +++ b/docs/reading-reports.md @@ -21,7 +21,7 @@ Mutation output is grouped by selection groups. Each group contains three sectio **Format**: -, ```text + ```text - [INTEGRATION]:0:[SPEC LOCATION]:[SPEC DESCRIPTION] - [INTEGRATION]:1:[SPEC LOCATION]:[SPEC DESCRIPTION] ```