[WIP] Documentation revamp

README.md

@@ -84,7 +84,7 @@ per developer. CI usage for licensed developers is included.
 
 Opensource projects have to acquire their free license per repository.
 
-The license distribution happens through the `mutant-license` gem in mutants dependencies.
+The license distribution happens through the `mutant-license` gem in mutant’s dependencies.
 This gem is dynamically generated per licensee and comes with a unique license gem source
 URL.
 
@@ -99,8 +99,7 @@ end
 
 The mutant license gem contains metadata that allows mutant to verify licensed use.
 
-For commercial licenses mutant checks the git commit author or the configured git email
-to be in the set of licensed developers.
+For commercial licenses mutant checks the git commit author or the configured git email to be in the set of licensed developers.
 
 For opensource licenses mutant checks the git remotes against the licensed git repositories.
 This allows the project maintainer to sign up and not bother collaborators with the details.
@@ -177,8 +176,7 @@ of their private time.
 Additionally, the following features where sponsored by organizations:
 
 * The `mutant-minitest` integration was sponsored by [Arkency](https://arkency.com/)
-* Mutant's initial concurrency support was sponsored by an undisclosed company that does
- currently not wish to be listed here.
+* Mutant's initial concurrency support was sponsored by an undisclosed company that does currently not wish to be listed here.
 
 ### Legal
 

docs/commercial-support.md

@@ -10,5 +10,5 @@ Covers 1 incident per quarter, with a max response time of 7 days.
 Scope is limited to mutant not the application or infrastructure.
 
 For support email [Markus Schirp](mailto:[email protected]?subject=Mutant%20Support).
-Please email using the same domain as the roiginal license email or explain
+Please email using the same domain as the original license email or explain
 your connection to the license.

docs/limitations.md

@@ -6,8 +6,8 @@ Subject
 
 Mutant cannot emit mutations for some subjects.
 
-* methods defined within a closure. For example, methods defined using `module_eval`, `class_eval`,
- `define_method`, or `define_singleton_method`:
+* methods defined within a closure. For example, methods defined using
+ `module_eval`, `class_eval`, `define_method`, or `define_singleton_method`:
 
  ```ruby
  class Example
@@ -29,7 +29,7 @@ Mutant cannot emit mutations for some subjects.
  end
  ```
 
-* singleton methods not defined on a constant or `self`
+* singleton methods not defined on a constant or on `self`
 
  ```ruby
  class Foo
@@ -38,6 +38,15 @@ Mutant cannot emit mutations for some subjects.
 
  myself = self
  def myself.qux; end # cannot mutate
+
+ class << self
+ def corge; end # ok
+ end
+ end
+
+ foo = "Hello"
+ class << foo
+ def greet!; end # cannot mutate
  end
  ```
 

docs/mutant-rspec.md

@@ -21,7 +21,7 @@ To add mutant to your rspec code base you need to:
 
 ## Run through example
 
-This uses [mbj/auom](https://github.com/mbj/auom) a small library that
+This uses [mbj/auom](https://github.com/mbj/auom), a small library that
 has 100% mutation coverage. Its tests execute very fast and do not have any IO
 so its a good playground example to interact with.
 
@@ -128,3 +128,5 @@ these example groups *must* kill the mutation.
 ```sh
 RAILS_ENV=test bundle exec mutant run -r ./config/environment --use rspec User
 ```
+
+TODO something about using `mutant_expression` metadata here.

docs/nomenclature.md

@@ -1,13 +1,33 @@
 Nomenclature
 ============
 
-The following explains several nouns you may experience in mutant's documentation.
-It's a good idea to familiarize yourself before moving on.
+This document explains several nouns you may experience in mutant's
+documentation. It's a good idea to familiarize yourself before moving on.
+
+## Mutation Testing
+The practice of systematically applying small changes one at a time to a
+codebase then re-running the (relevant) tests for each change. If the tests
+fail, then they cover the semantics of those changes. If the tests continue to
+pass with the changes in place, the tests do not cover the complete semantics of
+the changed parts of the codebase.
+
+Each type of change (for example, `a && b` to `a`) is known as a
+[Mutation](#mutation), and is applied to the pristine codebase - that is, the
+changes are not stacked up, but applied in isolation. The places in the code
+where the changes can be made are called [Subjects](#subject). The changed
+Subject is then referred to (potentially confusingly) as a Mutant, or sometimes
+a Mutation.
+
+Mutation testing can be useful for suggesting semantic simplifications to your
+code, as well as highlighting gaps in your tests as you are writing them.
 
 ## AST
 
-Acronym for [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree)
-and the level of abstraction mutant operates on.
+Acronym for [Abstract Syntax Tree][AST] and the level of abstraction mutant
+operates on. In short, this is a representation of the structure and content of
+your code stored in memory which mutant alters.
+
+[AST]: https://en.wikipedia.org/wiki/Abstract_syntax_tree
 
 ## Subject
 
@@ -18,38 +38,172 @@ Mutant currently supports the following subjects:
 * Instance methods
 * Singleton (class) methods
 
-Other subjects (constants, class bodies for DSLs, ...) are possible but aren't
-implemented in the OSS version.
+Other subjects are possible (even project-specific subjects) but aren't
+implemented in the OSS version. Some examples are:
+* Constants
+* Class bodies for DSLs
 
-## Mutation operator
+The more subjects that mutant can alter in your project, the more mutations it
+can create, and so the higher the confidence you can have that your tests cover
+the semantics of your application. Please get in touch if you require subjects
+beyond those implemented in Mutant already - support may be available in the
+commercial version.
 
-A transformation applied to the AST of a subject. Mutant knows the following high level operator
-classes:
+## Mutation
 
-* Semantic Reduction
-* Orthogonal Replacement
-* [Noop](#neutral-noop-tests)
+An alteration to your codebase. Each mutation represents a hypothesis that
+ideally gets falsified by the tests. Some examples of mutations and the
+hypotheses they represent:
 
-An exhaustive list can be found in the [mutant-meta](https://github.com/mbj/mutant/tree/master/meta)
-subdirectory of the source.
+```ruby
+# before application of mutation operator
+def equal_to_5?(number)
+ number == 5
+end
 
-## Mutation
+# after application of `#==` -> `#eql?`
+# hypothesis: the conversion semantics of #== are not necessary for this method
+def equal_to_5?
+ number.eql?(5)
+end
+
+# after removal of `number == 5`
+# hypothesis: that line of the method is dead code
+def equal_to_5?; end
+
+# after replacement of `number == 5` with `true`
+# hypothesis: the tests only check the happy-path
+def equal_to_5?(number)
+ true
+end
+```
+
+### Categories of mutation
+Mutations broadly fall into a couple of categories. There is no reason to
+deliberately seek to learn these categories (there’s no quiz at the end of this
+semester!) - we’re just providing them to give you some ideas of what high-level
+classes of changes mutant makes to your code. For a full list of all the types
+of mutation that mutant can perform, see the code in the [meta directory][meta]
+
+**Evil** mutations are mutations which should cause the test suite to fail,
+while **neutral** mutations are expected not to break the test suite.
+[No-ops](#no-op) are the only form of neutral mutation in mutant - all others
+are evil.
+
+[meta]: https://github.com/mbj/mutant/tree/master/meta
+
+#### Semantic Reduction
+
+This type of transformation replaces a piece of code which has somewhat complex
+semantics with one that has simpler semantics. To aid understanding, here are a
+couple of different sub-categories you could put them into.
+
+* **Method call replacement**
+
+ Example:
+
+ ```ruby
+ num == 1
+ # gets replaced by
+ num.eql?(1)
+ ```
+
+ `#==` commonly performs conversion between types in addition to checking
+ equality, while `#eql?` tends to check only that the class and instance
+ variables are equal. (sticking with numbers, `1 == 1.0`, but `!1.eql?(1.0)`)
+ Therefore, `#== -> #eql?` is a semantic reduction.
+
+ You could also think of a semantic reduction in these cases as an increase
+ in “strictness” of the code. `#equal?` (object identity) is a stricter
+ equality test than `#eql?`, which is stricter than `#==`.
+
+* **Code removal**
+
+ Example:
+
+ ```ruby
+ def my_method
+ if some_condition?
+ do_something
+ end
+ end
+
+ # replaced by
 
-The result of applying a mutation operator to the AST of a subject. A mutation represents a
-hypothesis that ideally gets falsified by the tests.
+ def my_method
+ end
+ ```
+
+ It's arguable whether or not this is its own category - we include it only
+ to show that entire chunks of code are removed in some mutations performed
+ by mutant. These mutations make mutant the ultimate ruby dead code finder!
+
+* **Interface reduction**
+
+ Example:
+
+ ```ruby
+ def join2(one, two, options)
+ one + two
+ end
+
+ # replaced by
+
+ def join2(one two)
+ one + two
+ end
+ ```
+
+#### Orthogonal Replacement
+Example:
+
+```ruby
+def true?
+ a.equal?(true)
+end
+
+def true?
+ a.equal?(false)
+end
+```
+
+Unlike semantic reduction, where the result is a simpler or stricter version of
+the input, an orthogonal replacement changes code with a given function into
+code which does something quite different (usually the opposite, or a different
+value of the same type). This category is probably better understood by
+examples:
+ * `true` -> `false`
+ * `#>` -> `#<`
+
+These mutations are less frequent, owing to their relative lack of use as
+suggestions for dead code removal or code simplification. The category really
+only exists to include constant replacements such as `true` -> `false` - which
+is needed to ensure that you have tested that `true` is indeed the required
+value in that circumstance.
+
+#### **No-Op**
+
+This type of mutation makes no changes to the code, but performs the test
+execution in exactly the same way as an "evil" mutation It is needed in order
+to ensure that mutant’s presence, and prior mutations’ side effects, do not
+cause the test suite to fail.
+
+A no-op mutation is added at the beginning of the mutation test run (to ensure
+that using mutant alone doesn't cause the test suite to fail)
 
 ## Insertion
 
 The process of inserting a mutation into the runtime environment.
 Mutant currently supports insertion via dynamically created monkeypatches.
 
-Other insertion strategies (such as "boot time") are possible but aren't implemented
-in the OSS version.
+Other insertion strategies (such as "boot time") are possible but aren't
+implemented in the OSS version.
 
 ## Isolation
 
-The attempt to isolate the (side) effects of killing a mutation via an integration
-to prevent a mutation leaking into adjacent concurrent, or future mutations.
+The attempt to isolate the (side) effects of killing a mutation via an
+integration to prevent a mutation leaking into adjacent concurrent, or future
+mutations.
 
 Examples of sources for leaks are
 
@@ -58,18 +212,23 @@ Examples of sources for leaks are
 * DB State
 * File system
 
-Natively, mutant offers fork isolation. This works for any state within the executing
-Ruby process. For all state reachable via IO, it's the test author's responsibility to
-provide proper isolation.
+Natively, mutant offers fork isolation. This works for any state within the
+executing Ruby process. For all state reachable via IO, it's the test author's
+responsibility to provide proper isolation (for example, by wrapping tests which
+touch the database in a transaction)
 
 ## Integration
 
-The method used to determine if a specific inserted mutation is covered by tests.
+The method used to determine if a specific inserted mutation is covered by
+tests.
 
 Currently mutant supports integrations for:
 
-* [mutant-rspec](/docs/mutant-rspec.md) for [rspec](https://rspec.info)
-* [mutant-minitest](/docs/mutant-minitest.md) for [minitest](https://github.com/seattlerb/minitest)
+* [mutant-rspec](/docs/mutant-rspec.md) for [rspec][rspec]
+* [mutant-minitest](/docs/mutant-minitest.md) for [minitest][minitest]
+
+[rspec]: https://rspec.info
+[minitest]: https://github.com/seattlerb/minitest
 
 ## Report
 
@@ -80,3 +239,6 @@ Mutant currently provides two different reporters:
 
 A reporter producing a machine readable report does not exist in the OSS version
 at the time of writing this documentation.
+
+See the [reading-reports.md](./reading-reports.md) file for documentation on the
+information provided by those reports.