TESTCASE generation using TCases

This is to show how testcases can be generated using https://github.com/Cornutum/tcases engine.

Let's assume we have some application, in this repository it is: src/main/java/class_under_test/MyUtil.java. To automatically generate testcases we need to:

1. add appropriate dependencies required by TCases in pom
2. create model file: like src/test/model/MyUtil-Input.json
3. create appropriate variable classes: like in class_under_test_dsl package
4. clean install project-> it generates non executable testcases json file: src/test/java/generated/MyUtil-Test.json and immediately copies it to target/test-classes as well
5. run src/main/java/TestcaseGenerator.java -> it generates executable java file: src/test/java/generated/testMyUtil.java basing on target/test-classes/MyUtil-Test.json
6. run testMyUtil -> it runs tests: src/main/java/class_under_test/MyUtil.java

In this repository it is required to execute just steps: 4-6 to see the demo. This example is considered to be just kind of proof-of-concept or a kind of draft demonstrating the possibility of generating executable testcases in specific way.

GIVEN/THEN

Code generation is possible, because of the main concept which is about the fact each variable holds information both about GIVEN and THEN part of executable testcase.

For example variable InputContent.Capitals with value FirstLetterCapitalized can produce given part by providing information about specific input type for application under test which should have 1. letter capital. However, at the same time, it also is capable of generating assertions as it also contains information what is the desired resulting state after application is run given this specific input. Here, the result should just be string with capitalized 1. letter. All other variables also bring givens and thens for each testcase.

WHEN

This part is done for each testcase by src/main/java/class_under_test_dsl/When.java class. InputIsNotNull variable in this model also constructs when part due to possibility of null input value. It also influences assertion in special way when assertThrows needs to be used. Things could be more simple, if Gherkin language was used or at least Groovy. Here we have most complicated scenario with pure Java.

DataGenerator

There is a need of data generator to supply input data according to variable values. It has to be capable of creating data set by multiple properties at the same time to fulfill specific testcase needs.

Technical concept

The main technical idea is to deserialize MyUtil-Test.json file into class_under_test_dsl class instances, so that they are filled with single testcase data on one hand while having knowledge about code generation on the other. This keeps model information separate from code generation items. Code is generated by closures which are part of variable related classes (Capitals, Dots, InputIsNotNull, Words) and When class.

Notes

There is significant work required to create classes needed for deserialization process. This is not meant for simple cases like unit testing, however. Only projects which require very strict coverage should have generated executable testcases so that the time needed for implementing the solution would pay off. I think this more about heavy acceptance testing. I am sure generating Cucumber, Spock or Groovy testcases should be easier to implement. I also think part of this functionality could be included in TCases - this would leave less work for the end user to achieve the goal.

Generating executable testcases is the Holy Grail of QA engineering and I am showing here how to make one step ahead.