Creating a Test Suite

Creating a Test Suite

The goal of this page is to go through the necessary pieces to create the test suite for a given puzzle. We will be using the Nurikabe test suite as an example.

Overview

The test suite runs using JUnitCore. The documentation for it can be found here if you want to understand the framework for running the tests. This is not necessary to know if you are just writing tests, but may be useful if we want to overhaul the framework.

To find the currently implemented tests, go to src/test/java/* All the current tests are in the /puzzles folder. Within each puzzle's folder, there is a list of classes, one for each rule for the puzzle (Case rules, Direct rules, and Contradiction rules). JUnitCore handles running the class and all functions within it.

There is a 'test' task within Gradle which will run all tests located within the src/test/java/puzzles folder; additionally, the tests are run during the 'build' task, which runs when confirming a merge into the dev branch.

Preparation

If you haven't already, read over the Adding Rules section on the Implementing Puzzles page. Test cases are built based on what the existing rule is testing. Without an understanding of how a given puzzle's rule works, it will be impossible to create good test cases for it.

As a summary, a rule's function will return either a string describing why the rule does not apply or null when the rule is valid. We want to create test cases for both scenarios. Each rule has a function that will run on a cell to make sure the move made at that point follows the given rule and an equivalent function that will check through all modified cells during a given transition.

Basic Rule and Contradiction Rule

Let's start by taking a look at a Direct Rule (or Basic Rule) in Nurikabe, src/test/java/puzzles/BlackBottleNeckDirectRuleTest. Creating a rule for a contradiction rule follows the same format.

Tests run by creating a fake scenario in which the player makes a move on the board, and the rule runs to check that the move made matches the conditions set by the rule. The first test does the equivalent of making the move depicted below and then clicking on the Black Bottleneck Direct Rule.

@Test
public void BlackBottleNeckDirectRule_TwoSurroundBlackTest() throws InvalidFileFormatException {
    TestUtilities.importTestBoard(
            "puzzles/nurikabe/rules/BlackBottleNeckDirectRule/SimpleBlackBottleNeck", nurikabe);
    TreeNode rootNode = nurikabe.getTree().getRootNode();
    TreeTransition transition = rootNode.getChildren().get(0);
    transition.setRule(RULE);

    NurikabeBoard board = (NurikabeBoard) transition.getBoard();
    NurikabeCell cell = board.getCell(2, 1);
    cell.setData(NurikabeType.BLACK.toValue());

    board.addModifiedData(cell);

    Assert.assertNull(RULE.checkRule(transition));

    for (int i = 0; i < board.getHeight(); i++) {
        for (int k = 0; k < board.getWidth(); k++) {
            Point point = new Point(k, i);
            if (point.equals(cell.getLocation())) {
                Assert.assertNull(RULE.checkRuleAt(transition, board.getCell(k, i)));
            } else {
                Assert.assertNotNull(RULE.checkRuleAt(transition, board.getCell(k, i)));
            }
        }
    }
}

In the first four lines, we start by creating a test board, which we load in from src/main/test/resources/puzzles/*. This board is then used to test the direct rule. We also grab a transition node that we can edit to create our next "move".

We then set the cell at position (2, 1) to black and "commit" it to the transition node. The proof tree now knows that the player has made that move and can check the logic accordingly. Then, we check that the transition node is correctly applied in the transition and that the rule correctly evaluates all points on the grid as an invalid application of the rule except for the modified cell. This is what we want! The rule should be considered valid at the cell we changed to black and invalid everywhere else. That way, if we were to place a black tile elsewhere on the grid, the proof tree would display a red arrow.

Of course, this is not the only case where a black tile may be bottlenecked. Thus, we need to create more mock boards that will go through all possible cases of a black bottleneck. There are two more cases that the function tests: A black stuck in a corner, and a case where there is no black bottleneck at all. I would recommend reading through both of those cases as well.

Contradiction Rule

Contradiction rules follow the same exact format as Direct Rules.

Case Rule

Case rules are slightly different to test. Because a case rule is valid to use at any time, the test shouldn't check for the rule's validity. Instead, it checks that the number of cases is correct (if the case rule should create multiple cases) and that the newly created boards didn't magically change any tiles it shouldn't have. Read the BlackOrWhiteCaseRuleTest for an example.