Skip to content
This repository has been archived by the owner on Apr 14, 2023. It is now read-only.

Commit

Permalink
Merge pull request #1313 from finos/mandatory-type-in-profile
Browse files Browse the repository at this point in the history 
feat(#1301): make type mandatory in profile
  • Loading branch information
@pdaulbyscottlogic
pdaulbyscottlogic authored Sep 13, 2019
2 parents 6d7dcff + 1cada5a commit 91120d2
Show file tree
Hide file tree
Showing 16 changed files with 573 additions and 1,183 deletions.
28 changes: 4 additions & 24 deletions .../java/com/scottlogic/deg/generator/inputs/validation/TypingRequiredPerFieldValidator.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ public void validate(Profile profile) {
List<String> untypedFields = profile.getFields().stream()
.filter(field -> !sufficientlyRestrictsFieldTypes(decisionTree.getRootNode(), field))
.map(nonCompliantField -> nonCompliantField.name +
" is untyped; add an ofType, equalTo or inSet constraint, or mark it as null")
" is untyped; add an ofType, or add its type to the field definition")
.collect(Collectors.toList());

if (!untypedFields.isEmpty()){
Expand All @@ -72,28 +72,8 @@ public void validate(Profile profile) {
}

private static boolean sufficientlyRestrictsFieldTypes(ConstraintNode node, Field fieldToCheck) {
// a constraint node is sufficient if any of its constraints, or any of its decision nodes,
// are sufficient
return
node.getAtomicConstraints().stream()
.anyMatch(constraint -> sufficientlyRestrictsFieldTypes(constraint, fieldToCheck))
||
node.getDecisions().stream()
.anyMatch(decisionNode -> sufficientlyRestrictsFieldTypes(decisionNode, fieldToCheck));
}

private static boolean sufficientlyRestrictsFieldTypes(DecisionNode node, Field fieldToCheck) {
// a decision node is sufficient if all of its branches are sufficient
return node.getOptions().stream()
.allMatch(constraintNode -> sufficientlyRestrictsFieldTypes(constraintNode, fieldToCheck));
}

private static boolean sufficientlyRestrictsFieldTypes(AtomicConstraint constraint, Field fieldToCheck) {
return
constraint.getField().equals(fieldToCheck)
&& (constraint instanceof IsOfTypeConstraint
|| constraint instanceof IsNullConstraint
|| constraint instanceof IsInSetConstraint
|| constraint instanceof EqualToConstraint);
return node.getAtomicConstraints().stream()
.filter(atomicConstraint -> atomicConstraint.getField().equals(fieldToCheck))
.anyMatch(atomicConstraint -> atomicConstraint instanceof IsOfTypeConstraint);
}
}
67 changes: 1 addition & 66 deletions ...test/java/com/scottlogic/deg/orchestrator/cucumber/features/ExhaustiveCombination.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -221,25 +221,6 @@ Feature: User can create data across multiple fields for all combinations availa
Then the profile is invalid because "Unable to determine correct type for `01-01-2010T00:00:00.000Z`.\nEnsure strings are wrapped in double-quotes."
And no data is created

Scenario: Running an exhaustive combination strategy with null values (empty strings) should be successful
Given the following fields exist:
| foo |
| bar |
And foo is anything but null
And bar is anything but null
And foo is in set:
| "" |
| 1 |
And bar is in set:
| 2 |
| 3 |
Then the following data should be generated:
| foo | bar |
| "" | 2 |
| 1 | 2 |
| "" | 3 |
| 1 | 3 |

Scenario: Running an exhaustive combination strategy with null values (null) should be successful
Given the following fields exist:
| foo |
Expand Down Expand Up @@ -680,46 +661,6 @@ Feature: User can create data across multiple fields for all combinations availa
| 1 | 20 | 13 |
| 1 | 20 | 14 |

Scenario: Running an exhaustive combination strategy across fields with a duplicate data option in different formats (integer & string) in a field should be successful
Given the following fields exist:
| foo1 |
| foo2 |
| foo3 |
And foo1 is anything but of type "datetime"
And foo1 is anything but null
And foo2 is anything but of type "datetime"
And foo2 is anything but null
And foo3 is anything but of type "datetime"
And foo3 is anything but null
And foo1 is in set:
| 1 |
And foo2 is in set:
| 20 |
| "20" |
| 20 |
| "20" |
And foo3 is in set:
| 10 |
| 11 |
| 12 |
| 13 |
| 14 |
| "14" |
Then the following data should be generated:
| foo1 | foo2 | foo3 |
| 1 | 20 | 10 |
| 1 | 20 | 11 |
| 1 | 20 | 12 |
| 1 | 20 | 13 |
| 1 | 20 | 14 |
| 1 | 20 | "14" |
| 1 | "20" | 10 |
| 1 | "20" | 11 |
| 1 | "20" | 12 |
| 1 | "20" | 13 |
| 1 | "20" | 14 |
| 1 | "20" | "14" |

Scenario: Running an exhaustive combination strategy across fields with non ordered data options should be successful
Given the following fields exist:
| foo1 |
Expand All @@ -729,7 +670,7 @@ Feature: User can create data across multiple fields for all combinations availa
And foo1 is anything but null
And foo2 is of type "string"
And foo2 is anything but null
And foo3 is anything but of type "datetime"
And foo3 is of type "string"
And foo3 is anything but null
And foo1 is in set:
| 1 |
Expand All @@ -750,32 +691,26 @@ Feature: User can create data across multiple fields for all combinations availa
| 1 | "zab" | "testZ" |
| 1 | "zab" | "0" |
| 1 | "zab" | "aaa" |
| 1 | "zab" | 14 |
| 1 | "zaa" | "!" |
| 1 | "zaa" | "testZ" |
| 1 | "zaa" | "0" |
| 1 | "zaa" | "aaa" |
| 1 | "zaa" | 14 |
| 1 | "test" | "!" |
| 1 | "test" | "testZ" |
| 1 | "test" | "0" |
| 1 | "test" | "aaa" |
| 1 | "test" | 14 |
| 0 | "zab" | "!" |
| 0 | "zab" | "testZ" |
| 0 | "zab" | "0" |
| 0 | "zab" | "aaa" |
| 0 | "zab" | 14 |
| 0 | "zaa" | "!" |
| 0 | "zaa" | "testZ" |
| 0 | "zaa" | "0" |
| 0 | "zaa" | "aaa" |
| 0 | "zaa" | 14 |
| 0 | "test" | "!" |
| 0 | "test" | "testZ" |
| 0 | "test" | "0" |
| 0 | "test" | "aaa" |
| 0 | "test" | 14 |

Scenario: Running an exhaustive combination strategy that includes an "if" statement should be successful
Given the generation strategy is full
Expand Down
1 change: 1 addition & 0 deletions orchestrator/src/test/java/com/scottlogic/deg/orchestrator/cucumber/features/Random.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ Feature: User can generate valid data for all types (string, integer, decimal, o

Scenario: The generator produces valid 'Null' data in random mode
Given foo is null
And foo is of type "string"
And the generator can generate at most 5 rows
Then the following data should be generated:
| foo |
Expand Down
1 change: 1 addition & 0 deletions orchestrator/src/test/java/com/scottlogic/deg/orchestrator/cucumber/features/Unique.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Feature: As a user
And foo is greater than 0
And foo is anything but null
And there is a field bar
And bar is of type "string"
And bar is equal to "not unique"
And the generation strategy is random
Then the following data should be generated:
Expand Down
18 changes: 4 additions & 14 deletions ...strator/src/test/java/com/scottlogic/deg/orchestrator/cucumber/features/Violation.feature
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,10 @@ Feature: The violations mode of the Data Helix app can be run in violations mode

Scenario: Running the generator in violate mode where equal to is not violated is successful
Given foo is equal to 8
And foo is of type "decimal"
And the generation strategy is full
And we do not violate any equal to constraints
And we do not violate any of type constraints
Then the following data should be generated:
| foo |
| 8 |
Expand Down Expand Up @@ -77,8 +79,10 @@ Feature: The violations mode of the Data Helix app can be run in violations mode

Scenario: The generator produces violating 'Null' data in random mode
Given foo is null
And foo is of type "decimal"
And the generation strategy is random
And the data requested is violating
And we do not violate any of type constraints
Then 5 rows of data are generated
And foo contains anything but null

Expand Down Expand Up @@ -137,17 +141,3 @@ Feature: The violations mode of the Data Helix app can be run in violations mode
And foo contains only string data
And foo contains strings matching /[a-z]{0,9}/

Scenario: The generator should produce correct violating data for anyOf construction
Given there is a constraint:
"""
{ "anyOf": [
{ "field": "foo", "is": "ofType", "value": "string" },
{ "field": "foo", "is": "ofType", "value": "decimal" }
]}
"""
And the generation strategy is random
And the data requested is violating
Then some data should be generated
And foo contains anything but string data
And foo contains anything but numeric data

Loading

0 comments on commit 91120d2

Please sign in to comment.