Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/split compliance chart #72

Merged
Merged
155 changes: 123 additions & 32 deletions docs/_docs/2. functional.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,78 @@ split into those covering _required_ vs _optional_ features.

### Summary of results

The chart below uses the **score** from the **overall** column in the [Summary results table](#summary-of-results) below to visually
present the feature completeness of each validator implementation.
The charts below illustrate the percentage of required and optional test cases that pass for each draft of the JSON Schema,
and aggregate/overall graphs that show the pass rate across all the drafts an implementation supports.
Results are broken down into separate _required_ and _optional_ graphs.

#### Overall

<div>
<canvas id="overallRequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="overallOptionalChart"></canvas>
</div>

#### Draft_2020_12

<div>
<canvas id="draft2020RequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="draft2020OptionalChart"></canvas>
</div>

#### Draft_2019_09

<div>
<canvas id="draft2019RequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="draft2019OptionalChart"></canvas>
</div>

#### Draft_07

<div>
<canvas id="draft07RequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="draft07OptionalChart"></canvas>
</div>

#### Draft_06

<div>
<canvas id="draft06RequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="draft06OptionalChart"></canvas>
</div>

#### Draft_04

<div>
<canvas id="draft04RequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="draft04OptionalChart"></canvas>
</div>

#### Draft_03

<div>
<canvas id="draft03RequiredChart"></canvas>
</div>
&nbsp;
<div>
<canvas id="summaryChart"></canvas>
<canvas id="draft03OptionalChart"></canvas>
</div>

#### Summary results table
Expand Down Expand Up @@ -110,37 +177,61 @@ Each table details the number of test cases that pass and fail for each test fil
}
});

new Chart(document.getElementById('summaryChart'),
{
type: 'bar',
data: {
labels: filteredRows.map(row => row[0]),
datasets: [{
data: filteredRows.map(row => row[1].score),
borderColor: filteredRows.map(row => implData.find(impl => impl.shortName === row[0]).color),
backgroundColor: filteredRows.map(row => implData.find(impl => impl.shortName === row[0]).color.replace('rgb', 'rgba').replace(')', ',0.2)')),
borderWidth: 1
}]
},
options: {
plugins: {
title: {
display: true,
text: 'Overall functional score (higher is better)'
const chartContainer = document.getElementById('complianceCharts');

const createChart = (id, column, title, field, suggestedMin) => {
const name = summaryData.headings[column];
const chartData = summaryData.rows
.filter(row => row[0] !== 'Jackson')
.filter(row => row[column][field] !== 0)
.toSorted((r1, r2) => r2[column][field] - r1[column][field]);
new Chart(document.getElementById(id),
{
type: 'bar',
data: {
labels: chartData.map(row => row[0]),
datasets: [{
data: chartData.map(row => row[column][field]),
borderColor: chartData.map(row => implData.find(impl => impl.shortName === row[0]).color),
backgroundColor: chartData.map(row => implData.find(impl => impl.shortName === row[0]).color.replace('rgb', 'rgba').replace(')', ',0.2)')),
borderWidth: 1
}]
},
options: {
plugins: {
title: {
display: true,
text: `[${name}] ${title}`
},
legend: {
display: false
}
},
legend: {
display: false
scales: {
y: {
beginAtZero: false,
suggestedMin: suggestedMin,
suggestedMax: 100
}
}
},
scales: {
y: {
beginAtZero: false,
suggestedMin: 80,
suggestedMax: 100
}
}
},
});
},
})
};

createChart('overallRequiredChart', 1, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('overallOptionalChart', 1, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
createChart('draft2020RequiredChart', 7, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('draft2020OptionalChart', 7, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
createChart('draft2019RequiredChart', 6, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('draft2019OptionalChart', 6, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
createChart('draft07RequiredChart', 5, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('draft07OptionalChart', 5, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
createChart('draft06RequiredChart', 4, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('draft06OptionalChart', 4, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
createChart('draft04RequiredChart', 3, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('draft04OptionalChart', 3, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
createChart('draft03RequiredChart', 2, 'REQUIRED tests pass percentage (higher is better)', 'requiredPassPct', 80);
createChart('draft03OptionalChart', 2, 'OPTIONAL tests pass percentage (higher is better)', 'optionalPassPct', 50);
</script>

[JSON-Schema-Test-Suite]: https://github.com/json-schema-org/JSON-Schema-Test-Suite
Expand Down