Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BIA-1352] - Adds support for Data Standard 5.0 #242

Merged
merged 60 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
a8e72df
Initial Commit. Adds new project for 5.0. Adds backup files for sql s…
DavidJGapCR Nov 8, 2023
86fdbd6
Fixes ContactPersonDim. Parent domain entity is now Contact
DavidJGapCR Nov 9, 2023
48d9e44
Fixes SectionDim. New table CourseAcademicSubject
DavidJGapCR Nov 13, 2023
48ff6a0
Fixes AcademicTimePeriodDim. GradingPeriodName instead of PeriodSeque…
DavidJGapCR Nov 13, 2023
4692b87
Fixes StudentSectionDim. New table CourseAcademicSubject.
DavidJGapCR Nov 13, 2023
6dc337e
Fixes fn_GetStudentGradesSummary. GradingPeriodName instead of Period…
DavidJGapCR Nov 13, 2023
3fae6bf
Uncommenting fn_GetStudentEnrollmentHistory and equity_StudentHistory…
DavidJGapCR Nov 13, 2023
8057f4e
Fixes ews_StudentEarlyWarningFact. StudentDisciplineIncidentBehaviorA…
DavidJGapCR Nov 13, 2023
43e724e
Fixes ews_StudentSectionGradeFact. GradingPeriodName instead of Gradi…
DavidJGapCR Nov 13, 2023
4638efd
Uncommenting views on Qews collection.
DavidJGapCR Nov 13, 2023
b281d68
Fixes on Postgres views. Getting started with fixing tests. AcademicT…
DavidJGapCR Nov 15, 2023
5daca44
Tests for StudentSchoolDim.
DavidJGapCR Nov 15, 2023
cfccda1
Tests for AllStudentSchoolDim.
DavidJGapCR Nov 15, 2023
04ebee7
Tests for AssessmentFact.
DavidJGapCR Nov 15, 2023
a27aafd
Tests for CandidateSurvey.
DavidJGapCR Nov 15, 2023
cf45956
Tests for ChronicAbsenteeismAttendanceFact
DavidJGapCR Nov 15, 2023
df46c13
Tests for ClassPeriodDim.
DavidJGapCR Nov 15, 2023
706d4f2
Tests for ContactPerson.
DavidJGapCR Nov 15, 2023
3db566d
Tests for DateDim
DavidJGapCR Nov 15, 2023
0d9000f
Tests for GradingPeriodDim.
DavidJGapCR Nov 15, 2023
163da88
Tests for LocalEducationAgencyDim.
DavidJGapCR Nov 15, 2023
5c48ac1
Tests for SchoolDim.
DavidJGapCR Nov 15, 2023
7188406
Tests for StudentProgramDim.
DavidJGapCR Nov 15, 2023
729ef47
Tests for SectionDim.
DavidJGapCR Nov 15, 2023
02b2598
Tests for StudentSectionDim.
DavidJGapCR Nov 15, 2023
464e658
Tests for DemographicDim.
DavidJGapCR Nov 16, 2023
e9e9f6c
Tests for EppDim.
DavidJGapCR Nov 16, 2023
148593a
Tests for EppSexDescriptorDim.
DavidJGapCR Nov 16, 2023
ae90dca
Tests for EvaluationElementRatingDim.
DavidJGapCR Nov 16, 2023
4daa469
Tests for FeederSchoolDim.
DavidJGapCR Nov 16, 2023
b0f27c7
Tests for FinancialAidFact
DavidJGapCR Nov 16, 2023
b31850f
Tests for RaceDescriptorDim.
DavidJGapCR Nov 16, 2023
6b2c1a7
Tests for StaffClassificationDescriptorScopeList.
DavidJGapCR Nov 16, 2023
5438989
Tests for StaffSectionDim.
DavidJGapCR Nov 16, 2023
a6a8a59
Tests for StudentAssessmentFact.
DavidJGapCR Nov 16, 2023
a9acaec
Tests for StudentDataAuthorization.
DavidJGapCR Nov 16, 2023
0fdc7e4
Tests for StudentLocalEducationAgencyDim.
DavidJGapCR Nov 16, 2023
567eed9
Tests for StudentProgramCohortDim.
DavidJGapCR Nov 16, 2023
f1c398a
Tests for StudentSchoolDemographicsBridge.
DavidJGapCR Nov 16, 2023
cba4e84
Tests for StudentSchoolFoodServiceProgramDim.
DavidJGapCR Nov 16, 2023
229972f
Tests for TermDescriptorDim.
DavidJGapCR Nov 16, 2023
2693df9
Tests for UserAuthorization.
DavidJGapCR Nov 16, 2023
53db06e
Tests for UserDim.
DavidJGapCR Nov 16, 2023
3f629ce
Tests for UserStudentDataAuthorization.
DavidJGapCR Nov 16, 2023
9a6c83c
Tests for StudentDisciplineActionDim.
DavidJGapCR Nov 16, 2023
4ea1f7d
Tests for StudentLocalEducationAgencyDemographicsBridge.
DavidJGapCR Nov 16, 2023
4c4220c
Tests for StudentEarlyWarningFact.
DavidJGapCR Nov 16, 2023
dfb982f
Tests for StudentHistoryDim.
DavidJGapCR Nov 16, 2023
e78295f
Tests for StudentHistoryDim.
DavidJGapCR Nov 16, 2023
0edb9db
Fixes on SectionDim tests.
DavidJGapCR Nov 16, 2023
905ea9e
Fixes on SectionDim tests for postgresql.
DavidJGapCR Nov 16, 2023
009e625
Fixes on StaffSectionDim tests.
DavidJGapCR Nov 16, 2023
4013b05
Fixes on StudentSectionDim tests.
DavidJGapCR Nov 16, 2023
79ce794
Engage Collection
DavidJGapCR Nov 16, 2023
9b184c2
Fixes on StudentLocalEducationAgencyDemographicsBridge tests.
DavidJGapCR Nov 17, 2023
3f55513
Fixes on StudentSectionDim tests.
DavidJGapCR Nov 20, 2023
429fcd4
Fixes on Operation tests.
DavidJGapCR Nov 20, 2023
a1ca28d
CI CD
DavidJGapCR Nov 20, 2023
c6364d5
Update target framework for new project.
DavidJGapCR Nov 20, 2023
4d3c260
Trying to fix actions.
DavidJGapCR Nov 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
9 changes: 9 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,15 @@ env:
SQLSERVER_DATABASE_DS32: "AnalyticsMiddleTier_Testing_Ds32"
SQLSERVER_DATABASE_DS33: "AnalyticsMiddleTier_Testing_Ds33"
SQLSERVER_DATABASE_DS40: "AnalyticsMiddleTier_Testing_Ds40"
SQLSERVER_DATABASE_DS50: "AnalyticsMiddleTier_Testing_Ds50"
SQLSERVER_INTEGRATED_SECURITY: "false"
SQLSERVER_USER: "sa"
SQLSERVER_PASS: ${{ secrets.MSSQL_PWD }}
POSTGRES_HOST: "localhost"
POSTGRES_DATABASE_DS32: "edfi_ods_tests_ds32"
POSTGRES_DATABASE_DS33: "edfi_ods_tests_ds33"
POSTGRES_DATABASE_DS40: "edfi_ods_tests_ds40"
POSTGRES_DATABASE_DS50: "edfi_ods_tests_ds50"
POSTGRES_PORT: "5432"
POSTGRES_USER: "postgres"
POSTGRES_PASS: ${{ secrets.POSTGRESQL_PWD }}
Expand Down Expand Up @@ -140,6 +142,13 @@ jobs:
env:
PGPASSWORD: ${{ env.POSTGRES_PASS }}

- name: Create Postgres DB DS v5.0
run: |
.\eng\CreateTestDbAndSchema.ps1 -port ${{ env.POSTGRES_PORT }} -d ${{ env.POSTGRES_DATABASE_DS50 }} -ds 5.0
shell: pwsh
env:
PGPASSWORD: ${{ env.POSTGRES_PASS }}

- name: Building AMT
run: |
.\build.ps1 build -Configuration ${{ env.CONFIGURATION }} -Version ${{ env.BUILD_VERSION }} -BuildCounter ${{ github.run_number }}
Expand Down
6 changes: 5 additions & 1 deletion eng/CreateTestDbAndSchema.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ param ( [string][Alias('h')]$pghost="localhost",
[string][Alias('u')]$user="postgres",
[string][Alias('p')]$port="5432",
[string][Alias('ds')]
[ValidateSet("3.2", "3.3", "4.0")]
[ValidateSet("3.2", "3.3", "4.0", "5.0")]
$datastandard,
[string][Alias('d')]$database="edfi_ods_tests_ds32",
[string][Alias('s')]$script="$PSScriptRoot/../src/EdFi.AnalyticsMiddleTier.Tests/EdFi.Ods32.Minimal.Template.sql")
Expand All @@ -88,6 +88,10 @@ elseif($datastandard -eq "4.0"){
$script="$PSScriptRoot/../src/EdFi.AnalyticsMiddleTier.Tests/EdFi.Ods40.Minimal.Template.sql"
$database="edfi_ods_tests_ds40"
}
elseif($datastandard -eq "5.0"){
$script="$PSScriptRoot/../src/EdFi.AnalyticsMiddleTier.Tests/EdFi.Ods50.Minimal.Template.sql"
$database="edfi_ods_tests_ds50"
}

$dropDatabase = "DROP DATABASE IF EXISTS" +" " + $database +";"
$createDatabase = "CREATE DATABASE" +" " + $database +";"
Expand Down
3 changes: 2 additions & 1 deletion src/EdFi.AnalyticsMiddleTier.Common/Component.cs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ public enum DataStandard
Ds31,
Ds32,
Ds33,
Ds40
Ds40,
Ds50
}
[SuppressMessage("ReSharper", "InconsistentNaming")]
public enum Engine
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ public class PostgresMigrationStrategy : IDatabaseMigrationStrategy
public const string JournalingVersionsTable = "AnalyticsMiddleTierSchemaVersion";
public const string DataStandardVersionTemplate =
@"SELECT CASE
WHEN EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'edfi' AND table_name = 'contact'
)
THEN
'Ds50'
WHEN EXISTS (
SELECT FROM information_schema.tables
WHERE table_schema = 'edfi' AND table_name = 'localbudget'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,16 @@ public class SqlServerMigrationStrategy : IDatabaseMigrationStrategy
ELSE IF (SELECT OBJECT_ID('dbo.VersionLevel')) IS NOT NULL
BEGIN
SELECT 'Ds31' AS version
END
ELSE IF (SELECT OBJECT_ID('edfi.Contact')) IS NOT NULL
BEGIN
SELECT 'Ds50' AS version
END
ELSE IF (SELECT OBJECT_ID('edfi.LocalBudget')) IS NOT NULL
BEGIN
SELECT 'Ds40' AS version
END

ELSE IF (SELECT OBJECT_ID('edfi.Survey')) IS NOT NULL
BEGIN
SELECT 'Ds33' AS version
Expand All @@ -38,7 +43,8 @@ ELSE IF (SELECT OBJECT_ID('dbo.DeployJournal')) IS NOT NULL
ELSE
BEGIN
SELECT 'InvalidDs' AS version
END";
END
";

private readonly IOrm _orm;
public SqlServerMigrationStrategy(IOrm orm){
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
<ProjectReference Include="..\EdFi.AnalyticsMiddleTier.DataStandard32\EdFi.AnalyticsMiddleTier.DataStandard32.csproj" />
<ProjectReference Include="..\EdFi.AnalyticsMiddleTier.DataStandard33\EdFi.AnalyticsMiddleTier.DataStandard33.csproj" />
<ProjectReference Include="..\EdFi.AnalyticsMiddleTier.DataStandard40\EdFi.AnalyticsMiddleTier.DataStandard40.csproj" />
<ProjectReference Include="..\EdFi.AnalyticsMiddleTier.DataStandard50\EdFi.AnalyticsMiddleTier.DataStandard50.csproj" />
</ItemGroup>

<ItemGroup>
Expand Down
4 changes: 4 additions & 0 deletions src/EdFi.AnalyticsMiddleTier.Console/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
using Ds32 = EdFi.AnalyticsMiddleTier.DataStandard32;
using Ds33 = EdFi.AnalyticsMiddleTier.DataStandard33;
using Ds40 = EdFi.AnalyticsMiddleTier.DataStandard40;
using Ds50 = EdFi.AnalyticsMiddleTier.DataStandard50;
using System.Text.RegularExpressions;

namespace EdFi.AnalyticsMiddleTier.Console
Expand Down Expand Up @@ -143,6 +144,9 @@ void RunWithOptions(Options options)
case DataStandard.Ds40:
install = new Ds40.Install(migrationStrategy);
break;
case DataStandard.Ds50:
install = new Ds50.Install(migrationStrategy);
break;
default:
message = _odsVersionNotSupportedMessage;
break;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
-- SPDX-License-Identifier: Apache-2.0
-- Licensed to the Ed-Fi Alliance under one or more agreements.
-- The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0.
-- See the LICENSE and NOTICES files in the project root for more information.

CREATE VIEW analytics.asmt_AssessmentFact
AS
SELECT CONCAT (
Assessment.AssessmentIdentifier
,'-'
,Assessment.Namespace
,'-'
,AssessmentAssessedGradeLevel.GradeLevelDescriptorId
,'-'
,AssessmentScore.AssessmentReportingMethodDescriptorId
,'-'
,AssessmentAcademicSubject.AcademicSubjectDescriptorId
,'-'
,ObjectiveAssessment.IdentificationCode
,'-'
,ObjectiveAssessment.ParentIdentificationCode
,'-'
,ObjectiveAssessmentScore.AssessmentReportingMethodDescriptorId
,'-'
,ObjectiveAssessmentLearningStandard.LearningStandardId
) AS AssessmentFactKey
,CONCAT (
Assessment.AssessmentIdentifier
,'-'
,Assessment.Namespace
) AS AssessmentKey
,Assessment.AssessmentIdentifier
,Assessment.Namespace
,Assessment.AssessmentTitle AS Title
,COALESCE(Assessment.AssessmentVersion, 0) AS Version
,COALESCE(CategoryDescriptor.Description, '') AS Category
,COALESCE(Descriptor.Description, '') AS AssessedGradeLevel
,COALESCE(AcademicSubjectDescriptor.Description, '') AS AcademicSubject
,COALESCE(ResultDataTypeDescriptorAssessment.Description, ResultDataTypeDescriptorObjectiveAssessment.Description, '') AS ResultDataType
,COALESCE(AssessmentReportingMethodDescriptorDist.Description, ReportingMethodDescriptorObjectiveAssessment.Description, '') AS ReportingMethod
,CASE
WHEN ObjectiveAssessment.AssessmentIdentifier IS NOT NULL
THEN CONCAT (
ObjectiveAssessment.AssessmentIdentifier
,'-'
,ObjectiveAssessment.IdentificationCode
,'-'
,ObjectiveAssessment.Namespace
)
ELSE ''
END AS ObjectiveAssessmentKey
,COALESCE(ObjectiveAssessment.IdentificationCode, '') AS IdentificationCode
,CASE
WHEN ObjectiveAssessment.ParentIdentificationCode IS NOT NULL
THEN CONCAT (
ParentAssesment.AssessmentIdentifier
,'-'
,ParentAssesment.IdentificationCode
,'-'
,ParentAssesment.Namespace
)
ELSE ''
END AS ParentObjectiveAssessmentKey
,COALESCE(ObjectiveAssessment.Description, '') AS ObjectiveAssessmentDescription
,COALESCE(ObjectiveAssessment.PercentOfAssessment, 0) AS PercentOfAssessment
,COALESCE(AssessmentScore.MinimumScore, ObjectiveAssessmentScore.MinimumScore, '') AS MinScore
,COALESCE(AssessmentScore.MaximumScore, ObjectiveAssessmentScore.MaximumScore, '') AS MaxScore
,COALESCE(ObjectiveAssessmentLearningStandard.LearningStandardId, '') AS LearningStandard
FROM edfi.Assessment
LEFT JOIN edfi.AssessmentAssessedGradeLevel
ON Assessment.AssessmentIdentifier = AssessmentAssessedGradeLevel.AssessmentIdentifier
AND Assessment.Namespace = AssessmentAssessedGradeLevel.Namespace
LEFT JOIN edfi.AssessmentScore
ON Assessment.AssessmentIdentifier = AssessmentScore.AssessmentIdentifier
AND Assessment.Namespace = AssessmentScore.Namespace
LEFT JOIN edfi.Descriptor AS AssessmentReportingMethodDescriptorDist
ON AssessmentScore.AssessmentReportingMethodDescriptorId = AssessmentReportingMethodDescriptorDist.DescriptorId
LEFT JOIN edfi.Descriptor
ON Descriptor.DescriptorId = AssessmentAssessedGradeLevel.GradeLevelDescriptorId
LEFT JOIN edfi.AssessmentAcademicSubject
ON Assessment.AssessmentIdentifier = AssessmentAcademicSubject.AssessmentIdentifier
AND Assessment.Namespace = AssessmentAcademicSubject.Namespace
LEFT JOIN edfi.Descriptor AcademicSubjectDescriptor
ON AcademicSubjectDescriptor.DescriptorId = AssessmentAcademicSubject.AcademicSubjectDescriptorId
LEFT JOIN edfi.Descriptor CategoryDescriptor
ON CategoryDescriptor.DescriptorId = Assessment.AssessmentCategoryDescriptorId
LEFT JOIN edfi.Descriptor AS ResultDataTypeDescriptorAssessment
ON AssessmentScore.ResultDatatypeTypeDescriptorId = ResultDataTypeDescriptorAssessment.DescriptorId
LEFT JOIN edfi.ObjectiveAssessment
ON Assessment.AssessmentIdentifier = ObjectiveAssessment.AssessmentIdentifier
AND Assessment.Namespace = ObjectiveAssessment.Namespace
LEFT JOIN edfi.ObjectiveAssessmentScore
ON ObjectiveAssessment.AssessmentIdentifier = ObjectiveAssessmentScore.AssessmentIdentifier
AND ObjectiveAssessment.IdentificationCode = ObjectiveAssessmentScore.IdentificationCode
AND ObjectiveAssessment.Namespace = ObjectiveAssessmentScore.Namespace
LEFT JOIN edfi.Descriptor AS ResultDataTypeDescriptorObjectiveAssessment
ON ObjectiveAssessmentScore.ResultDatatypeTypeDescriptorId = ResultDataTypeDescriptorObjectiveAssessment.DescriptorId
LEFT JOIN edfi.Descriptor AS ReportingMethodDescriptorObjectiveAssessment
ON ObjectiveAssessmentScore.AssessmentReportingMethodDescriptorId = ReportingMethodDescriptorObjectiveAssessment.DescriptorId
LEFT JOIN edfi.ObjectiveAssessment ParentAssesment
ON ParentAssesment.AssessmentIdentifier = ObjectiveAssessment.AssessmentIdentifier
AND ParentAssesment.IdentificationCode = ObjectiveAssessment.ParentIdentificationCode
AND ParentAssesment.Namespace = ObjectiveAssessment.Namespace
LEFT JOIN edfi.ObjectiveAssessmentLearningStandard
ON ObjectiveAssessment.AssessmentIdentifier = ObjectiveAssessmentLearningStandard.AssessmentIdentifier
AND ObjectiveAssessment.IdentificationCode = ObjectiveAssessmentLearningStandard.IdentificationCode
AND ObjectiveAssessment.Namespace = ObjectiveAssessmentLearningStandard.Namespace;
Loading