Feature request to compare what has changed in an asssessment file format

[dpsjallen]

If we need to load a previous year, current year, and historical for the same assessment but they each may have slightly different formats how can a bundle accommodate this?

[ejoranlienea]

For minor changes a single bundle can be made compatible across multiple versions. The NWEA MAP bundle does this: columns that have been renamed in the last few years are included in the bundle and mapped to the newest name. See for example course here -- it used to be called Measurement Scale, so either of these columns will be loaded as course by the MAP bundle, allowing it to handle files from before or after this change consistently.

This leaves a few issues unaddressed however:

1. If one of these columns were to change names again, what should the updated bundle do? It could still map it to the name that was current at the time the bundle was made, but that's odd. It could map all prior versions of the column to the new name, but then historic assessments will use different naming conventions depending on when they were loaded, or you have to reload old assessments to match the new naming convention.
2. More significant year-over-year changes that are harder to address this way. This could be handled by bifurcating into two templates, but this can become unwieldy to manage.
3. The same problem, but for changes to the Ed-Fi standard. These are less frequent, but in the worst case scenario you have an assessment format x ed-fi version matrix to contend with.

The Data Import Tool doesn't handle this particularly well either -- there are different templates that are marked with years, and I suppose you're meant to assume that if there's a 2018 and 2020 version, you should use 2018 for 2018 - 2019 and the 2020 version for 2020 - 2024, but this isn't really specified. There are also sometimes versions just marked 'V2', which is not terribly instructive.

We have a few lines of thinking going that could help address some of these issues.
One is a way to layer/import bundles, such that you can have layers of functionality, adding customization on top of a base bundle. This could help with tweaking or extending a base template without wholesale duplication.
Another is a metadata/packaging service that could track bundle compatibility with assessment file formats and Ed-Fi versions, so in principle you could declare an import of the (NWEA MAP, 2019 format, Ed-Fi 6.1, CO state extensions) and get the right thing (assuming it exists).

In the future more vendors will support direct integrations, potentially making some of these bundles obsolete. This has pros and cons. Pro: the vendor can support Ed-Fi compatibility more directly without passing through the middle layer of a file extract format. Cons: you're subject to the decisions they make about data mapping, version support, error handling, historic data loading, etc.