Merge pull request #550 from DFE-Digital/adr-private-gias

Update 0013-private-gias-trust-contacts-data.md

docs/adrs/0013-private-gias-trust-contacts-data.md

@@ -12,7 +12,11 @@ Currently, the private GIAS (Get Information About Schools) data, which includes
 
 At present, the private GIAS data is received weekly in a spreadsheet. An individual manually inputs this data into KIM, from which it is pipelined into the Academies DB. FIAT then retrieves the data from Academies DB. However, due to the limitations of KIM, we are currently missing emails for CFOs and Chairs of Trustees in FIAT, creating gaps in our data.
 
-To resolve this, we have decided to bypass KIM for this specific dataset. The weekly GIAS spreadsheet will be fed directly into the Academies DB, populating a new table called `TadTrustGovernance`. This ensures that all required email addresses (including those of CFOs and Chairs of Trustees) will be captured accurately. The `MstrTrustGovernance` table, which sources its data from KIM, will no longer be used for this purpose. The data engineers on the programme will take responsibility for managing this ingestion process.
+To resolve this, we have decided to bypass KIM for this specific dataset. The weekly GIAS spreadsheet will be fed directly into the Academies DB, populating a new table called `TadTrustGovernance`. This ensures that all required email addresses (including those of CFOs and Chairs of Trustees) will be captured accurately. The `MstrTrustGovernance` table, which sources its data from KIM, will no longer be used for this purpose. 
+
+As an interim solution, a member of the data team will manually download the GIAS file and pass it to the data engineer who will load it into the `TadTrustGovernance` table. In the future, this process is intended to be automated, where the file will be saved in a designated storage location and automatically loaded into the appropriate table by a data pipeline. At that point, the data will be moved into a different schema (`gias-nonpublic`), as the `tad` schema is temporary and will be phased out.
+
+The data engineers on the program will take responsibility for managing this ingestion process until the automated pipeline is in place.
 
 This decision provides us with up-to-date and complete information but introduces an overhead for the data engineers who will now manage this process manually. Mid- to long-term, this process will need to be reviewed as KIM is decommissioned, and an alternative data propagation method is identified.
 
@@ -21,14 +25,17 @@ This decision provides us with up-to-date and complete information but introduce
 We will:
 
 - Stop relying on KIM for capturing emails for CFOs and Chairs of Trustees.
-- Feed the private GIAS data spreadsheet directly into the Academies DB to populate the `TadTrustGovernance` table.
+- Feed the private GIAS data spreadsheet directly into the Academies DB to populate the `TadTrustGovernance` table temporarily.
 - Discontinue the use of the `MstrTrustGovernance` table for email retrieval.
-- Task the data engineers with managing this weekly ingestion process.
+- Implement a manual download and data loading process as an interim measure, with a view to automate this in the future.
+- Move the data ingestion process to the `gias-nonpublic` schema once the automated pipeline is in place, and adjust FIAT to retrieve from the new schema accordingly.
+- Task the data engineers with managing this weekly ingestion process in the interim.
 
 This decision is made to ensure that FIAT has access to complete and accurate contact information for the Accounting Officer, CFO, and Chair of Trustees. The current method via KIM is no longer sufficient, given its limitations in capturing emails for non-Accounting Officer roles.
 
 ## Consequences  
 
 - We will have full access to the required email addresses, ensuring FIAT serves accurate contact data.
-- The data engineers will have additional responsibilities in managing the weekly ingestion process, which introduces an overhead.
-- This approach ensures immediate resolution of the missing data issue but may need to be revisited as KIM is decommissioned in the future. At that point, alternative methods for propagating the trust governance data will be explored.
+- The data engineers will have additional responsibilities in managing the weekly ingestion process until automation is implemented.
+- The `tad` schema will be used temporarily, with a plan to transition the data ingestion to the `gias-nonpublic` schema in the long term.
+- This approach ensures an immediate resolution of the missing data issue but may need to be revisited as KIM is decommissioned and alternative methods for propagating trust governance data are explored.