AGV reproducible parse_crossings

docs/workflow/level-controlled.qmd

@@ -6,17 +6,17 @@ Workflow for deriving level controlled (peilbeheerst) regional models.
 All code can be found under [`src/peilbeheerst_model`](https://github.com/Deltares/Ribasim-NL/tree/main/src/peilbeheerst_model).
 The paths below are relative to this path.
 
-1. Run the preprocessing notebooks. One notebook per water board, path: `peilbeheerst_model/preprocess_data/`
-2. Run the postprocessing notebook. One notebook per water board, path: `peilbeheerst_model/postprocess_data/`
-3. Run the crossings notebook. One notebook, path: `01_parse_crossings.ipynb`
-4. Run shortest paths notebooks. One notebook per water board, path: `Shortest_path/`
-5. Run crossings to Ribasim notebook. One notebook, all water boards are below each other, path: `02_crossings_to_ribasim_notebook.ipynb`
-6. Run parametrize notebooks. One notebook per water board, for now only Amstel, Gooi en Vecht (AGV), path: `Parametrize/AmstelGooienVecht_parametrize.ipynb`
+1. Run the preprocessing scripts. One script per water board, path: `peilbeheerst_model/preprocess_data/`
+2. Run the postprocessing script. One script per water board, path: `peilbeheerst_model/postprocess_data/`
+3. Run the crossings script. One script, path: `01_parse_crossings.ipynb`. Moving to one script per water board under `peilbeheerst_model/parse_crossings/`.
+4. Run shortest paths scripts. One script per water board, path: `Shortest_path/`
+5. Run crossings to Ribasim script. One script, all water boards are below each other, path: `02_crossings_to_ribasim_notebook.ipynb`. Moving to one script per water board under `peilbeheerst_model/crossings_to_ribasim/`.
+6. Run parametrize scripts. One script per water board, for now only Amstel, Gooi en Vecht (AGV), path: `Parametrize/AmstelGooienVecht_parametrize.ipynb`
 
-We originally had more parametrize notebooks, but because so much has changed I have now saved these in our backup.
+We originally had more parametrize scripts, but because so much has changed I have now saved these in our backup.
 We will only use these to see if there were any additional manual adjustments.
 For the rest, it follows the same workflow as `AmstelGooienVecht_parametrize.ipynb`.
 
-Finally: step 1 started with a clear notebook per water board.
-During the process of 1.5 years, these notebooks have become increasingly larger and more confusing, whereby not every line is needed anymore.
+Finally: step 1 started with a clear script per water board.
+During the process of 1.5 years, these scripts have become increasingly larger and more confusing, whereby not every line is needed anymore.
 For now, there is no priority to clean this up, partly because this is a major risk that the data will (unintentionally) change, which will change the networks and the feedback forms can no longer be used.

src/peilbeheerst_model/01_parse_crossings.py

@@ -25,8 +25,9 @@
 for waterschap, waterschap_struct in waterschap_data.items():
     print(f"\n{waterschap}...")
 
-    init_settings, crossing_settings = waterschap_struct.values()
-    init_settings["logfile"] = pathlib.Path(init_settings["output_path"]).with_suffix("").with_suffix(".log")
+    crossing_settings = waterschap_struct["find_crossings_with_peilgebieden"]
+    init_settings = waterschap_struct["init"]
+    init_settings["logfile"] = pathlib.Path(init_settings["output_path"]).with_suffix(".log")
 
     if waterschap not in ["HHNK"]:
         continue

src/peilbeheerst_model/01b_ad_krw_to_peilgebieden.py

@@ -21,7 +21,9 @@
 for waterschap, waterschap_struct in waterschap_data.items():
     print(f"\n{waterschap}...")
 
-    init_settings, crossing_settings = waterschap_struct.values()
+    crossing_settings = waterschap_struct["find_crossings_with_peilgebieden"]
+    init_settings = waterschap_struct["init"]
+
     gpkg = pathlib.Path(init_settings["output_path"])
     if not gpkg.exists():
         raise ValueError(gpkg)

src/peilbeheerst_model/parse_crossings/AmstelGooienVecht.py

@@ -0,0 +1,38 @@
+# %%
+
+from peilbeheerst_model import ParseCrossings, waterschap_data
+from ribasim_nl import CloudStorage
+
+# %%
+waterschap = "AmstelGooienVecht"
+waterschap_struct = waterschap_data[waterschap]
+
+cloud = CloudStorage()
+verwerkt_dir = cloud.joinpath(waterschap, "verwerkt")
+cloud.download_verwerkt(waterschap)
+cloud.download_basisgegevens()
+
+# %%
+
+crossing_settings = waterschap_struct["find_crossings_with_peilgebieden"]
+init_settings = waterschap_struct["init"]
+
+init_settings["gpkg_path"] = verwerkt_dir / "postprocessed.gpkg"
+init_settings["krw_path"] = cloud.joinpath("Basisgegevens/KRW/KRW_lichamen_per_waterschap.gpkg")
+init_settings["output_path"] = verwerkt_dir / "crossings.gpkg"
+init_settings["logfile"] = verwerkt_dir / "crossings.log"
+
+# Crossings class initializeren
+cross = ParseCrossings(**init_settings)
+
+# Crossings bepalen en wegschrijven
+if crossing_settings["filterlayer"] is None:
+    df_hydro = cross.find_crossings_with_peilgebieden("hydroobject", **crossing_settings)
+    cross.write_crossings(df_hydro)
+else:
+    df_hydro, df_dsf, df_hydro_dsf = cross.find_crossings_with_peilgebieden("hydroobject", **crossing_settings)
+    cross.write_crossings(df_hydro, crossing_settings["filterlayer"], df_dsf, df_hydro_dsf)
+
+# %%
+
+cloud.upload_verwerkt(waterschap)