Merge branch 'clustering' into spatial_study

.gitignore

@@ -1,3 +1,4 @@
+Logs
 src/Data/*
 src/Output/*
 !src/Data/IncFilePreSuf/README.md
@@ -7,9 +8,13 @@ __pycache__
 .new_env/*
 prebalm
 *.pyc
-snakemake_dag
-snakemake_dag.*
-!snakemake_dag.pdf
+
+clusteroutput
+clustering_dot_source.txt
+clustering_dag
+preprocessing_dot_source.txt
+preprocessing_dag
+
 gurobi.log
 
 exo_elec_dem_conversion_dictionaries.pkl

README.md

@@ -1,6 +1,8 @@
 # Balmorel Pre-Processing
 
-These scripts process raw data into Balmorel input. An application presented at the EGU24 conference is illustrated in the poster below (check Zenodo link below for a high-res pdf). For more info, read README in the src folder.
+These scripts process raw data into Balmorel input for a 2050 greenfield model of the Danish energy system at municipal spatial resolution (99 nodes). 
+
+An application presented at the EGU24 conference is illustrated in the poster below (check Zenodo link below for a high-res pdf). For more info, read README in the src folder.
 ![Application example](https://github.com/Mathias157/balmorel-preprocessing/blob/master/Raw%20Data%20Processing/Conference%20Poster%20for%20Analysis%20of%20Spatial%20Resolutions%20for%20Modelling%20Sector-Coupled%20Energy%20Systems.png)
 
 Data can be downloaded in the Zenodo link below and should be placed in src/
@@ -13,4 +15,46 @@ The necessary python packages can be installed in a virtual environment by follo
 
 ```` 
 conda env create -f environment.yaml
-````
+````
+
+This requires [anaconda](https://www.anaconda.com/download?utm_source=anacondadoc&utm_medium=documentation&utm_campaign=download&utm_content=topnavalldocs) (or the more lightweight [miniconda](https://docs.anaconda.com/miniconda/#miniconda-latest-installer-links)) installs an environment with the following packages:
+````
+name: spatialstudy
+channels:
+  - conda-forge
+  - bioconda
+dependencies:
+  - Cartopy=0.23.0
+  - geopandas=1.0.1
+  - matplotlib=3.9.2
+  - ipywidgets=8.1.3
+  - nbformat=5.9.2
+  - numpy=1.26.2
+  - openpyxl=3.1.2
+  - pandas=2.1.4
+  - plotly=5.16.1
+  - pyarrow=15.0.1
+  - pyproj=3.6.1
+  - scipy=1.11.2
+  - shapely=2.0.2
+  - atlite=0.2.12
+  - xarray=2024.2.0
+  - xlrd=2.0.1
+  - graphviz
+  - python-graphviz
+  - click
+  - snakemake-minimal
+  - pip
+  - pip:
+    - gamsapi[transfer]==45.7.0
+    - pybalmorel==0.3.10
+````
+
+## Get Started
+
+The processing is initiated through a snakemake command in a command-line interface in the src directory. If using windows, the pre-processing can be run by calling `preprocessing` and the clustering through `clustering`. If on other systems, do either of the following commands:
+```
+snakemake -s workflow/preprocessing
+snakemake -s workflow/clustering
+```
+A plot of the processes can be found [here](src/Analysis/preprocessing_dag.pdf). Note that vre_profiles.py and existing_powerplants is currently not part of the snakemake process, but the output from vre_profiles.py is used in the offshore_wind process. 

environment.yaml

@@ -26,4 +26,4 @@ dependencies:
   - pip
   - pip:
     - gamsapi[transfer]==45.7.0
-    - pybalmorel==0.3.10
+    - pybalmorel==0.3.11

src/Analysis/plot_dag.py

@@ -3,17 +3,27 @@
 https://graphviz.org/download/
 """
 
+import click
 from graphviz import Source
 
-# Do snakemake --dag > analysis/dot_source.txt to within the dot_source below
-with open('Analysis/dot_source.txt', 'r') as f:
-    dot_source = f.read()
+@click.command()
+@click.option('--workflow', type=str, required=True, help="Which workflow to plot")
+@click.option('--view', type=bool, required=False, help="View the plotted DAG?")
+def main(workflow: str, view: bool = False):
+    # Do snakemake --dag > analysis/dot_source.txt to within the dot_source below
+    with open('Analysis/%s_dot_source.txt'%workflow, 'r') as f:
+        dot_source = f.read()
 
-# Create a Source object from the DOT source
-graph = Source(dot_source)
+    # Create a Source object from the DOT source
+    graph = Source(dot_source)
 
-# Render the diagram as a PNG file
-graph.render('Analysis/snakemake_dag', format='svg', cleanup=False)
+    # Render the diagram as a PNG file
+    graph.render('Analysis/%s_dag'%workflow, format='pdf', cleanup=False)
 
-# Optionally view the diagram (depends on your environment)
-graph.view()
+    if view:
+        # Optionally view the diagram (depends on your environment)
+        graph.view()
+
+
+if __name__ == '__main__':
+    main()