Merge pull request #210 from ktehranchi/master

Update Docs for Opts

.gitignore

@@ -268,4 +268,3 @@ connect.sh
 config/config.cluster.yaml
 /workflow/repo_data/dag.png
 !.pre-commit-config.yaml
-

.test_sh

@@ -1,4 +1,4 @@
-# SPDX-FileCopyrightText: : 2021-2024 The PyPSA-Eur Authors 
+# SPDX-FileCopyrightText: : 2021-2024 The PyPSA-Eur Authors
 # ADAPTED FOR PyPSA-USA
 # SPDX-License-Identifier: CC0-1.0
 

docs/source/config-wildcards.md

@@ -26,24 +26,6 @@ A visual representation of each `{interconnect}` is shown below:
     :scale: 100 %
 ```
 
-(cutout_wc)=
-## The `{cutout}` wildcard
-
-The `{cutout}` wildcard facilitates running the rule :mod:`build_cutout`
-for all cutout configurations specified under `atlite: cutouts:`. Each cutout
-is descibed in the form `{dataset}_{year}`. These cutouts will be stored in a
-folder specified by `{cutout}`.
-
-Valid dataset names include: `era5`
-Valid years can be from `1940` to `2022`
-
-```{note}
-Data for `era5_2019` has been pre-pared for the user and will be automatically downloaded
-during the workflow. If other years are needed, the user will need to prepaer the
-cutout themself.
-```
-
-
 <!-- (simpl)=
 ## The ``{simpl}`` wildcard -->
 
@@ -104,6 +86,7 @@ currently:
    :file: configtables/opts.csv
 ```
 
+
 (sector)=
 ## The `{sector}` wildcard
 
@@ -127,3 +110,21 @@ Used in the following rules:
 - `build_heat_demands`
 - `build_temperature_profiles`
 - `build_solar_thermal_profiles`
+
+
+(cutout_wc)=
+## The `{cutout}` wildcard
+
+The `{cutout}` wildcard facilitates running the rule :mod:`build_cutout`
+for all cutout configurations specified under `atlite: cutouts:`. Each cutout
+is descibed in the form `{dataset}_{year}`. These cutouts will be stored in a
+folder specified by `{cutout}`.
+
+Valid dataset names include: `era5`
+Valid years can be from `1940` to `2022`
+
+```{note}
+Data for `era5_2019` has been pre-pared for the user and will be automatically downloaded
+during the workflow. If other years are needed, the user will need to prepaer the
+cutout themself.
+```

docs/source/configtables/opts.csv

@@ -1,14 +1,15 @@
-Trigger, Description, Definition, Status,
-``nH``; i.e. ``2H``-``6H``, Resample the time-resolution by averaging over every ``n`` snapshots, ``prepare_network``: `average_every_nhours() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L110>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L146>`__), In active use,
-``nSEG``; e.g. ``4380SEG``," ""Apply time series segmentation with `tsam <https://tsam.readthedocs.io/en/latest/index.html>`_ package to ``n`` adjacent snapshots of varying lengths based on capacity factors of varying renewables"," hydro inflow and load.""", ``prepare_network``: apply_time_segmentation(), In active use
-``Co2L``, Add an overall absolute carbon-dioxide emissions limit configured in ``electricity: co2limit``. If a float is appended an overall emission limit relative to the emission level given in ``electricity: co2base`` is added (e.g. ``Co2L0.05`` limits emissisions to 5% of what is given in ``electricity: co2base``), ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`__, In active use,
-``RCo2L``, Add regional carbon-dioxide emissions limits configured in ``electricity: regional_Co2_limits``. These can be specified in the file linked at ``electricity: regional_Co2_limits`` in the configuration. File defaults to ``repo_data/regional_co2_limits.csv``., ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`__, In active use,
-``RPS``,"Add Renewable Portfolio Standard (RPS) constraints to shares of generator production (MWh) per carrier for individual countries. Standards can be set for single carriers or groups of carriers. Each constraint can be designated for a specified planning horizon in multi-period models. Opts and path for portfolio_standards.csv must be defined", ``solve_network``, In active use,
-``Ep``, Add cost for a carbon-dioxide price configured in ``costs: emission_prices: co2`` to ``marginal_cost`` of generators (other emission types listed in ``network.carriers`` possible as well), ``prepare_network``: `add_emission_prices() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L24>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L158>`__, In active use,
-``CCL``,"Add minimum or maximum levels of generator nominal capacity per carrier for individual countries. Each constraint can be designated for a specified planning horizon in multi-period models. Opts and path for agg_p_nom_minmax.csv must be defined", ``solve_network``, In active use,
-``EQ``," ""Require each country or node to on average produce a minimal share of its total consumption itself. Example: ``EQ0.5c`` demands each country to produce on average at least 50% of its consumption; ``EQ0.5`` demands each node to produce on average at least 50% of its consumption.""", ``solve_network``, In active use,
-``ATK``," ""Require each node to be autarkic. Example: ``ATK`` removes all lines and links. ``ATKc`` removes all cross-border lines and links.""", ``prepare_network``, In active use,
-``BAU``, Add a per-``carrier`` minimal overall capacity; i.e. at least ``40GW`` of ``OCGT`` in Europe; configured in ``electricity: BAU_mincapacities``, ``solve_network``: `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L66>`__, Untested,
-``SAFE``, Add a capacity reserve margin of a certain fraction above the peak demand to which renewable generators and storage do *not* contribute. Ignores network., ``solve_network`` `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L73>`__, Untested,
-``carrier+{c|p|m}factor``,"Alter the capital cost (``c``), installable potential (``p``) or marginal costs (``m``) of a carrier by a factor. Example: ``solar+c0.5`` reduces the capital cost of solar to 50\% of original values.", ``prepare_network``, In active use,
-``CH4L``,"Add an overall absolute gas limit. If configured in ``electricity: gaslimit`` it is given in MWh thermal, if a float is appended, the overall gaslimit is assumed to be given in TWh thermal (e.g. ``CH4L200`` limits gas dispatch to 200 TWh termal)", ``prepare_network``: ``add_gaslimit()``, In active use,
+Trigger, Description, Definition, Status
+``nH``; i.e. ``2H``-``6H``, Resample the time-resolution by averaging over every ``n`` snapshots, ``prepare_network``: `average_every_nhours() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L110>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L146>`__), In active use
+``nSEG``; e.g. ``4380SEG``, Apply time series segmentation with `tsam <https://tsam.readthedocs.io/en/latest/index.html>`_ package to ``n`` adjacent snapshots of varying lengths based on capacity factors of varying renewables  hydro inflow and load., ``prepare_network``: apply_time_segmentation(),In active use
+``Co2L``, Add an overall absolute carbon-dioxide emissions limit configured in ``electricity: co2limit``. If a float is appended an overall emission limit relative to the emission level given in ``electricity: co2base`` is added (e.g. ``Co2L0.05`` limits emissisions to 5% of what is given in ``electricity: co2base``), ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`__, In active use
+``RCo2L``, Add regional carbon-dioxide emissions limits configured in ``electricity: regional_Co2_limits``. These can be specified in the file linked at ``electricity: regional_Co2_limits`` in the configuration. File defaults to ``repo_data/regional_co2_limits.csv``., ``prepare_network``: `add_co2limit() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L19>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L154>`__, In active use
+``RPS``,Add Renewable Portfolio Standard (RPS) constraints to shares of generator production (MWh) per carrier for individual countries. Standards can be set for single carriers or groups of carriers. Each constraint can be designated for a specified planning horizon in multi-period models. Opts and path for portfolio_standards.csv must be defined, ``solve_network``, In active use
+``SAFE``, Add a capacity reserve margin of a certain fraction above the peak demand to which renewable generators and storage do *not* contribute. Ignores network., ``solve_network`` `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L73>`__,In active use
+``SAFER``,Adds Regional Capacity Reserve Margin (SAFE) Constraints for defined region- set to a percentage above peak demand level for which renewables and storage do not contribute to. , ``solve_network``,In active use
+``Ep``, Add cost for a carbon-dioxide price configured in ``costs: emission_prices: co2`` to ``marginal_cost`` of generators (other emission types listed in ``network.carriers`` possible as well), ``prepare_network``: `add_emission_prices() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L24>`_ and its `caller <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/prepare_network.py#L158>`__, In active use
+``CCL``,Add minimum or maximum levels of generator nominal capacity per carrier for individual countries. Each constraint can be designated for a specified planning horizon in multi-period models. Opts and path for agg_p_nom_minmax.csv must be defined, ``solve_network``, In active use
+``EQ``," ""Require each country or node to on average produce a minimal share of its total consumption itself. Example: ``EQ0.5c`` demands each country to produce on average at least 50% of its consumption; ``EQ0.5`` demands each node to produce on average at least 50% of its consumption.""", ``solve_network``, In active use
+``ATK``," ""Require each node to be autarkic. Example: ``ATK`` removes all lines and links. ``ATKc`` removes all cross-border lines and links.""", ``prepare_network``, In active use
+``BAU``, Add a per-``carrier`` minimal overall capacity; i.e. at least ``40GW`` of ``OCGT`` in Europe; configured in ``electricity: BAU_mincapacities``, ``solve_network``: `add_opts_constraints() <https://github.com/PyPSA/pypsa-eur/blob/6b964540ed39d44079cdabddee8333f486d0cd63/scripts/solve_network.py#L66>`__, Untested
+``carrier+{c|p|m}factor``,"Alter the capital cost (``c``), installable potential (``p``) or marginal costs (``m``) of a carrier by a factor. Example: ``solar+c0.5`` reduces the capital cost of solar to 50\% of original values.", ``prepare_network``, In active use
+``CH4L``,"Add an overall absolute gas limit. If configured in ``electricity: gaslimit`` it is given in MWh thermal, if a float is appended, the overall gaslimit is assumed to be given in TWh thermal (e.g. ``CH4L200`` limits gas dispatch to 200 TWh termal)", ``prepare_network``: ``add_gaslimit()``, In active use

docs/source/rules-retrieving-data.md

@@ -158,4 +158,4 @@ Documentation of the configuration file ``config/config.yaml`` at
 ## Rule `retrieve_caiso_data`
 ```{eval-rst}
 .. automodule:: retrieve_caiso_data
-```
+```

workflow/Snakefile

@@ -64,8 +64,8 @@ wildcard_constraints:
 
 # Merge subworkflow configs and main config
 # configfile: "config/tests/config.validation.yaml"
-configfile: "config/tests/config.test_simple.yaml"
-# configfile: "config/config.default.yaml"
+# configfile: "config/tests/config.test_simple.yaml"
+configfile: "config/config.default.yaml"
 configfile: "config/config.cluster.yaml"
 configfile: "config/config.osw.yaml"
 configfile: "config/config.plotting.yaml"
@@ -92,6 +92,7 @@ include: "rules/solve_electricity.smk"
 include: "rules/postprocess.smk"
 include: "rules/validate.smk"
 
+
 if "E" not in config["scenario"]["sector"]:
     if not config["scenario"]["sector"]:
         config["scenario"]["sector"] = "E"

workflow/config/config.default.yaml

@@ -12,8 +12,8 @@ run:
 scenario:
   interconnect: western #"usa|texas|western|eastern"
   clusters: [100]
-  opts: [RCo2L-SAFER-CCL]
-  ll: [v1.15]
+  opts: [RCo2L-SAFER-RPS]
+  ll: [v1.0, v1.05]
   scope: "total" # "urban", "rural", or "total"
   sector: "" # G
   planning_horizons:
@@ -75,7 +75,7 @@ electricity:
   retirement: economic # "economic" or "technical"
   SAFE_reservemargin: 0.14
   regional_Co2_limits: 'config/policy_constraints/regional_Co2_limits.csv'
-  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'  
+  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'
   portfolio_standards: 'config/policy_constraints/portfolio_standards.csv'
   SAFE_regional_reservemargins: 'config/policy_constraints/SAFE_regional_prm.csv'
 
@@ -263,7 +263,7 @@ clustering:
     algorithm: kmeans # choose from: [hac, kmeans]
     feature: solar+onwind-time # only for hac. choose from: [solar+onwind-time, solar+onwind-cap, solar-time, solar-cap, solar+offwind-cap] etc.
   cluster_network:
-    algorithm: kmeans
+    algorithm: hac # choose from: [hac, kmeans]
     feature: solar+onwind-time
     aggregation_zones: 'state' # [balancing_area, state]
     exclude_carriers: []

workflow/config/policy_constraints/SAFE_regional_prm.csv

@@ -1,12 +1,12 @@
-region, prm 
-California, 0.14 
-Oregon, 0.14 
-Washington, 0.14 
-Colorado, 0.10 
-New Mexico, 0.14 
-Arizona, 0.14 
-Utah, 0.19 
-Wyoming, 0.14 
-Idaho, 0.14 
-Texas, 0.14 
-Montana, 0.14 
+region, prm
+California, 0.14
+Oregon, 0.14
+Washington, 0.14
+Colorado, 0.10
+New Mexico, 0.14
+Arizona, 0.14
+Utah, 0.19
+Wyoming, 0.14
+Idaho, 0.14
+Texas, 0.14
+Montana, 0.14

workflow/config/policy_constraints/agg_p_nom_minmax.csv

@@ -1,2 +1,2 @@
-planning_horizon,region,carrier,min,max
-2030,California,onwind,0,
+planning_horizon,region,carrier,min,max
+2030,California,onwind,0,

workflow/config/policy_constraints/portfolio_standards.csv

@@ -12,4 +12,4 @@ planning_horizon,region,carrier,pct
 2040,New Mexico,"onwind, solar, offwind_floating, geothermal, offwind, hydro",0.5
 2040,Nevada,"onwind, solar, offwind_floating, geothermal, offwind, hydro",0.5
 2040,Oregon,"onwind, solar, offwind_floating, geothermal, offwind, hydro",0.93
-2040,Washington,"onwind, solar, offwind_floating, geothermal, offwind, hydro",1
+2040,Washington,"onwind, solar, offwind_floating, geothermal, offwind, hydro",1

workflow/config/policy_constraints/regional_Co2_limits.csv

@@ -1,3 +1,3 @@
 region,planning_horizon,limit,import_emissions_factor
 California,2030,30000000,0.428
-California,2040,18000000,0.428
+California,2040,18000000,0.428

workflow/config/tests/config.test.yaml

@@ -78,7 +78,7 @@ electricity:
   retirement: economic # "economic" or "technical"
   SAFE_reservemargin: 0.14
   regional_Co2_limits: 'config/policy_constraints/regional_Co2_limits.csv'
-  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'  
+  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'
   portfolio_standards: 'config/policy_constraints/portfolio_standards.csv'
   SAFE_regional_reservemargins: 'config/policy_constraints/SAFE_regional_prm.csv'
 

workflow/config/tests/config.test_simple.yaml

@@ -74,7 +74,7 @@ electricity:
   retirement: economic # "economic" or "technical"
   SAFE_reservemargin: 0.14
   regional_Co2_limits: 'config/policy_constraints/regional_Co2_limits.csv'
-  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'  
+  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'
   portfolio_standards: 'config/policy_constraints/portfolio_standards.csv'
   SAFE_regional_reservemargins: 'config/policy_constraints/SAFE_regional_prm.csv'
 

workflow/config/tests/config.validation.yaml

@@ -75,7 +75,7 @@ electricity:
   retirement: economic # "economic" or "technical"
   SAFE_reservemargin: 0.14
   regional_Co2_limits: 'config/policy_constraints/regional_Co2_limits.csv'
-  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'  
+  agg_p_nom_limits: 'config/policy_constraints/agg_p_nom_minmax.csv'
   portfolio_standards: '../config/policy_constraints/portfolio_standards.csv'
   SAFE_regional_reservemargins: 'config/policy_constraints/SAFE_regional_prm.csv'
 
@@ -91,8 +91,8 @@ electricity:
 
   extendable_carriers:
     Generator: []
-    StorageUnit: [] 
-    Store: [] 
+    StorageUnit: []
+    Store: []
     Link: []
 
   demand: #EFS used for given planning_horizons year
@@ -276,7 +276,7 @@ clustering:
       ramp_limit_down: max
 
 focus_weights:
-  # California: 0.5
+  California: 0.5
 
 # docs :
 solving:

workflow/rules/build_electricity.smk

@@ -241,6 +241,7 @@ rule build_demand:
     script:
         "../scripts/build_demand.py"
 
+
 rule build_fuel_prices:
     params:
         snapshots=config["snapshots"],
@@ -262,7 +263,6 @@ rule build_fuel_prices:
         "../scripts/build_fuel_prices.py"
 
 
-
 rule add_electricity:
     params:
         length_factor=config["lines"]["length_factor"],
@@ -313,7 +313,6 @@ rule add_electricity:
         demand=RESOURCES + "{interconnect}/demand.csv",
         fuel_costs="repo_data/eia_mappings/fuelCost22.csv",
         ng_electric_power_price=RESOURCES + "{interconnect}/ng_fuel_prices.csv",
-
     output:
         RESOURCES + "{interconnect}/elec_base_network_l_pp.nc",
     log:

workflow/rules/postprocess.smk

@@ -47,4 +47,3 @@ rule plot_figures:
         mem_mb=5000,
     script:
         "../scripts/plot_figures.py"
-

workflow/rules/retrieve.smk

@@ -191,17 +191,18 @@ rule retrieve_cost_data_usa:
     script:
         "../scripts/retrieve_cost_data_usa.py"
 
+
 rule retrieve_caiso_data:
     params:
-        fuel_year= config['costs']['ng_fuel_year']
+        fuel_year=config["costs"]["ng_fuel_year"],
     input:
-        fuel_regions = 'repo_data/wecc_fuelregions.xlsx'
+        fuel_regions="repo_data/wecc_fuelregions.xlsx",
     output:
-        fuel_prices=DATA + "costs/ng_caiso_prices.csv"
+        fuel_prices=DATA + "costs/ng_caiso_prices.csv",
     log:
         LOGS + "retrieve_caiso_data.log",
     shadow:
-        "minimal",
+        "minimal"
     resources:
         mem_mb=2000,
     script:

workflow/rules/solve_electricity.smk

@@ -28,7 +28,7 @@ rule solve_network:
             BENCHMARKS
             + "solve_network/{interconnect}/elec_s_{clusters}_ec_l{ll}_{opts}_{sector}"
         )
-    threads: 4
+    threads: 8
     resources:
         mem_mb=memory,
         walltime=config["solving"].get("walltime", "12:00:00"),