Odd results for Kenya in 2030, 2040, 2050

[Haiping007]

Hi, I have some preliminary results for Kenya in 2030, 2040 and 2050.
The first run for Kenya (projection in 2030 and 2050) was conducted in September which looked reasonable (Figure 1-2).

Then the first run for Kenya (projection in 2040) was conducted in November. It shows the share of geothermal in 2040 is higher than in 2030 and 2050 while solar is far lower than in 2030 and 2050. This is a bit odd (figure 3-4)

So I have rerun the model for Kenya (projection in 2030 and 2050) several times recently. The results for 2030 are so different from the first run that its share of geothermal also increased a lot and solar became very low, similar to the results for 2040. However, the results for 2050 are nearly the same as the results in the first run. (Figure 5-6). The results of H2 and battery capacities also have similar changes in 2030 (very low than the first run) while there are small changes for 2050.

I have two questions:

1. Do you have any suggestions on why there are big changes in the results (for 2030) run in September and November with the same setting? Did you make any modifications to the model?
2. The share of electricity between geothermal and solar looks odd over time. I was expecting the share of solar would be higher in 2030 and 2040 with smooth growth over time as the results show there is a dramatic soar in 2050. Is this because of the cost setting between geothermal and solar in the model? Kenya has blessed geothermal resources, I was expecting there to be some shares both for geothermal and solar in Kenya in the Future but with a smooth increase. What do you think?

Thank you!
Haiping

[Haiping007]

Also, the result of the share of coal generation looks odd in 2040, much higher than in 2030.
The CO2 limits are based on some reports of Kenya that there is a pick of emission in 2030 and 0 in 2050, and the emission in 2040 is set between these two.

[Haiping007]

@davide-f
I have some thoughts related to the results for 2030, 2040 and 2050:

1. one is in the "config" file, should the "geothermal" be moved to the generators of extendable_carriers as the geothermal resource in Kenya is huge (currently only used 10%)?
2. The second is the cost of geothermal in Kenya should be lower than the average cost of geothermal in the cost file (Data/cost.cvs).
3. Data/cost.cvs only contains the cost in 2030? so the cost of different technologies are the same in 2040 and 2050?
   Thank you!

[ekatef]

Hello @Haiping007! Thanks a lot for sharing. Your results look very interesting and it would be great to discuss them in details, although it may require a bit of time.

As a quick answer to a more technical part of your questions:

1. Yes, it sounds like it may be a good idea to specify geothermal as an extendable carrier. That could be rather straightforward, if you see that the geothermal potential is under-used in the current locations of geothermal plants. However, some iterations may be needed, as currently data for geothermal potential are not implemented into the model. So, you will need to cross-check the resulted value of the optimised geothermal capacity against the available one. Happy to discuss more and support you in investigating this feature in more details. We would be very interested in improving representation of geothermal in the model.
2. Current values of the costs are based on review of the published sources, and do not include regional differences. So, please feel free to suggest improvements in case you have more precise data on that.
3. It is possible to extract costs for any year of interest. To activate this option, you need to set retrieve_cost_data: true and define a year of interest in costs -> year: 2030.

Regarding your questions related to modeling, agree that the results look a bit weird. There have been changes in the project throughout the previous month, but not sure if any of these changes would result to significant altering in the modeling output. My feeling is that it's definitely worth to analyse the model in details to understand the reasons of the behaviour you observe. @davide-f may definitely have valuable insight on this.

Happy to continue the discussion. Would it be convenient for you to join one of our developers meeting?

[davide-f]

Hello @Haiping007 :D
Thanks @ekatef for the comments :)

I agree that results are a bit odd.
As comments:

• what solver are you using?
• what spatial (number of clusters) and temporal resolution (hourly for a year?) are you using?
• please, check the optimality status at the end of the optimization: has the solver converged? there may be numerical issues

Geothermal would be interesting to have it expandable but that would require a methodology to estimate the expandability potential that is currently missing.
If you may be interesting to do that let us know :D @LukasFrankenQ has this PR in europe for similar context that provides some insights PyPSA/pypsa-eur#733

[Haiping007]

Hi! @ekatef Thank you for your detailed answer! Yes, I am happy to join one of your developers meeting.

I have tried to add "geothermal" to the generators of extendable_carriers. The results are still the same as Figure 5-6 posted.

In order to understand the model better. I rerun the model with different levels of CO2 emission settings in 2050 for Kenya.
There are some interesting results as follows (Figure7-8). Solar seems very sensitive to the CO2 emission constraint. When there is little CO2 emission allowed, geothermal is reduced a lot while solar is boosted although the capacity of geothermal is the same with different levels of CO2 emission settings. Is there a big difference in CO2 emissions between these two technologies in the model?

Hi! @davide-f thank you for your support!
-I am using GUROBI as the solver
-20 clusters and 1H
-How to check the optimality status at the end of the optimization? It looks like the optimization is successful.

The last job of running the model in the command window is as follows:

......
Barrier solved model in 153 iterations and 687.59 seconds (322.04 work units)
Optimal objective 1.03609365e+09

INFO:pypsa.linopf:Optimization successful. Objective value: 1.04e+09
C:\Users\haipings\AppData\Local\miniconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py:1281: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.
values = variables.applymap(lambda x: variables_sol.loc[x])
C:\Users\haipings\AppData\Local\miniconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py:1281: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.
values = variables.applymap(lambda x: variables_sol.loc[x])
C:\Users\haipings\AppData\Local\miniconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py:1343: FutureWarning: DataFrame.applymap has been deprecated. Use DataFrame.map instead.
duals = constraints.applymap(
C:\Users\haipings\AppData\Local\miniconda3\envs\pypsa-earth\lib\site-packages\pypsa\linopf.py:1415: FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do frame.T.groupby(...) without axis instead.
pd.concat(
INFO:pypsa.io:Exported network elec_s_20_ec_lcopt_Co2L-1H.nc has stores, loads, generators, carriers, links, global_constraints, buses, lines, storage_units
Benchmark: unable to collect cpu and memory benchmark statistics
CRITICAL:snakemake.logging:Benchmark: unable to collect cpu and memory benchmark statistics
[Tue Dec 19 16:01:54 2023]
INFO:snakemake.logging:[Tue Dec 19 16:01:54 2023]
Finished job 1.
INFO:snakemake.logging:Finished job 1.
19 of 20 steps (95%) done
INFO:snakemake.logging:19 of 20 steps (95%) done
Select jobs to execute...
WARNING:snakemake.logging:Select jobs to execute...

INFO:snakemake.logging:
[Tue Dec 19 16:01:54 2023]
INFO:snakemake.logging:[Tue Dec 19 16:01:54 2023]
localrule solve_all_networks:
input: results/networks/elec_s_20_ec_lcopt_Co2L-1H.nc
jobid: 0
reason: Input files updated by another job: results/networks/elec_s_20_ec_lcopt_Co2L-1H.nc
resources: tmpdir=C:\Users\haipings\AppData\Local\Temp
INFO:snakemake.logging:localrule solve_all_networks:
input: results/networks/elec_s_20_ec_lcopt_Co2L-1H.nc
jobid: 0
reason: Input files updated by another job: results/networks/elec_s_20_ec_lcopt_Co2L-1H.nc
resources: tmpdir=C:\Users\haipings\AppData\Local\Temp

INFO:snakemake.logging:
[Tue Dec 19 16:01:54 2023]
INFO:snakemake.logging:[Tue Dec 19 16:01:54 2023]
Finished job 0.
INFO:snakemake.logging:Finished job 0.
20 of 20 steps (100%) done
INFO:snakemake.logging:20 of 20 steps (100%) done
Complete log: .snakemake\log\2023-12-19T153634.693587.snakemake.log
WARNING:snakemake.logging:Complete log: .snakemake\log\2023-12-19T153634.693587.snakemake.log

[ekatef]

Hello @Haiping007, as a short heads-up: the developers meeting is scheduled to tomorrow 4pm UTC/UK time on PyPSA-Earth Discord channel (https://discord.com/invite/AnuJBk23FU). That will be a monthly one focused on high-level updates, where we'll discuss our plans and goals.

Thanks a lot for the updates and happy to look into them a bit later.

[Haiping007]

Hi, @ekatef , thank you for the invitation. See you guys tomorrow.

[ekatef]

Hello @Haiping007! Sorry for taking a while to add my answer.

It looks like your solver has been quite happy with the solution. You can also additionally check the logs in your pypsa-erath/logs folder: there shold be solve_network for each scenario run (specified as run keyword in config.yaml).

Otherwise, settings of your run looks perfect to me. I think the most likely reason for the inconsistencies you found are some mess-up in the input data. Sometimes it may happen that the model fetches inputs from outdated runs which lead to some weird outputs. That is why we recommend to clean-up outputs of the previous runs and use run argument to keep results of runs for different scenarios in dedicated folders.