Errors in EHV clustering

[birgits]

When I try to run the EHV clustering I get the following error:

INFO:etrago.cluster.electrical:Start ehv clustering

Busmap does not exist and will be created.

Unexpected exception formatting exception. Falling back to standard exception

Traceback (most recent call last):
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/tmp/ipykernel_10186/207841767.py", line 1, in <module>
    etrago.ehv_clustering()
  File "/home/birgit/virtualenvs/hyBit/git_repos/eTraGo/etrago/cluster/electrical.py", line 475, in ehv_clustering
    busmap = busmap_from_psql(self)
  File "/home/birgit/virtualenvs/hyBit/git_repos/eTraGo/etrago/cluster/spatial.py", line 545, in busmap_from_psql
  File "/home/birgit/virtualenvs/hyBit/git_repos/eTraGo/etrago/cluster/spatial.py", line 457, in busmap_by_shortest_path
    assert (buses[~mask].v_nom.astype(int).isin(tolvl)).all()
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 2105, in showtraceback
    stb = self.InteractiveTB.structured_traceback(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1396, in structured_traceback
    return FormattedTB.structured_traceback(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1287, in structured_traceback
    return VerboseTB.structured_traceback(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1140, in structured_traceback
    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1055, in format_exception_as_a_whole
    frames.append(self.format_record(record))
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 955, in format_record
    frame_info.lines, Colors, self.has_colors, lvals
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 778, in lines
    return self._sd.lines
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/core.py", line 734, in lines
    pieces = self.included_pieces
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/core.py", line 681, in included_pieces
    pos = scope_pieces.index(self.executing_piece)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/core.py", line 660, in executing_piece
    return only(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/executing/executing.py", line 190, in only
    raise NotOneValueFound('Expected one value, found 0')
executing.executing.NotOneValueFound: Expected one value, found 0

Also, when I restore the network from csv and then try to run the EHV clustering, I get an error, that the session is not set. It could also be useful to fix that.

INFO:etrago.cluster.electrical:Start ehv clustering

Unexpected exception formatting exception. Falling back to standard exception

Traceback (most recent call last):
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/tmp/ipykernel_10186/207841767.py", line 1, in <module>
    etrago.ehv_clustering()
  File "/home/birgit/virtualenvs/hyBit/git_repos/eTraGo/etrago/cluster/electrical.py", line 475, in ehv_clustering
    busmap = busmap_from_psql(self)
  File "/home/birgit/virtualenvs/hyBit/git_repos/eTraGo/etrago/cluster/spatial.py", line 533, in busmap_from_psql
  File "/home/birgit/virtualenvs/hyBit/git_repos/eTraGo/etrago/cluster/spatial.py", line 524, in fetch
    query = (
AttributeError: 'Etrago' object has no attribute 'session'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 2105, in showtraceback
    stb = self.InteractiveTB.structured_traceback(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1396, in structured_traceback
    return FormattedTB.structured_traceback(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1287, in structured_traceback
    return VerboseTB.structured_traceback(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1140, in structured_traceback
    formatted_exception = self.format_exception_as_a_whole(etype, evalue, etb, number_of_lines_of_context,
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 1055, in format_exception_as_a_whole
    frames.append(self.format_record(record))
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 955, in format_record
    frame_info.lines, Colors, self.has_colors, lvals
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/IPython/core/ultratb.py", line 778, in lines
    return self._sd.lines
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/core.py", line 734, in lines
    pieces = self.included_pieces
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/core.py", line 681, in included_pieces
    pos = scope_pieces.index(self.executing_piece)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/utils.py", line 144, in cached_property_wrapper
    value = obj.__dict__[self.func.__name__] = self.func(obj)
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/stack_data/core.py", line 660, in executing_piece
    return only(
  File "/home/birgit/virtualenvs/hyBit/lib/python3.8/site-packages/executing/executing.py", line 190, in only
    raise NotOneValueFound('Expected one value, found 0')
executing.executing.NotOneValueFound: Expected one value, found 0

To reconstruct the errors, you can use the etrago notebook for the hyBit workshops.

[CarlosEpia]

Hi Birgit. I was able to run eHV using your notebook but only when "foreign_lines": {"carrier": "AC"}. When the carrier is set to "DC", the next error appears:
INFO:etrago.cluster.electrical:Start ehv clustering
WARNING:pypsa.io:The following Line have buses which are not defined:
Index(['25029', '18661', '26486', '1349', '16096', '7656', '7701'], dtype='object', name='Line')
WARNING:pypsa.io:The following Link have buses which are not defined:
Index(['N29220', 'N29221', 'N29222', 'N29223', 'N29224', 'N29225', 'N29228',
'N29229', 'N29230', 'N29231',
...
'N28102', 'N8338', 'N8236', 'N3937', 'N20802', 'N28101', 'N28243',
'N20616', 'N22629', 'N20904'],
dtype='object', name='Link', length=144)
WARNING:pypsa.io:The following Link have buses which are not defined:
Index(['N29222', 'N29225', 'N29245', 'N29248', 'N29249', 'N29262', 'N29264',
'N29265', 'N29272', 'N29273',
...
'N3937', 'N20802', 'N28101', 'N28243', 'N20616', 'N22629', 'N20904',
'N7497', 'N16326', 'N16569'],
dtype='object', name='Link', length=105)

File ~/env/etrago310/lib/python3.10/site-packages/spyder_kernels/py3compat.py:356 in compat_exec
exec(code, globals, locals)

File ~/git/eTraGo/etrago/appl.py:671
etrago = run_etrago(args, json_path=None)

File ~/git/eTraGo/etrago/appl.py:637 in run_etrago
etrago.ehv_clustering()

File ~/git/eTraGo/etrago/cluster/electrical.py:477 in ehv_clustering

File ~/git/eTraGo/etrago/cluster/electrical.py:382 in cluster_on_extra_high_voltage

File ~/env/etrago310/lib/python3.10/site-packages/pypsa/components.py:1237 in determine_network_topology
adjacency_matrix = self.adjacency_matrix(

File ~/env/etrago310/lib/python3.10/site-packages/pypsa/graph.py:165 in adjacency_matrix
return sp.sparse.coo_matrix(

File ~/env/etrago310/lib/python3.10/site-packages/scipy/sparse/_coo.py:204 in init
self._check()

File ~/env/etrago310/lib/python3.10/site-packages/scipy/sparse/_coo.py:297 in _check
raise ValueError('negative column index found')

ValueError: negative column index found

I will deal with this problem.

[ClaraBuettner]

Was this issue solved by merging #676?

[CarlosEpia]

Yes. It was.