add_example_database() broken by bw2data.parameters

[michaelweinold]

In a fresh virtual environment with

bw2data==4.0.dev51
bw2io==0.9.dev34
bw2calc==2.0.dev18

And the simple command:

import bw2io as bi
bi.add_example_database(searchable=False)

I get:

/Users/michaelweinold/github/bw_webapp/my_env_name/lib/python3.11/site-packages/bw2calc/__init__.py:44: UserWarning: 
It seems like you have an ARM architecture, but haven't installed scikit-umfpack:

    https://pypi.org/project/scikit-umfpack/

Installing it could give you much faster calculations.

  warnings.warn(UMFPACK_WARNING)
/Users/michaelweinold/github/bw_webapp/my_env_name/lib/python3.11/site-packages/openpyxl/worksheet/_reader.py:329: UserWarning: Unknown extension is not supported and will be removed
  warn(msg)
Extracted 4 worksheets in 0.01 seconds
Applying strategy: csv_restore_tuples
Applying strategy: csv_restore_booleans
Applying strategy: csv_numerize
Applying strategy: csv_drop_unknown
Applying strategy: csv_add_missing_exchanges_section
Applying strategy: strip_biosphere_exc_locations
Applying strategy: set_code_by_activity_hash
Applying strategy: assign_only_product_as_production
Applying strategy: drop_falsey_uncertainty_fields_but_keep_zeros
Applying strategy: convert_uncertainty_types_to_integers
Applying strategy: convert_activity_parameters_to_list
Applied 11 strategies in 0.00 seconds
Applying strategy: link_iterable_by_fields
Not able to determine geocollections for all datasets. This database is not ready for regionalization.
  0%|          | 0/9 [00:00<?, ?it/s]/Users/michaelweinold/github/bw_webapp/my_env_name/lib/python3.11/site-packages/bw2data/backends/typos.py:93: UserWarning: Possible incorrect exchange key found: Given `Comment` but `comment` is more common
  warnings.warn(
100%|██████████| 9/9 [00:00<00:00, 6559.29it/s]
Vacuuming database 
Created database: Mobility example
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/peewee.py:7285, in BaseModelSelect.get(self, database)
   7284 try:
-> 7285     return clone.execute(database)[0]
   7286 except IndexError:

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/peewee.py:4581, in CursorWrapper.__getitem__(self, item)
   4580     self.fill_cache(item if item > 0 else 0)
-> 4581     return self.row_cache[item]
   4582 else:

IndexError: list index out of range

During handling of the above exception, another exception occurred:

ActivityParameterDoesNotExist             Traceback (most recent call last)
Cell In[2], line 1
----> 1 bi.add_example_database(searchable=False)

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/bw2io/data/__init__.py:483, in add_example_database(overwrite, searchable)
    480 for ds in Database("Mobility example"):
    481     parameters.add_exchanges_to_group(group, ds)
--> 483 parameters.recalculate()
    485 ipcc = Method(("IPCC", "simple"))
    486 ipcc.register()

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/bw2data/parameters.py:1620, in ParameterManager.recalculate(self)
   1618 if obj.name in databases or obj.name == "project":
   1619     continue
-> 1620 ActivityParameter.recalculate(obj.name)

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/bw2data/parameters.py:824, in ActivityParameter.recalculate(group)
    820 # Update activity parameter values
    821 data = ActivityParameter.load(group)
    822 static = {
    823     k: v
--> 824     for k, v in ActivityParameter._static_dependencies(group).items()
    825     if k not in data
    826 }
    827 ParameterSet(data, static).evaluate_and_set_amount_field()
    828 with parameters.db.atomic():

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/bw2data/parameters.py:649, in ActivityParameter._static_dependencies(group)
    643 @staticmethod
    644 def _static_dependencies(group):
    645     """Get dictionary of ``{name: amount}`` for all variables defined in dependency chain.
    646 
    647     Be careful! This could have variables which overlap with local variable names. Designed for internal use.
    648     """
--> 649     database = ActivityParameter.get(group=group).database
    651     chain = [ProjectParameter.static(), DatabaseParameter.static(database)] + [
    652         ActivityParameter.static(g) for g in Group.get(name=group).order[::-1]
    653     ]
    655     result = {}

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/peewee.py:6833, in Model.get(cls, *query, **filters)
   6831 if filters:
   6832     sq = sq.filter(**filters)
-> 6833 return sq.get()

File ~/github/bw_webapp/my_env_name/lib/python3.11/site-packages/peewee.py:7288, in BaseModelSelect.get(self, database)
   7286 except IndexError:
   7287     sql, params = clone.sql()
-> 7288     raise self.model.DoesNotExist('%s instance matching query does '
   7289                                   'not exist:\nSQL: %s\nParams: %s' %
   7290                                   (clone.model, sql, params))

ActivityParameterDoesNotExist: <Model: ActivityParameter> instance matching query does not exist:
SQL: SELECT "t1"."id", "t1"."group", "t1"."database", "t1"."code", "t1"."name", "t1"."formula", "t1"."amount", "t1"."data" FROM "activityparameter" AS "t1" WHERE ("t1"."group" = ?) LIMIT ? OFFSET ?
Params: ['2', 1, 0]

Interestingly, this seems to work find on Brightway Live with the same package versions.

[michaelweinold]

The issue is with the parameters.recalculate() function in combination with the overwrite=True option of the bi.add_example_database() function not clearing/deleting parameters.

@cmutel, since I did not find a function to clear/delete all parameters, how about this:

bd.parameters.new_project_parameters(data=[])

as part of the overwrite=True functionality?

[bfdb]

I ran into the same issue, though I'm not sure that #272 resolves this.

That is, if starting from scratch, 'Mobility example' is not in databases, and thus we do not enter the else statement in line 451.

However, I still get the same error message.

Moreover, I ran bd.parameters.new_project_parameters(data=[]) in my main script before running bi.add_example_database(), and this did not resolve the error message.

Lastly, I ran bi.add_example_database(overwrite=False) and this also gave the same error message.

I'm using version '0.9.DEV37' on Windows 11 Business.

[mfastudillo]

we may need to specify that bw2data needs bw2parameters>=1.1.0

[bfdb]

bw2parameters is 1.1.0 on my machine

[mfastudillo]

with bw2data 4.0.dev51 , bw2parameters 1.1.0 and bw2io 0.9.dev30 it is working in my machine.. maybe there breaking changes are in bw2io