Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write Datapackage.json #31

Closed
5 of 7 tasks
FelixMau opened this issue Jun 12, 2023 · 4 comments · Fixed by #37
Closed
5 of 7 tasks

Write Datapackage.json #31

FelixMau opened this issue Jun 12, 2023 · 4 comments · Fixed by #37
Assignees

Comments

@FelixMau
Copy link
Contributor

FelixMau commented Jun 12, 2023

To be able to create an energy system from a data package, you must provide a datapackage.json

It is necessary to write down foreign_keys whilst build_datapackage to keep track of timeseries column names and resource names. (Closes #23 )
Note: There is already a method within DataPackage class prepared for this purpose

The data package must provide the following information:

  • The path where the resource (and foreign resources) is located.

  • Information on fields, including their types and metadata (defaults could be used)

  • Foreign keys that provide additional information, such as bus data.

  • "Foreign keys" pointing to time series data, (different from ordinary foreign keys.)

  • Additionally, a dataset needs to be built to contain all the buses (whilst build_datapackage).

To monitor and check whether code is working properly:

  • write Test
  • define expected and Input data
@FelixMau FelixMau linked a pull request Jun 26, 2023 that will close this issue
@FelixMau FelixMau self-assigned this Jun 28, 2023
@FelixMau FelixMau linked a pull request Jul 10, 2023 that will close this issue
@FelixMau
Copy link
Contributor Author

FelixMau commented Jul 26, 2023

When initializing classes with attributes that can be either a sequence or a scalar sequence() is being called from __init__. This Process turns eventually given default values like the fixed_costs within facades.Storage from integer into _Sequence that is emulating a list with "unknown length" as explained in this example:

>>> s = _Sequence(default=42)
    >>> len(s)
    0
    >>> s[1]
    42

The Sequence class cannot be saved within a regular Datapackage.

Suggestion 1:

When unpacking the AdapterClass to a dict with as_dict (to save it as an Pandas.DataFrame) I suggest that we test if we have recieved a sequence and if yes writing the default value of the _Sequence to the dict.

Suggestion 2:

As it is now a empty list is being returned and when deseralizing the Datapackage it is interpreted as full Sequence leading to errors since the sequence is actually empty. I think we could also improve on the tabular deserialization part and improve its stability by catching this error and take the default value if a faulty iterable is passed. This might be helpful as well for upcomming implementations of periodically changing scalars.
Edit: Problem here might be that we do not know the full timeseries length (or amount of steps) yet when initializing the facades, correct?

I would like your feedback regarding the plans on scalars within multi period feature.

@nailend
Copy link
Contributor

nailend commented Jul 26, 2023

@FelixMau

Suggestion 1:

I agree with you, thank you for tackling this. We will probably have to rework this after the deserialization is solved, but we will do this in a separate issue.

Suggestion 2:

Currently, the empty list is detected as empty list which is fine, as we actually save it in the wrong way. So we have to think of a different way of passing a _Sequence via the datapackage.
Concerning the periodic-scalar, I would wait until this is/or if it is fixed on the oemof.solph side, but I am keen enhancing _sequence for this. It makes sense to think this together. But we probably have to take care of #116 before

@nailend
Copy link
Contributor

nailend commented Jul 27, 2023

I gave the plumbing a try, see here, do you think thats possible to implement from the adapter side? to somehow catch it if the parameter change per period and are then written as a dict?

But lets tackle this is in a new Issue

@FelixMau
Copy link
Contributor Author

FelixMau commented Aug 10, 2023

Test improvements moved to #43
Developing Period merging and reading moved to #40

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants