Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add function to output ParameterNode as YAML #295

Draft
wants to merge 15 commits into
base: master
Choose a base branch
from

Conversation

SylviaDu99
Copy link
Collaborator

WIP

fixes: #274

Added write_yaml function to output ParameterNode data to a YAML file; test_write_yaml test to produce a sample output

Copy link

codecov bot commented Oct 16, 2024

Codecov Report

Attention: Patch coverage is 72.83951% with 22 lines in your changes missing coverage. Please review.

Project coverage is 84.62%. Comparing base (9fbe198) to head (731c921).

Files with missing lines Patch % Lines
policyengine_core/parameters/parameter_scale.py 20.00% 12 Missing ⚠️
policyengine_core/parameters/parameter_node.py 85.71% 2 Missing and 2 partials ⚠️
policyengine_core/parameters/parameter.py 86.36% 1 Missing and 2 partials ⚠️
tests/core/test_parameters.py 84.61% 2 Missing ⚠️
policyengine_core/parameters/at_instant_like.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #295      +/-   ##
==========================================
- Coverage   84.73%   84.62%   -0.12%     
==========================================
  Files         191      191              
  Lines        9773     9845      +72     
  Branches     1018     1031      +13     
==========================================
+ Hits         8281     8331      +50     
- Misses       1204     1222      +18     
- Partials      288      292       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Copy link
Contributor

@anth-volk anth-volk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your continued work on this, @SylviaDu99. This is a challenging component, so it's great to have your experience here. I made a couple of structural recommendations, but a lot boils down to these two things:

  1. I'd move the meat of the function back to ParameterNode and limit the method you wrote in AtInstantLike to do just what it says - get a dictionary of attributes
  2. I'd programmatically iterate over all attrs, then remove ones we don't want, instead of hard-coding what we do want, as I think the former is more maintainable.

Thanks for your work, and I look forward to chatting with you about it tomorrow.


def get_attr_dict(self) -> dict:
attr_dict = {}
attr_list = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the best practice here would be to keep this method as simple as possible, then do a bit more work with it where we actually use it. What I would do here is:

  1. Programmatically loop over all attributes and feed into a relevant data structure
  2. Return said structure

I wouldn't recurse downward here, either. I think we can do that inside our custom ParameterNode method, where we're more broadly defining what we're doing.

@anth-volk
Copy link
Contributor

@SylviaDu99 Thanks for your revisions to this. I decided to test this out by printing out the entirety of the -us package's parameter structure within its "gov" folder, which I'd be happy to share with you. A couple issues came up:

  • The code still writes out the parent value of a relevant parameter, meaning there may be an issue in get_attr_dict's processing of the exclusion list. It also appears to add children at certain points, but I'm not certain on that, as the output file is extremely large
  • I'm not 100% sure this method is 100% recursive. I think - as long as it isn't memory-intensive enough to crash - you would probably want to follow the following flow:
    • Create exclusion list in ParameterNode; perhaps don't add children here
    • Run get_attr_dict, possible even without applying the exclusion list, because we'll use children a bit lower
    • Iterate over dict and delete items contained within exclusion list
    • For child in children, write a key equivalent to the child's name value, then have the output of write_yaml tacked onto the output at that key

@@ -274,3 +277,27 @@ def get_child(self, path: str) -> "ParameterNode":
f"Could not find the parameter (failed at {name})."
)
return node

def get_attr_dict(self) -> dict:
data = self.__dict__.copy()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor suggestion: in this function, could we process every attr that isn't in child_dict, then process all the children? This makes keys at the end of the alphabet (like "tracer") more easily readable.

Copy link
Contributor

@anth-volk anth-volk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your work on this @SylviaDu99. I just went through and ran the code, and I really appreciate the effort you've put into this. I had just one minor change I was hoping you might make, but after that, I think we can move onto the second step of the process, as we discussed on Wednesday.

@anth-volk
Copy link
Contributor

@SylviaDu99 Yesterday, I messaged to say that based on its output, this all looked good. I went through the code more thoroughly this time, and I think this is what we're looking for. I'm going to chat with Nikhil early tomorrow about what sort of testing strategy should be implemented upon this to be triple sure that we're in a good state, and then I'm hoping to chat tomorrow during our meeting about next steps. Thanks again for your work on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add fully-functional ParameterNode.__repr__
2 participants