feat: use bhepop2 package for income assignment #243
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
describe attribute modalities when giving attribute selection
municipality stage now returns a DataFrame containing income deciles per attributes in addition to the usual global deciles. Two columns "attribute" and "modality" have been added to specify the related attribute and the value it takes (modality). Attribute and modality for global deciles are "all". Filter on "all" attribute and modality have been added where data.income.municipality were used.
merge municipality_attributes.py into municipality.py move compare_methods.py to analysis/methods/income/ created a utils.py module in synthesis/population/income/ to store common functions added test dataset and tests --------- Co-authored-by: leo-desbureaux-tellae <[email protected]>
|
Hi! Thanks a lot, I'm at hEART this week and a bit busy the week after, but I'll look at it asap! |
sebhoerl
requested changes
Jul 2, 2024
|
Thanks a lot, LGTM |
sebhoerl
approved these changes
Jul 4, 2024
|
Issues from your first review are fixed ! Let me know if there is something more that needs changes. |
|
Resolved conflict in changelog |
|
Thanks, looks all good, you can merge! |
|
Perfect, thank you very much @leo-desbureaux-tellae, thanks @sebhoerl for the review !! ⛵ 🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduction of the Bhepop2 package for income assignment
data.income.municipality
The distributions DataFrame returned by data.income.municipality is now tagged by attribute and modality
Attributes describes a property present on the the population agents. A modality is a value taken by this attribute.
In Eqasim, we use two attributes:
New columns of the returned DataFrame are ["commune_id", "q1", "q2", "q3", "q4", "q5", "q6", "q7", "q8", "q9", "attribute", "modality", "is_imputed", "is_missing", "reference_median"]
Global distributions (those that were returned in the previous version of municipality.py) are tagged with attribute and modality "all".
synthesis.population.income
New config option "income_assignation_method" (should it be "income_assignment_method" ?). This config allows choosing the method used to assign an income to population agents.
The former method is called via the config "uniform" (what should be the default config ?).
A new assignation method has been added, called via the config "bhepop2". This method uses the Bhepop2 package to match per attribute distributions instead of just matching the global one.
analysis.methods.income.compare_methods
A new analysis module has been added to compare the assignation methods. It can be run using the path
analysis.methods.income.compare_methods.This module generates plots comparing income distributions of each assignation method and the source distribution (here, Filosofi data).
This comparison is done per attribute. For instance, we compare the income distribution of individuals with attribute "family_comp" equal to "Single_parent" for the two methods, and see what method matches best the source distribution.
Another output of this module is a table measuring the distance of each method to the source distribution, here again per attribute. This allows a more measurable comparison between assignation methods.