Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PySpark Imputation Methods #77

Open
dombean opened this issue Apr 6, 2024 · 0 comments
Open

Add PySpark Imputation Methods #77

dombean opened this issue Apr 6, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@dombean
Copy link
Member

dombean commented Apr 6, 2024

Add Imputation Methods to rdsa_utils/methods/imputation:

  • Rollback Method
  • Growth Rate (Forward) Method
  • Growth Rate (Backwards) Method

To ensure our rdsa-utils package, particularly within the methods/imputation directory, remains well-organised and user-friendly, we're implementing a clear naming convention for our file structure.

This approach allows us to clearly denote which library (PySpark or Pandas) a file is intended for, using prefixes (pyspark_ or pandas_) followed by the method name, such as rollback, growth_rate_forward, or growth_rate_backward.

This naming strategy facilitates the inclusion of library-specific methods without the need for each method to have a counterpart in the other library, thereby providing flexibility in our development process. It also helps in reducing unnecessary constraints and ensuring that users can easily navigate and identify the functionalities relevant to their needs. For shared logic that applies across both libraries, we use a common_utils.py file.

rdsa-utils/
│
├── methods/
│   ├── imputation/
│   │   ├── pyspark_rollback.py
│   │   ├── pyspark_growth_rate_forward.py
│   │   ├── pyspark_growth_rate_backward.py
│   │   ├── pandas_rollback.py             # Optional, only if exists
│   │   ├── pandas_growth_rate_forward.py  # Optional, only if exists
│   │   ├── pandas_growth_rate_backward.py # Optional, only if exists
│   │   └── common_utils.py                # For shared logic, if any
@dombean dombean added the enhancement New feature or request label Apr 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant