-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add user meaning plugin #44
Open
Tristramg
wants to merge
1
commit into
dataiku:master
Choose a base branch
from
entrepreneur-interet-general:user_meaning
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
{ | ||
// Meta data for display purposes | ||
"meta" : { | ||
"label" : "User meaning", | ||
"author" : "Ministère de l’Intérieur — EIG 2017", | ||
"tags" : [] | ||
}, | ||
|
||
"kind" : "PYTHON", | ||
|
||
"inputRoles" : [{ | ||
"name": "input", | ||
"label": "Input datasset", | ||
"arity": "UNARY", | ||
"required": true, | ||
"acceptsDataset": true | ||
}], | ||
"outputRoles" : [{ | ||
"name": "output", | ||
"label": "Output dataset", | ||
"arity": "UNARY", | ||
"required": true, | ||
"acceptsDataset": true | ||
}], | ||
|
||
"params": [ | ||
{ | ||
"name": "meanings", | ||
"label" : "Meaning to map", | ||
"type": "TEXTAREA", | ||
"description": "A json document: each entry is the name of a column. For entry is a dictionnary mapping from a constant to an user meaning", | ||
"mandatory" : true | ||
} | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
# coding=utf-8 | ||
|
||
u"""Map constants to user meaning | ||
|
||
This plugin was developped by the Ministère de l’Intérieur (French ministry of interior) | ||
in the context of the program Entrepreneurs d’Intérêt Général 2017 | ||
""" | ||
|
||
from concurrent import futures | ||
import dataiku | ||
from dataiku.customrecipe import get_input_names_for_role | ||
from dataiku.customrecipe import get_output_names_for_role | ||
from dataiku.customrecipe import get_recipe_config | ||
import json | ||
import pandas as pd | ||
|
||
# We read the addresses from the input dataset | ||
# And write the coordinates in the output dataset | ||
input_name = get_input_names_for_role('input')[0] | ||
input_dataset = dataiku.Dataset(input_name) | ||
|
||
output_name = get_output_names_for_role('output')[0] | ||
output_dataset = dataiku.Dataset(output_name) | ||
|
||
meanings = get_recipe_config()['meanings'].encode('utf8') | ||
meanings = json.loads(meanings) | ||
|
||
df = input_dataset.get_dataframe(infer_with_pandas=False) | ||
|
||
for column, mappings in meanings.iteritems(): | ||
df[column] = df[column].map(mappings) | ||
|
||
output_dataset.write_with_schema(df) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
// This file is the descriptor for the DSS Plugin geocoding_BAN | ||
{ | ||
// The identifier of the plugin. | ||
// This must be globally unique, and only contain A-Za-z0-9_- | ||
"id" : "user_meaning", | ||
|
||
// Version. It is highly recommended to use Semantic Versioning | ||
"version" : "0.0.1", | ||
|
||
// Meta data for display purposes | ||
"meta" : { | ||
// Name of this plugin that appears in the interface. | ||
"label": "Map constants to user meaning", | ||
|
||
"description": "Transform multiple column values constants to more meaningful values", | ||
"author": "Ministère de l’Intérieur — EIG 2017", | ||
|
||
// The icon of a plugin must be one of the FontAwesome 3.1 icons | ||
"icon": "icon-list-ul", | ||
|
||
"licenseInfo" : "Apache Software License", | ||
|
||
// List of tags for filtering the list of plugins | ||
"tags": [] | ||
} | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a bit of a nitpick (and doesn't seem to be done elsewhere in codebase), but it might be a good idea to switch key strings to variables. Although, I guess this is all only important depending on how parameter keys to a plugin are versioned. Anyways, just a suggestion but feel free to ignore