Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add user meaning plugin #44

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions user_meaning/custom-recipes/user-meaning/recipe.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
// Meta data for display purposes
"meta" : {
"label" : "User meaning",
"author" : "Ministère de l’Intérieur — EIG 2017",
"tags" : []
},

"kind" : "PYTHON",

"inputRoles" : [{
"name": "input",
"label": "Input datasset",
"arity": "UNARY",
"required": true,
"acceptsDataset": true
}],
"outputRoles" : [{
"name": "output",
"label": "Output dataset",
"arity": "UNARY",
"required": true,
"acceptsDataset": true
}],

"params": [
{
"name": "meanings",
"label" : "Meaning to map",
"type": "TEXTAREA",
"description": "A json document: each entry is the name of a column. For entry is a dictionnary mapping from a constant to an user meaning",
"mandatory" : true
}
]
}
33 changes: 33 additions & 0 deletions user_meaning/custom-recipes/user-meaning/recipe.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# coding=utf-8

u"""Map constants to user meaning

This plugin was developped by the Ministère de l’Intérieur (French ministry of interior)
in the context of the program Entrepreneurs d’Intérêt Général 2017
"""

from concurrent import futures
import dataiku
from dataiku.customrecipe import get_input_names_for_role
from dataiku.customrecipe import get_output_names_for_role
from dataiku.customrecipe import get_recipe_config
import json
import pandas as pd

# We read the addresses from the input dataset
# And write the coordinates in the output dataset
input_name = get_input_names_for_role('input')[0]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a bit of a nitpick (and doesn't seem to be done elsewhere in codebase), but it might be a good idea to switch key strings to variables. Although, I guess this is all only important depending on how parameter keys to a plugin are versioned. Anyways, just a suggestion but feel free to ignore

input_dataset = dataiku.Dataset(input_name)

output_name = get_output_names_for_role('output')[0]
output_dataset = dataiku.Dataset(output_name)

meanings = get_recipe_config()['meanings'].encode('utf8')
meanings = json.loads(meanings)

df = input_dataset.get_dataframe(infer_with_pandas=False)

for column, mappings in meanings.iteritems():
df[column] = df[column].map(mappings)

output_dataset.write_with_schema(df)
26 changes: 26 additions & 0 deletions user_meaning/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// This file is the descriptor for the DSS Plugin geocoding_BAN
{
// The identifier of the plugin.
// This must be globally unique, and only contain A-Za-z0-9_-
"id" : "user_meaning",

// Version. It is highly recommended to use Semantic Versioning
"version" : "0.0.1",

// Meta data for display purposes
"meta" : {
// Name of this plugin that appears in the interface.
"label": "Map constants to user meaning",

"description": "Transform multiple column values constants to more meaningful values",
"author": "Ministère de l’Intérieur — EIG 2017",

// The icon of a plugin must be one of the FontAwesome 3.1 icons
"icon": "icon-list-ul",

"licenseInfo" : "Apache Software License",

// List of tags for filtering the list of plugins
"tags": []
}
}