Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/9 add filterDefinition & resolve filter details #182

Draft
wants to merge 18 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
efbfce9
#9 first draft at adding filter definitions
JoernBerkefeld Apr 12, 2021
7b1208d
Merge branch 'develop' into feature/i9-Add-support-for-filterDefiniti…
JoernBerkefeld Jan 14, 2022
e6a80f9
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Apr 14, 2022
d4a0e53
#9: make branch ready for 4.0.0 release
JoernBerkefeld Apr 17, 2022
7c41b1b
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Apr 19, 2022
2fb794e
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Apr 22, 2022
ba2ff2b
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld May 7, 2022
9de5793
Merge branch 'feature/310-add-optional-key-parameter-to-deploy' into …
JoernBerkefeld May 18, 2022
9687068
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Jul 5, 2022
11c0341
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Dec 15, 2022
9aa53bf
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Aug 28, 2023
0e519db
#9: ran lint:fix
JoernBerkefeld Aug 29, 2023
a742713
#9: improve retrieve filterDefinition
JoernBerkefeld Aug 29, 2023
15548a6
#9: switch to folder-based collection retrieve; resolve all filterDef…
JoernBerkefeld Aug 30, 2023
8cb78b0
#9: make filters available to type automation
JoernBerkefeld Aug 30, 2023
45f8657
#325: save notificationEmailAddress as array of emails to enhance UX
JoernBerkefeld Aug 30, 2023
d962bab
Merge branch 'develop' into feature/9-Add-filterDefinition-&-resolve-…
JoernBerkefeld Aug 30, 2023
e0a4655
#325: adapt tests to change in 45f8657ba77f75fa8f5f9f7d74b7dad9dc9d02cb
JoernBerkefeld Aug 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions lib/MetadataTypeDefinitions.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ const MetadataTypeDefinitions = {
eventDefinition: require('./metadataTypes/definitions/EventDefinition.definition'),
fileTransfer: require('./metadataTypes/definitions/FileTransfer.definition'),
filter: require('./metadataTypes/definitions/Filter.definition'),
filterDefinition: require('./metadataTypes/definitions/FilterDefinition.definition'),
folder: require('./metadataTypes/definitions/Folder.definition'),
ftpLocation: require('./metadataTypes/definitions/FtpLocation.definition'),
importFile: require('./metadataTypes/definitions/ImportFile.definition'),
Expand Down
1 change: 1 addition & 0 deletions lib/MetadataTypeInfo.js
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ const MetadataTypeInfo = {
eventDefinition: require('./metadataTypes/EventDefinition'),
fileTransfer: require('./metadataTypes/FileTransfer'),
filter: require('./metadataTypes/Filter'),
filterDefinition: require('./metadataTypes/FilterDefinition'),
folder: require('./metadataTypes/Folder'),
ftpLocation: require('./metadataTypes/FtpLocation'),
importFile: require('./metadataTypes/ImportFile'),
Expand Down
193 changes: 192 additions & 1 deletion lib/metadataTypes/Filter.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,32 @@
'use strict';

/**
* @typedef {Object} FilterItem
* @property {number} categoryId folder id
* @property {string} [createdDate] -
* @property {string} customerKey key
* @property {string} destinationObjectId DE/List ID
* @property {1|2|3|4} destinationTypeId 1:SubscriberList, 2:DataExtension, 3:GroupWizard, 4:BehavioralData
* @property {string} filterActivityId ?
* @property {string} filterDefinitionId ObjectID of filterDefinition
* @property {string} modifiedDate -
* @property {string} name name
* @property {string} sourceObjectId DE/List ID
* @property {1|2|3|4} sourceTypeId 1:SubscriberList, 2:DataExtension, 3:GroupWizard, 4:BehavioralData
* @property {number} statusId ?
*
* @typedef {Object.<string, FilterItem>} FilterMap
*/

const MetadataType = require('./MetadataType');
const Util = require('../util/util');

const dataTypes = {
1: 'List',
2: 'DataExtension',
3: 'Group Wizard',
4: 'Behavioral Data',
};

/**
* Filter MetadataType
Expand All @@ -13,11 +39,176 @@ class Filter extends MetadataType {
* but only with some of the fields. So it is needed to loop over
* Filters with the endpoint /automation/v1/filters/{id}
* @param {String} retrieveDir Directory where retrieved metadata directory will be saved
* @returns {Promise} Promise
* @returns {Promise<{metadata:FilterMap,type:string}>} Promise of items
*/
static async retrieve(retrieveDir) {
return super.retrieveREST(retrieveDir, '/automation/v1/filters/', null);
}
/**
* manages post retrieve steps
* @param {FilterItem} item a single record
* @returns {FilterItem} parsed metadata definition
*/
static postRetrieveTasks(item) {
return this.parseMetadata(item);
}
/**
* parses retrieved Metadata before saving
* @param {FilterItem} metadata a single record
* @returns {FilterItem} parsed metadata definition
*/
static parseMetadata(metadata) {
try {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this error is already handled in the saveResults method (only failing the individual record) so should be avoided here as will likely hide other errors/require additional checks in saveResults method

// folder
metadata.r__folder_Path = Util.getFromCache(
this.cache,
'folder',
metadata.categoryId,
'ID',
'Path'
);
delete metadata.categoryId;

// filterDefinition
metadata.r__filterDefinition_CustomerKey = Util.getFromCache(
this.cache,
'filterDefinition',
metadata.filterDefinitionId,
'id',
'key'
);
delete metadata.filterDefinitionId;

// source
if (metadata.sourceTypeId === 1) {
// list
} else if (metadata.sourceTypeId === 2) {
// dataExtension
metadata.r__source_dataExtension_CustomerKey = Util.getFromCache(
this.cache,
'dataExtension',
metadata.sourceObjectId,
'ObjectID',
'CustomerKey'
);
delete metadata.sourceObjectId;
delete metadata.sourceTypeId;
} else {
Util.logger.error(
`Filter '${metadata.name}' (${metadata.customerKey}): Unsupported source type ${
metadata.sourceTypeId
}=${dataTypes[metadata.sourceTypeId]}`
);
}

// target
if (metadata.destinationTypeId === 1) {
// list
} else if (metadata.destinationTypeId === 2) {
// dataExtension
metadata.r__destination_dataExtension_CustomerKey = Util.getFromCache(
this.cache,
'dataExtension',
metadata.destinationObjectId,
'ObjectID',
'CustomerKey'
);
delete metadata.destinationObjectId;
delete metadata.destinationTypeId;
} else {
Util.logger.error(
`Filter '${metadata.name}' (${
metadata.customerKey
}): Unsupported destination type ${metadata.destinationTypeId}=${
dataTypes[metadata.destinationTypeId]
}`
);
}
} catch (ex) {
Util.logger.error(`Filter '${metadata.name}' (${metadata.customerKey}): ${ex.message}`);
}
return metadata;
}
/**
* prepares a record for deployment
* @param {FilterItem} metadata a single record
* @returns {Promise<FilterItem>} Promise of updated single record
*/
static async preDeployTasks(metadata) {
// folder
if (metadata.r__folder_Path) {
metadata.categoryId = Util.getFromCache(
this.cache,
'folder',
metadata.r__folder_Path,
'Path',
'ID'
);
delete metadata.r__folder_Path;
}

// filterDefinition
if (metadata.r__filterDefinition_CustomerKey) {
metadata.filterDefinitionId = Util.getFromCache(
this.cache,
'filterDefinition',
metadata.r__filterDefinition_CustomerKey,
'CustomerKey',
'ObjectID'
);
delete metadata.r__filterDefinition_CustomerKey;
}

// source
if (metadata.sourceTypeId === 1) {
// list
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if list then?

} else if (metadata.r__source_dataExtension_CustomerKey) {
// dataExtension
metadata.sourceObjectId = Util.getFromCache(
this.cache,
'dataExtension',
metadata.r__source_dataExtension_CustomerKey,
'CustomerKey',
'ObjectID'
);
metadata.sourceTypeId = 2;
delete metadata.r__source_dataExtension_CustomerKey;
} else {
// assume the type id is still in the metadata
throw new Error(
`Filter '${metadata.name}' (${metadata.customerKey}): Unsupported source type ${
metadata.sourceTypeId
}=${dataTypes[metadata.sourceTypeId]}`
);
}

// target
if (metadata.destinationTypeId === 1) {
// list
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then?

} else if (metadata.r__destination_dataExtension_CustomerKey) {
// dataExtension
metadata.destinationObjectId = Util.getFromCache(
this.cache,
'dataExtension',
metadata.r__destination_dataExtension_CustomerKey,
'CustomerKey',
'ObjectID'
);
metadata.destinationTypeId = 2;
delete metadata.r__destination_dataExtension_CustomerKey;
} else {
// assume the type id is still in the metadata
throw new Error(
`Filter '${metadata.name}' (${
metadata.customerKey
}): Unsupported destination type ${metadata.destinationTypeId}=${
dataTypes[metadata.destinationTypeId]
}`
);
}

return metadata;
}
}

// Assign definition to static attributes
Expand Down
Loading