Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UI-9566 - Ensure a best effort migration of the KPI titles even when some titles could not be successfully migrated. #129

Merged
merged 23 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
dd3f796
UI-9566 - Ensure kpis with titles corresponding to empty measures hav…
NZepeda Jul 29, 2024
620462a
Minor improvement
NZepeda Jul 30, 2024
88bab6b
Drop kpi titles that have an empty location.
NZepeda Jul 30, 2024
5e66d80
Update src/4.3_to_5.0/getMigratedKpiTitles.ts
NZepeda Jul 30, 2024
206fc72
Update test resource
NZepeda Jul 30, 2024
fd9896d
Merge branch 'task/nze/UI-9566-support-migrating-empty-titles' of git…
NZepeda Jul 30, 2024
3876da1
Apply suggestions from code review
NZepeda Jul 31, 2024
85700d8
Update tests
NZepeda Jul 31, 2024
96a710a
UI-9566 - Ensure a best effort migration of the kpi titles. If there …
NZepeda Jul 31, 2024
6f50c45
Update src/migrateContentServer.test.ts
NZepeda Jul 31, 2024
71fbb2d
Fixups
NZepeda Jul 31, 2024
1d5c86c
Merge branch 'task/nze/UI-9566-support-migrating-empty-titles' of git…
NZepeda Jul 31, 2024
643e807
Merge main
NZepeda Aug 1, 2024
53e4964
Use an empty folder test resource and a generator function to create …
NZepeda Aug 14, 2024
22b5547
Update previously added tests to utilize the new test resource function
NZepeda Aug 14, 2024
31e1754
Remove the now unnecessary test resource files
NZepeda Aug 14, 2024
bdcdec4
Move the location of addLegacyBookmarkToUIFolder to the test resource…
NZepeda Sep 18, 2024
1eb0b41
The empty legacy ui folder test resource is truly empty
NZepeda Sep 18, 2024
22ecfc2
Split out the invalid legacy bookmarks and their folder into their ow…
NZepeda Sep 18, 2024
1cde91f
Split out the legacy kpi bookmark and folder into their own resource …
NZepeda Sep 18, 2024
2280099
Create the smallLegacy ui folder with the new add legacy bookmark to …
NZepeda Sep 18, 2024
26adf4b
Apply suggestions from code review
NZepeda Sep 18, 2024
dee3129
Ensure the test snapshots have not lost data
NZepeda Sep 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 5 additions & 92 deletions src/4.3_to_5.0/__snapshots__/migrate_43_to_50.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -2247,12 +2247,8 @@ exports[`migrate_43_to_50 returns a valid ActiveUI5 /ui folder on a small input
"content": "{"name":"hidden grand totals"}",
"isDirectory": true,
"lastEditor": "admin",
"owners": [
"admin",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here too, it would be nice to not lose this, in order to ensure that the migration script does not erase bookmark permissions.

],
"readers": [
"admin",
],
"owners": [],
"readers": [],
"timestamp": 1607879735685,
},
},
Expand All @@ -2262,12 +2258,8 @@ exports[`migrate_43_to_50 returns a valid ActiveUI5 /ui folder on a small input
"canWrite": true,
"isDirectory": true,
"lastEditor": "admin",
"owners": [
"admin",
],
"readers": [
"admin",
],
"owners": [],
"readers": [],
"timestamp": 1607879735685,
},
},
Expand Down Expand Up @@ -2357,6 +2349,7 @@ exports[`migrate_43_to_50 returns a valid ActiveUI5 /ui folder on a small input
"organization_settings": {
"entry": {
"content": "{}",
"isDirectory": false,
"owners": [
"ROLE_CS_ROOT",
],
Expand Down Expand Up @@ -2404,86 +2397,6 @@ exports[`migrate_43_to_50 returns a valid ActiveUI5 /ui folder on a small input
},
},
"users": {
"children": {
"admin": {
"children": {
"activity": {
"entry": {
"content": "{"userFilters":["[Geography].[City].[ALL].[AllMember].[Berlin]"]}",
"owners": [
"admin",
],
"readers": [
"admin",
],
},
},
"settings": {
"entry": {
"content": "{"theme":"dark-activeviam","search.maxResults":10,"userFilters.areEnabled":true,"drillthrough.defaultSelectedColumns":{"EquityDerivativesCube":["delta","gamma","pnlVega","Desk","Currency","Date","HostName"],"EquityDerivativesCubeDist":["BumpedMtmDown","ProductQtyMultiplier","vega","rho","productId","pnlVega","pnlDelta","pnl","gamma","delta","TradeId","ProductBaseMtm"]}}",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, it looks like we're losing the settings conversion part of this test snapshot. It's probably because some settings that were part of a legacy UI folder test resource were lost in the process. Can you please make sure that we don't lose this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added a new commit to avoid losing this information. However, it means re-adding the settings to the emptyLegacyUIFolder since we are now generating test resources such as smallLegacyUIFolder via the emptyLegacyUIFolder and addLegacyBookmarkToUIFolder.

"owners": [
"admin",
],
"readers": [
"admin",
],
},
},
},
"entry": {
"canRead": true,
"canWrite": true,
"isDirectory": true,
"lastEditor": "admin",
"owners": [
"admin",
],
"readers": [
"admin",
],
"timestamp": 1607879735685,
},
},
"user1": {
"children": {
"activity": {
"entry": {
"content": "{}",
"owners": [
"user1",
],
"readers": [
"user1",
],
},
},
"settings": {
"entry": {
"content": "{}",
"owners": [
"user1",
],
"readers": [
"user1",
],
},
},
},
"entry": {
"canRead": true,
"canWrite": true,
"isDirectory": true,
"lastEditor": "admin",
"owners": [
"user1",
],
"readers": [
"user1",
],
"timestamp": 1607879735685,
},
},
},
"entry": {
"isDirectory": true,
"owners": [
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import { ContentRecord } from "@activeviam/activeui-sdk-5.1";
import { produce } from "immer";

/**
* Returns a new `legacyUIFolder` with the given `legacyBookmarksToAdd` added.
* Useful for creating legacyUIFolders for tests.
* Does not mutate `legacyUIFolder`.
*/
export function addLegacyBookmarkToUIFolder(
legacyUIFolder: ContentRecord,
legacyBookmarksToAdd: { [id: string]: any },
): ContentRecord<any, "response"> {
const content = legacyUIFolder.children?.bookmarks.children?.content;
const structure = legacyUIFolder.children?.bookmarks.children?.structure;

if (!content || !structure) {
throw new Error(
"Expected `legacyUIFolder` to contain the `content` and `structure` properties. Please ensure these properties exist",
);
}

return produce(legacyUIFolder, (draftFolder) => {
const existingBookmarks =
draftFolder.children!.bookmarks.children!.content!.children || {};

// Add the bookmark content
draftFolder.children!.bookmarks.children!.content!.children = {
...existingBookmarks,
...legacyBookmarksToAdd,
};

const keysOfBookmarksToAdd = Object.keys(legacyBookmarksToAdd);

const bookmarkStructureToAdd = keysOfBookmarksToAdd.reduce(
(acc: { [key: string]: ContentRecord }, key: string) => {
acc[key] = {
entry: {
isDirectory: true,
owners: [],
readers: [],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
};
return acc;
},
{},
);

const existingStructure =
draftFolder.children!.bookmarks.children!.structure!.children || {};

// Add the bookmark structure
draftFolder.children!.bookmarks.children!.structure!.children = {
...existingStructure,
...bookmarkStructureToAdd,
};
});
}
112 changes: 112 additions & 0 deletions src/4.3_to_5.0/__test_resources__/emptyLegacyUIFolder.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
/**
* Contains an empty /ui folder of a Content Server, useful for injecting content to be used in unit tests.
*/
export const emptyLegacyUIFolder = {
entry: {
isDirectory: true,
owners: ["admin"],
readers: ["admin"],
timestamp: 1607879725132,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
children: {
bookmarks: {
entry: {
isDirectory: true,
owners: ["ROLE_CS_ROOT"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
children: {
content: {
entry: {
isDirectory: true,
owners: ["ROLE_USER"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
children: {},
},
i18n: {
entry: {
isDirectory: true,
owners: ["ROLE_CS_ROOT"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
children: {
"en-US": {
entry: {
isDirectory: true,
owners: ["ROLE_CS_ROOT"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
},
"fr-FR": {
entry: {
isDirectory: true,
owners: ["ROLE_CS_ROOT"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
},
},
},
structure: {
entry: {
isDirectory: true,
owners: ["ROLE_USER"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
children: {},
},
},
},
settings: {
entry: {
isDirectory: true,
owners: ["ROLE_CS_ROOT"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
children: {},
},
version: {
entry: {
content: '{"package":"4.3.8","contentServerApi":"0.1.0"}',
isDirectory: false,
owners: ["ROLE_CS_ROOT"],
readers: ["ROLE_USER"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
},
},
};
37 changes: 37 additions & 0 deletions src/4.3_to_5.0/__test_resources__/invalidLegacyWidgets.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
const widgetWithInvalidContainerKey = {
entry: {
content:
'{"description":"Widget with invalid container key","name":"Invalid widget","type":"container","value":{"style":{},"showTitleBar":false,"containerKey":"invalid-container-key","body":{"serverUrl":"","mdx":"SELECT NON EMPTY [Measures].[contributors.COUNT] ON COLUMNS FROM [EquityDerivativesCube] WHERE [Geography].[City].[ALL].[AllMember].[New York] CELL PROPERTIES VALUE, FORMATTED_VALUE, BACK_COLOR, FORE_COLOR, FONT_FLAGS","contextValues":{},"updateMode":"once","ranges":{}}}}',
isDirectory: false,
owners: ["admin"],
readers: ["admin"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
};

const widgetWithFilterOnInvalidHierarchy = {
entry: {
content:
'{"description":"Widget with filter on invalid hierarchy","name":"Invalid widget","type":"container","value":{"style":{},"showTitleBar":false,"containerKey":"pivot-table","body":{"serverUrl":"","mdx":"SELECT NON EMPTY [Measures].[contributors.COUNT] ON COLUMNS FROM [EquityDerivativesCube] WHERE [Geography].[InvalidHierarchy].[ALL].[AllMember].[Member] CELL PROPERTIES VALUE, FORMATTED_VALUE, BACK_COLOR, FORE_COLOR, FONT_FLAGS","contextValues":{},"updateMode":"once","ranges":{}}}}',
isDirectory: false,
owners: ["admin"],
readers: ["admin"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
};

/**
* Contains two legacy bookmark widgets.
* 1. Widget with an invalid container container key.
* 2. Widget with a filter on an invalid hierarchy.
*/
export const invalidLegacyWidgets = {
"158": widgetWithInvalidContainerKey,
"1231": widgetWithFilterOnInvalidHierarchy,
};
16 changes: 16 additions & 0 deletions src/4.3_to_5.0/__test_resources__/legacyDashboardBookmark.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/**
* Content entry representing a legacy dashboard containing a single pivot table.
* Useful for unit tests.
*/
export const legacyDashboardBookmark = {
entry: {
content: `{"name":"hidden grand totals","type":"container","value":{"style":{},"showTitleBar":false,"body":{"pages":[{"content":[{"key":"1","bookmark":{"name":"Untitled Pivot Table","type":"container","value":{"style":{},"showTitleBar":true,"body":{"serverUrl":"","mdx":"SELECT NON EMPTY Crossjoin(Hierarchize(DrilldownLevel([Geography].[City].[ALL].[AllMember])), Hierarchize(DrilldownLevel([Currency].[Currency].[ALL].[AllMember]))) ON ROWS, NON EMPTY [Measures].[contributors.COUNT] ON COLUMNS FROM [EquityDerivativesCube] CELL PROPERTIES VALUE, FORMATTED_VALUE, BACK_COLOR, FORE_COLOR, FONT_FLAGS","contextValues":{"mdx.hiddengrandtotals":"1"},"updateMode":"once","ranges":{"row":{"chunkSize":2000,"thresholdPercentage":0.1},"column":{"chunkSize":50,"thresholdPercentage":0.2}},"configuration":{"tabular":{"pinnedHeaderSelector":"member","sortingMode":"non-breaking","addButtonFilter":"numeric","cellRenderers":["tree-layout"],"statisticsShown":true,"columnsGroups":[{"captionProducer":"firstColumn","cellFactory":"kpi-status","selector":"kpi-status"},{"captionProducer":"firstColumn","cellFactory":"lookup","selector":"lookup"},{"captionProducer":"expiry","cellFactory":"expiry","selector":"kpi-expiry"},{"captionProducer":"columnMerge","cellFactory":{"args":{},"key":"treeCells"},"selector":"member"}],"hideAddButton":true,"defaultOptions":{},"expansion":{"automaticExpansion":true}}}},"containerKey":"pivot-table"},"writable":true}},{"key":"2","bookmark":{"name":"Untitled Chart","type":"container","value":{"style":{},"showTitleBar":true,"body":{"configuration":{"type":"plotly-line-chart","mapping":{"xAxis":["[Currency].[Currency].[Currency]"],"values":["[Measures].[pnl.FOREX]"],"splitBy":["[Booking].[Desk].[LegalEntity]"],"horizontalSubplots":[],"verticalSubplots":[]},"switchedTo":"plotly-clustered-column-chart"},"query":{"serverUrl":"","mdx":"SELECT NON EMPTY Crossjoin(Hierarchize(DrilldownLevel([Currency].[Currency])), Hierarchize(DrilldownLevel([Booking].[Desk].[ALL].[AllMember]))) ON ROWS, NON EMPTY [Measures].[pnl.FOREX] ON COLUMNS FROM [EquityDerivativesCube]","contextValues":{},"updateMode":"once"}},"containerKey":"chart"},"writable":true}}],"layout":{"children":{"0":{"ck":"2"},"1":{"ck":"1"}},"direction":"row"},"name":"Page 1","filters":{"EquityDerivativesCube":[]}}]},"containerKey":"dashboard"}}`,
isDirectory: false,
owners: ["admin"],
readers: ["admin"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/**
* Legacy content entry representing a bookmark of an invalid dashboard.
NZepeda marked this conversation as resolved.
Show resolved Hide resolved
* Useful for unit tests.
*/
export const legacyInvalidDashboardBookmark = {
entry: {
content:
'{"name":"hidden grand totals","type":"container","value":{"style":{},"showTitleBar":false,"body":{"pages":[{"content":[{"key":"1","bookmark":{"name":"Untitled Pivot Table","type":"container","value":{"style":{},"showTitleBar":true,"body":{"serverUrl":"","mdx":"SELECT NON EMPTY Crossjoin(Hierarchize(DrilldownLevel([Geography].[City].[ALL].[AllMember])), Hierarchize(DrilldownLevel([Currency].[Currency].[ALL].[AllMember]))) ON ROWS, NON EMPTY [Measures].[contributors.COUNT] ON COLUMNS FROM [foo] CELL PROPERTIES VALUE, FORMATTED_VALUE, BACK_COLOR, FORE_COLOR, FONT_FLAGS","contextValues":{"mdx.hiddengrandtotals":"1"},"updateMode":"once","ranges":{"row":{"chunkSize":2000,"thresholdPercentage":0.1},"column":{"chunkSize":50,"thresholdPercentage":0.2}},"configuration":{"tabular":{"pinnedHeaderSelector":"member","sortingMode":"non-breaking","addButtonFilter":"numeric","cellRenderers":["tree-layout"],"statisticsShown":true,"columnsGroups":[{"captionProducer":"firstColumn","cellFactory":"kpi-status","selector":"kpi-status"},{"captionProducer":"firstColumn","cellFactory":"lookup","selector":"lookup"},{"captionProducer":"expiry","cellFactory":"expiry","selector":"kpi-expiry"},{"captionProducer":"columnMerge","cellFactory":{"args":{},"key":"treeCells"},"selector":"member"}],"hideAddButton":true,"defaultOptions":{},"expansion":{"automaticExpansion":true}}}},"containerKey":"pivot-table"},"writable":true}},{"key":"2","bookmark":{"name":"Untitled Chart","type":"container","value":{"style":{},"showTitleBar":true,"body":{"configuration":{"type":"plotly-line-chart","mapping":{"xAxis":["[Currency].[Currency].[Currency]"],"values":["[Measures].[pnl.FOREX]"],"splitBy":["[Booking].[Desk].[LegalEntity]"],"horizontalSubplots":[],"verticalSubplots":[]},"switchedTo":"plotly-clustered-column-chart"},"query":{"serverUrl":"","mdx":"SELECT NON EMPTY Crossjoin(Hierarchize(DrilldownLevel([Currency].[Currency])), Hierarchize(DrilldownLevel([Booking].[Desk].[ALL].[AllMember]))) ON ROWS, NON EMPTY [Measures].[pnl.FOREX] ON COLUMNS FROM [EquityDerivativesCube]","contextValues":{},"updateMode":"once"}},"containerKey":"chart"},"writable":true}}],"layout":{"children":{"0":{"ck":"2"},"1":{"ck":"1"}},"direction":"row"},"name":"Page 1","filters":{"EquityDerivativesCube":[]}}]},"containerKey":"dashboard"}}',
isDirectory: false,
owners: ["admin"],
readers: ["admin"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/**
* Content entry representing a legacy filter containing an invalid property
NZepeda marked this conversation as resolved.
Show resolved Hide resolved
* Useful for unit tests.
*/
export const legacyInvalidFilterBookmark = {
entry: {
content:
'{"name":"AUI4 filter","type":"mdx","invalidvalue":{"shouldReplace":true,"type":"filter","mdx":"{[Geography].[City].[ALL].[AllMember].[Berlin], [Geography].[City].[ALL].[AllMember].[London]}","cube":"EquityDerivativesCube"}}',
isDirectory: false,
owners: ["admin"],
readers: ["admin"],
timestamp: 1607879735685,
lastEditor: "admin",
canRead: true,
canWrite: true,
},
};
Loading
Loading