Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic Backend Configuration #732

Merged
merged 93 commits into from
Jun 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
c3f84d5
Basic backend configuration setup
garrettmflynn Apr 5, 2024
3713f6c
Allow for skipping configuration
garrettmflynn Apr 5, 2024
2d165bf
Fix skip
garrettmflynn Apr 5, 2024
98c3814
Fixed configuration example when passing
garrettmflynn Apr 5, 2024
f1e4c6b
Add exceptions as comments
garrettmflynn Apr 5, 2024
e4966b0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 5, 2024
2f2ed5f
Basic backend configuration rendering and editing
garrettmflynn Apr 9, 2024
8d839a6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 9, 2024
d687e5f
Fix list content rendering and provide notes for future changes
garrettmflynn Apr 9, 2024
16150a1
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn Apr 9, 2024
877a80b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 9, 2024
e7837d3
Improve messages and skipping for backend configuration
garrettmflynn Apr 9, 2024
5fdfff9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 9, 2024
bfcdd3d
Provide metadata to display adjacent to dataset configuration options
garrettmflynn Apr 9, 2024
ea4c26f
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn Apr 9, 2024
21afe3c
Merge branch 'main' into backend-configuration
garrettmflynn Apr 9, 2024
fd2d6df
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 9, 2024
e5520d6
Merge branch 'allow-starting-generated-pipelines' into backend-config…
garrettmflynn Apr 9, 2024
b3c1012
Allow specifying a backend in preform and use throughout the pipeline
garrettmflynn Apr 9, 2024
cd6d7b2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 9, 2024
68c5576
Streamline configuration options request
garrettmflynn Apr 10, 2024
5ec5159
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn Apr 10, 2024
626f1fb
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 10, 2024
7992db0
Allow session-specific backend selection. Additional improvements bas…
garrettmflynn Apr 10, 2024
54b18f7
Dynamically update descriptions based on dtype itemsize
garrettmflynn Apr 10, 2024
9f2d4c6
Enumerate strict string enums in a select element. Confirm full conve…
garrettmflynn Apr 11, 2024
75cea81
Dynamically change schema constraints alongside descriptions
garrettmflynn Apr 11, 2024
0b9189a
Only skip forwards
garrettmflynn Apr 11, 2024
b123bfb
Add updates from GUIDE meeting
garrettmflynn Apr 11, 2024
2dee3f0
Resolve schema from backend
garrettmflynn Apr 11, 2024
d2f32a7
Allow for validation of configuration options without full conversion
garrettmflynn Apr 11, 2024
bc027b9
Fix save
garrettmflynn Apr 11, 2024
027c7d5
Merge branch 'main' into backend-configuration
garrettmflynn Apr 18, 2024
dd5db4c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 18, 2024
c62f873
Create array editor for each axis. Change to native JSON Schema minim…
garrettmflynn Apr 18, 2024
8f383f6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 18, 2024
a459f02
Allow basic backend configuration to pass while hiding complicated op…
garrettmflynn Apr 25, 2024
473d6b2
Only re-render headers and section availability, not section dropdown…
garrettmflynn Apr 25, 2024
fb5cc35
Merge branch 'main' into backend-configuration
garrettmflynn Apr 26, 2024
dcdaf93
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 26, 2024
d438b6b
Update create.json
garrettmflynn May 29, 2024
3c2b2ef
Merge branch 'main' into backend-configuration
garrettmflynn May 31, 2024
27c69a3
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 31, 2024
b089798
Fix merge imports
garrettmflynn May 31, 2024
abbf995
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn May 31, 2024
7cfc2be
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 31, 2024
459ff84
Fix rendering. Remove extra file and copy changes to live file
garrettmflynn May 31, 2024
86202c9
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn May 31, 2024
b73a1fc
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 31, 2024
f476488
Fix functionality
garrettmflynn May 31, 2024
b64059c
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn May 31, 2024
da45584
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 31, 2024
14671e4
Merge branch 'main' into backend-configuration
garrettmflynn Jun 3, 2024
7497629
Update logging and update management
garrettmflynn Jun 3, 2024
e54b0f5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
8a9261a
Improve page transition awaiting and workflow configuration resolution
garrettmflynn Jun 3, 2024
4149005
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn Jun 3, 2024
25f0814
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
a808cdf
Ensure page render on each transition
garrettmflynn Jun 3, 2024
6f6aed7
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn Jun 3, 2024
86805ad
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
4bf639d
Always use run_conversion
garrettmflynn Jun 3, 2024
81cf131
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
dc0e598
refactor configuraiton function
CodyCBakerPhD Jun 3, 2024
1bbd42c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
5fc9132
more refactoring
Jun 3, 2024
f276c06
removed unused helper
Jun 3, 2024
0b347d5
Fix logger
garrettmflynn Jun 3, 2024
a5eaeec
Merge branch 'backend-configuration-uniform' of https://github.com/Ne…
garrettmflynn Jun 3, 2024
dbb8205
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
3cbc7d3
Notify of configuration issue
garrettmflynn Jun 3, 2024
ef72ae8
Merge branch 'backend-configuration-uniform' of https://github.com/Ne…
garrettmflynn Jun 3, 2024
5fa437d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 3, 2024
5ab0c16
restore stattr
Jun 3, 2024
2c739b7
Merge branch 'backend-configuration-uniform' of https://github.com/ne…
Jun 3, 2024
dcfd4b2
Merge branch 'main' into backend-configuration
CodyCBakerPhD Jun 4, 2024
f6feef7
Merge branch 'backend-configuration' into backend-configuration-uniform
CodyCBakerPhD Jun 4, 2024
dde939b
fix for stub mode
CodyCBakerPhD Jun 4, 2024
2302d33
Fix display issues suggested by Cody
garrettmflynn Jun 4, 2024
336e040
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 4, 2024
8060c4d
Merge branch 'backend-configuration' into backend-configuration-uniform
CodyCBakerPhD Jun 4, 2024
0024572
Merge pull request #821 from NeurodataWithoutBorders/backend-configur…
CodyCBakerPhD Jun 4, 2024
c4114ab
Fix tests
garrettmflynn Jun 4, 2024
7eb1eb0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 4, 2024
d6a05ac
Ensure checking state properly
garrettmflynn Jun 4, 2024
8e7c997
Merge branch 'main' into backend-configuration
CodyCBakerPhD Jun 4, 2024
155de1d
Merge branch 'main' into backend-configuration
CodyCBakerPhD Jun 5, 2024
a9b7f37
Merge branch 'main' into backend-configuration
garrettmflynn Jun 5, 2024
a3cf150
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 5, 2024
df7fde0
Fix merge
garrettmflynn Jun 5, 2024
f92ddaf
Merge branch 'backend-configuration' of https://github.com/NeurodataW…
garrettmflynn Jun 5, 2024
598724b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 5, 2024
00c6516
Update src/electron/frontend/core/components/pages/guided-mode/setup/…
CodyCBakerPhD Jun 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion src/electron/frontend/core/components/BasicTable.js
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,6 @@ export class BasicTable extends LitElement {
};

#readTSV(text) {
console.log(text, text.split("\n"));
let data = text.split("\n").map((row) =>
row.split("\t").map((v) => {
try {
Expand Down
12 changes: 6 additions & 6 deletions src/electron/frontend/core/components/Dashboard.js
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ export class Dashboard extends LitElement {
else if (typeof page === "object") return this.getPage(Object.values(page)[0]);
}

updateSections({ sidebar = true, main = false } = {}, globalState = this.page.info.globalState) {
updateSections({ sidebar = true, main = false, header = false } = {}, globalState = this.page.info.globalState) {
const info = this.page.info;
let parent = info.parent;

Expand All @@ -199,7 +199,7 @@ export class Dashboard extends LitElement {
page: this.page,
sections,
});
}
} else if (header) this.main.header.sections = sections; // Update header sections

return sections;
}
Expand Down Expand Up @@ -332,7 +332,7 @@ export class Dashboard extends LitElement {
let state = globalState.sections[section];
if (!state)
state = globalState.sections[section] = {
open: false,
open: undefined,
active: false,
pages: {},
};
Expand Down Expand Up @@ -378,9 +378,9 @@ export class Dashboard extends LitElement {
this.#transitionPromise.value ??
(this.#transitionPromise.value = new Promise(
(resolve) =>
(this.#transitionPromise.trigger = (value) => {
delete this.#transitionPromise.value;
resolve(value);
(this.#transitionPromise.trigger = (v) => {
this.#transitionPromise.value = null; // Reset promise
resolve(v);
})
));

Expand Down
48 changes: 28 additions & 20 deletions src/electron/frontend/core/components/InstanceManager.js
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ export class InstanceManager extends LitElement {
.controls > div {
display: flex;
gap: 10px;
align-items: center;
}

#new-info {
Expand Down Expand Up @@ -357,6 +358,11 @@ export class InstanceManager extends LitElement {
});
}

const controls = this.#controls();
const controlDiv = this.shadowRoot.querySelector(".controls > div");
controlDiv.innerHTML = "";
controlDiv.append(...controls);

this.#onSelected();
};

Expand All @@ -370,6 +376,27 @@ export class InstanceManager extends LitElement {

#hasMultiple = () => this.#items.length > 1;

#controls = () => {
return this.controls.map((item) => {
if (item instanceof HTMLElement)
return item; // Custom element
else if (typeof item === "function") return item(this.#selected); // Function

// Button configuration
const { name, icon, primary, onClick } = item;
return html`<nwb-button
size="small"
icon=${icon}
.primary=${primary ?? false}
@click=${function () {
const activeContentElement = this.shadowRoot.querySelector("#instance-display > div:not([hidden])");
onClick.call(this, activeContentElement.getAttribute("data-instance"), activeContentElement);
}}
>${name}</nwb-button
>`;
});
};

render() {
this.#info = {};
this.#items = [];
Expand Down Expand Up @@ -445,26 +472,7 @@ export class InstanceManager extends LitElement {
<div id="content">
<div class="controls">
<span id="selectedName"></span>
<div>
${this.controls.map(({ name, icon, primary, onClick }) => {
return html`<nwb-button
size="small"
icon=${icon}
.primary=${primary ?? false}
@click=${function () {
const activeContentElement = this.shadowRoot.querySelector(
"#instance-display > div:not([hidden])"
);
onClick.call(
this,
activeContentElement.getAttribute("data-instance"),
activeContentElement
);
}}
>${name}</nwb-button
>`;
})}
</div>
<div>${this.#controls()}</div>
</div>

<div id="instance-display">
Expand Down
3 changes: 2 additions & 1 deletion src/electron/frontend/core/components/JSONSchemaForm.js
Original file line number Diff line number Diff line change
Expand Up @@ -581,7 +581,8 @@ export class JSONSchemaForm extends LitElement {

const allErrors = flaggedInputs
.map((inputElement) => Array.from(inputElement.nextElementSibling.children).map((li) => li.message))
.flat();
.flat()
.filter((v) => !!v);

const nMissingRequired = allErrors.reduce((acc, curr) => {
return (acc += curr.includes(this.#isARequiredPropertyString) ? 1 : 0);
Expand Down
154 changes: 125 additions & 29 deletions src/electron/frontend/core/components/JSONSchemaInput.js
Original file line number Diff line number Diff line change
Expand Up @@ -501,6 +501,19 @@ export class JSONSchemaInput extends LitElement {
};
}

#description;
get description() {
return this.#description ?? this.schema.description;
}

set description(value) {
this.#description = value;

const descriptionEl = this.shadowRoot.querySelector(".guided--text-input-instructions");
if (!descriptionEl) return;
descriptionEl.innerHTML = value;
}

// Enforce dynamic required properties
attributeChangedCallback(key, _, latest) {
super.attributeChangedCallback(...arguments);
Expand Down Expand Up @@ -537,6 +550,7 @@ export class JSONSchemaInput extends LitElement {
controls = [];
// required;
validateOnChange = true;
allowNaN = true;

constructor(props = {}) {
super();
Expand Down Expand Up @@ -664,7 +678,11 @@ export class JSONSchemaInput extends LitElement {
>${(schema.title ? unsafeHTML(schema.title) : null) ?? header(this.path.slice(-1)[0])}</label
>`
: ""}
<main>${input}${this.controls ? html`<div id="controls">${this.controls}</div>` : ""}</main>
<main>
${input}${this.controls && this.controls.length
? html`<div id="controls">${this.controls}</div>`
: ""}
</main>
${descriptionHTML}
</div>
`;
Expand Down Expand Up @@ -857,6 +875,17 @@ export class JSONSchemaInput extends LitElement {
#render() {
const { validateOnChange, schema, path: fullPath } = this;

// Resolve anyof inside the schema
const anyOf = schema.anyOf;
if (anyOf) {
delete schema.anyOf;
for (let key in anyOf[0]) {
schema[key] = anyOf[0][key];
}

// schema = {...schema, ...anyOf[0]}
}

this.removeAttribute("data-table");

// Do your best to fill in missing schema values
Expand Down Expand Up @@ -907,26 +936,53 @@ export class JSONSchemaInput extends LitElement {
return filesystemSelectorElement;
};

// Transform to single item if maxItems is 1
if (isArray && schema.maxItems === 1 && !isTable) {
return new JSONSchemaInput({
value: this.value?.[0],
schema: {
...schema.items,
strict: schema.strict,
},
path: fullPath,
validateEmptyValue: this.validateEmptyValue,
required: this.required,
validateOnChange: () => (validateOnChange ? this.#triggerValidation(name, path) : ""),
form: this.form,
onUpdate: (value) => this.#updateData(fullPath, [value]),
// Transform to single item if maxItems is 1 OR the array has a fixed length
if (
isArray &&
(schema.maxItems === 1 || (schema.maxItems && schema.minItems && schema.maxItems === schema.minItems)) &&
!isTable
) {
const len = schema.maxItems ?? 1;
const array = this.value ?? [];

// JSONified arrays will convert undefined to null
const jsonify = (value) => (value === undefined ? null : value);
const jsonschemify = (value) => (value === null ? undefined : value);

this.required = false;

const inputs = Array.from({ length: len }).map((_, i) => {
const input = new JSONSchemaInput({
value: jsonify(array[i]),
schema: {
...schema.items,
strict: schema.strict,
},
path: fullPath,
validateEmptyValue: this.validateEmptyValue,
required: this.required,
allowNaN: false,
validateOnChange: () => (validateOnChange ? this.#triggerValidation(name, path) : ""),
form: this.form,
onUpdate: (value) => {
array[i] = jsonschemify(value);
this.#updateData(fullPath, [...array]);
},
});

array[i] = jsonschemify(array[i]);

return input;
});

return inputs;
}

if (isArray || canAddProperties) {
// if ('value' in this && !Array.isArray(this.value)) this.value = [ this.value ]

const editableInline = ["string", "number"];

const allowPatternProperties = isPatternProperties(this.pattern);
const allowAdditionalProperties = isAdditionalProperties(this.pattern);

Expand All @@ -949,12 +1005,18 @@ export class JSONSchemaInput extends LitElement {
const fileSystemFormat = isFilesystemSelector(name, itemSchema?.format);
if (fileSystemFormat) return createFilesystemSelector(fileSystemFormat);
// Create tables if possible
else if (itemSchema?.type === "string" && !itemSchema.properties) {
else if (editableInline.includes(itemSchema?.type) && !itemSchema.properties) {
const postprocess = (v) => {
if (itemSchema?.type === "number") return parseFloat(v);
else return v;
};

const list = new List({
items: this.value,
emptyMessage: "No items",
emptyMessage: schema.empty ?? "No items selected.",
unordered: false,
onChange: ({ items }) => {
this.#updateData(fullPath, items.length ? items.map((o) => o.value) : undefined);
this.#updateData(fullPath, items.length ? items.map((o) => postprocess(o.value)) : undefined);
if (validateOnChange) this.#triggerValidation(name, path);
},
});
Expand All @@ -972,6 +1034,7 @@ export class JSONSchemaInput extends LitElement {
};
}),
value: this.value,
placeholder: schema.placeholder,
listMode: schema.strict === false ? "click" : "append",
showAllWhenEmpty: false,
onSelect: async ({ label, value }) => {
Expand All @@ -981,12 +1044,16 @@ export class JSONSchemaInput extends LitElement {
},
});

search.style.height = "auto";
return html`<div style="width: 100%;">${search}${list}</div>`;
Object.assign(search.style, { width: "100%" });

return html`<div style="width: 100%; margin-bottom: 5px;">
<div style="margin-bottom: 10px;">${search}</div>
${list}
</div>`;
} else {
const input = document.createElement("input");
input.classList.add("guided--input");
input.placeholder = "Provide an item for the list";
input.placeholder = schema.placeholder ?? "Provide an item for the list";

const submitButton = new Button({
label: "Submit",
Expand All @@ -1006,11 +1073,13 @@ export class JSONSchemaInput extends LitElement {
});

return html`<div
style="width: 100%;"
style="width: 100%; margin-bottom: 5px;"
class="schema-input"
@change=${() => validateOnChange && this.#triggerValidation(name, path)}
>
<div style="display: flex; gap: 10px; align-items: center;">${input}${submitButton}</div>
<div style="display: flex; gap: 10px; align-items: center; margin-bottom: 10px;">
${input}${submitButton}
</div>
${list}
</div>`;
}
Expand Down Expand Up @@ -1150,6 +1219,7 @@ export class JSONSchemaInput extends LitElement {
const search = new Search({
options,
strict: schema.strict,
placeholder: schema.placeholder,
value: {
value: this.value,
key: this.value,
Expand Down Expand Up @@ -1188,7 +1258,7 @@ export class JSONSchemaInput extends LitElement {
if (isInteger) schema.type = "number";
const isNumber = schema.type === "number";

const isRequiredNumber = isNumber && this.required;
const isRequiredNumber = isNumber && this.required && this.allowNaN;

const fileSystemFormat = isFilesystemSelector(name, schema.format);
if (fileSystemFormat) return createFilesystemSelector(fileSystemFormat);
Expand Down Expand Up @@ -1240,11 +1310,37 @@ export class JSONSchemaInput extends LitElement {
else if (isNumber) value = newValue = parseFloat(value);
else if (isDateTime) value = newValue = resolveDateTime(value);

const isStrict = schema.strict ? true : false;
if (isNumber) {
if ("min" in schema && newValue < schema.min) newValue = schema.min;
else if ("max" in schema && newValue > schema.max) newValue = schema.max;

if (isNaN(newValue)) newValue = undefined;
// exclusiveMinimum
if ("exclusiveMinimum" in schema && newValue <= schema.exclusiveMinimum) {
if (isStrict)
newValue = this.value; // Set back to last value
else newValue = schema.exclusiveMinimum + 1; // (schema.step ?? 1);
}
// exclusiveMaximum
else if ("exclusiveMaximum" in schema && newValue >= schema.exclusiveMaximum) {
if (isStrict)
newValue = this.value; // Set back to last value
else newValue = schema.exclusiveMaximum - 1; // (schema.step ?? 1);
}
// minimum
else if ("minimum" in schema && newValue < schema.minimum) {
if (isStrict)
newValue = this.value; // Set back to last value
else newValue = schema.minimum;
}
// maximum
else if ("maximum" in schema && newValue > schema.maximum) {
if (isStrict)
newValue = this.value; // Set back to last value
else newValue = schema.maximum;
}
if (isNaN(newValue)) {
if (isStrict)
newValue = this.value; // Set back to last value
else newValue = undefined;
}
}

if (schema.transform) newValue = schema.transform(newValue, this.value, schema);
Expand All @@ -1256,7 +1352,7 @@ export class JSONSchemaInput extends LitElement {
// }

if (isNumber && newValue !== value) {
ev.target.value = newValue;
if (newValue !== undefined) ev.target.value = newValue; // Avoids unnecessary error message
value = newValue;
}

Expand Down
Loading
Loading