Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Follow up on object store selection PR. #15654

Merged
merged 29 commits into from
Mar 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
e2530b4
private objectstores & dataset.sharable
jmchilton Feb 7, 2023
50ffad2
Rename test_selection to be more specific.
jmchilton Jun 11, 2022
8e0a01e
get_quota in SQL
jmchilton Sep 12, 2020
f7a4992
Implement quota tracking options per ObjectStore.
jmchilton Feb 7, 2023
d1a2eab
Fix unique constraint add/remove in quota source labels...
jmchilton Feb 21, 2023
c7f200c
Rebuild client typescript schema.
jmchilton Feb 9, 2023
dfd4fd1
unit test fix
jmchilton Feb 14, 2023
6d91292
Lint fixes...
jmchilton Feb 14, 2023
4432497
Drop nice_total_disk_usage from usage APIs.
jmchilton Feb 14, 2023
4785834
Improved error message in api_asserts.
jmchilton Feb 16, 2023
017c612
Small hacks... are these needed?
jmchilton Jun 17, 2022
46b6d85
implement preferred object store id
jmchilton Feb 9, 2023
32d12d6
Fix for subworkflows...
jmchilton Feb 14, 2023
1c6f9d0
nested workflow tests for this intermediate/output objectstore selection
jmchilton Feb 14, 2023
0e1bc0d
fix object store workflow selection with collections...
jmchilton Feb 14, 2023
d04d79a
More work on object store selection of collections.
jmchilton Feb 14, 2023
45179db
Fix dynamic collection outputs to respect job's specified object store
jmchilton Feb 16, 2023
17baf67
Fix object store selection for workflows with dynamic output collecti…
jmchilton Feb 16, 2023
03cfc77
test case for object store selection with doubly nested workflow output
jmchilton Feb 16, 2023
c80daac
Rebuild client schema for object store APIs
jmchilton Feb 16, 2023
2cdcf15
Cleanup object store selection client.
jmchilton Feb 21, 2023
76c0f98
Pydantic the new object store API.
jmchilton Feb 27, 2023
5462b7a
Convert SelectObjectStore to fetcher pattern.
jmchilton Feb 27, 2023
dabdb95
Migrate new user quota usage APIs to FastAPI.
jmchilton Feb 27, 2023
d1b7594
pydantic the new user quota usage APIs
jmchilton Feb 27, 2023
d92d606
Rework object store selection jest tests for fetcher pattern.
jmchilton Mar 1, 2023
fcd616d
Migrate SelectObjectStore to composition API.
jmchilton Mar 2, 2023
c40672a
Only show objectstore selection in the GUI if actual selection makes …
jmchilton Mar 2, 2023
9964d4d
More consistent icon for object store selection.
jmchilton Mar 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,18 @@ import { getLocalVue } from "tests/jest/helpers";
import flushPromises from "flush-promises";
import MockAdapter from "axios-mock-adapter";
import axios from "axios";
import MarkdownIt from "markdown-it";

const localVue = getLocalVue();

const TEST_STORAGE_API_RESPONSE_WITHOUT_ID = {
object_store_id: null,
};
const TEST_STORAGE_API_RESPONSE_WITH_ID = {
object_store_id: "foobar",
};
const TEST_STORAGE_API_RESPONSE_WITH_NAME = {
object_store_id: "foobar",
name: "my cool storage",
description: "My cool **markdown**",
private: false,
};
const TEST_DATASET_ID = "1";
const TEST_STORAGE_URL = `/api/datasets/${TEST_DATASET_ID}/storage`;
const TEST_RENDERED_MARKDOWN_AS_HTML = "<p>My cool <strong>markdown</strong>\n";
const TEST_ERROR_MESSAGE = "Opps all errors.";

// works fine without mocking but I guess it is more JS unit-y with the mock?
jest.mock("markdown-it");
MarkdownIt.mockImplementation(() => {
return {
render(markdown) {
return TEST_RENDERED_MARKDOWN_AS_HTML;
},
};
});

describe("Dataset Storage", () => {
describe("DatasetStorage.vue", () => {
let axiosMock;
let wrapper;

Expand All @@ -46,9 +27,6 @@ describe("Dataset Storage", () => {
wrapper = shallowMount(DatasetStorage, {
propsData: { datasetId: TEST_DATASET_ID },
localVue,
stubs: {
"loading-span": true,
},
});
}

Expand All @@ -62,6 +40,7 @@ describe("Dataset Storage", () => {
mount();
await wrapper.vm.$nextTick();
expect(wrapper.findAll("loading-span-stub").length).toBe(1);
expect(wrapper.findAll("describe-object-store-stub").length).toBe(0);
});

it("test error rendering...", async () => {
Expand All @@ -78,44 +57,8 @@ describe("Dataset Storage", () => {
it("test dataset storage with object store without id", async () => {
await mountWithResponse(TEST_STORAGE_API_RESPONSE_WITHOUT_ID);
expect(wrapper.findAll("loading-span-stub").length).toBe(0);
expect(wrapper.vm.descriptionRendered).toBeNull();
const header = wrapper.findAll("h2");
expect(header.length).toBe(1);
expect(header.at(0).text()).toBe("Dataset Storage");
const byIdSpan = wrapper.findAll(".display-os-by-id");
expect(byIdSpan.length).toBe(0);
const byNameSpan = wrapper.findAll(".display-os-by-name");
expect(byNameSpan.length).toBe(0);
const byDefaultSpan = wrapper.findAll(".display-os-default");
expect(byDefaultSpan.length).toBe(1);
});

it("test dataset storage with object store id", async () => {
await mountWithResponse(TEST_STORAGE_API_RESPONSE_WITH_ID);
expect(wrapper.findAll("loading-span-stub").length).toBe(0);
expect(wrapper.vm.storageInfo.object_store_id).toBe("foobar");
expect(wrapper.vm.descriptionRendered).toBeNull();
const header = wrapper.findAll("h2");
expect(header.length).toBe(1);
expect(header.at(0).text()).toBe("Dataset Storage");
const byIdSpan = wrapper.findAll(".display-os-by-id");
expect(byIdSpan.length).toBe(1);
const byNameSpan = wrapper.findAll(".display-os-by-name");
expect(byNameSpan.length).toBe(0);
});

it("test dataset storage with object store name", async () => {
await mountWithResponse(TEST_STORAGE_API_RESPONSE_WITH_NAME);
expect(wrapper.findAll("loading-span-stub").length).toBe(0);
expect(wrapper.vm.storageInfo.object_store_id).toBe("foobar");
expect(wrapper.vm.descriptionRendered).toBe(TEST_RENDERED_MARKDOWN_AS_HTML);
const header = wrapper.findAll("h2");
expect(header.length).toBe(1);
expect(header.at(0).text()).toBe("Dataset Storage");
const byIdSpan = wrapper.findAll(".display-os-by-id");
expect(byIdSpan.length).toBe(0);
const byNameSpan = wrapper.findAll(".display-os-by-name");
expect(byNameSpan.length).toBe(1);
expect(wrapper.findAll("describe-object-store-stub").length).toBe(1);
expect(wrapper.vm.storageInfo.private).toEqual(false);
});

afterEach(() => {
Expand Down
24 changes: 4 additions & 20 deletions client/src/components/Dataset/DatasetStorage/DatasetStorage.vue
Original file line number Diff line number Diff line change
Expand Up @@ -16,30 +16,21 @@
</p>
</div>
<div v-else>
<p>
This dataset is stored in
<span v-if="storageInfo.name" class="display-os-by-name">
a Galaxy object store named <b>{{ storageInfo.name }}</b>
</span>
<span v-else-if="storageInfo.object_store_id" class="display-os-by-id">
a Galaxy object store with id <b>{{ storageInfo.object_store_id }}</b>
</span>
<span v-else class="display-os-default"> the default configured Galaxy object store </span>.
</p>
<div v-html="descriptionRendered"></div>
<describe-object-store what="This dataset is stored in" :storage-info="storageInfo" />
</div>
</div>
</template>

<script>
import axios from "axios";
import { getAppRoot } from "onload/loadConfig";
import LoadingSpan from "components/LoadingSpan";
import MarkdownIt from "markdown-it";
import { errorMessageAsString } from "utils/simple-error";
import DescribeObjectStore from "components/ObjectStore/DescribeObjectStore";
import LoadingSpan from "components/LoadingSpan";

export default {
components: {
DescribeObjectStore,
LoadingSpan,
},
props: {
Expand All @@ -58,7 +49,6 @@ export default {
data() {
return {
storageInfo: null,
descriptionRendered: null,
errorMessage: null,
};
},
Expand Down Expand Up @@ -94,13 +84,7 @@ export default {
methods: {
handleResponse(response) {
const storageInfo = response.data;
const description = storageInfo.description;
this.storageInfo = storageInfo;
if (description) {
this.descriptionRendered = MarkdownIt({ html: true }).render(storageInfo.description);
} else {
this.descriptionRendered = null;
}
},
},
};
Expand Down
152 changes: 103 additions & 49 deletions client/src/components/History/CurrentHistory/HistoryCounter.vue
Original file line number Diff line number Diff line change
@@ -1,67 +1,112 @@
<template>
<div class="history-size my-1 d-flex justify-content-between">
<b-button
v-b-tooltip.hover
title="History Size"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="onDashboard">
<icon icon="database" />
<span>{{ historySize | niceFileSize }}</span>
</b-button>
<b-button-group>
<CurrentUser v-slot="{ user }">
<div class="history-size my-1 d-flex justify-content-between">
<b-button
v-b-tooltip.hover
title="Show active"
title="History Size"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="setFilter('')">
<span class="fa fa-map-marker" />
<span>{{ numItemsActive }}</span>
@click="onDashboard">
<icon icon="database" />
<span>{{ historySize | niceFileSize }}</span>
</b-button>
<b-button
v-if="numItemsDeleted"
v-b-tooltip.hover
title="Show deleted"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="setFilter('deleted:true')">
<icon icon="trash" />
<span>{{ numItemsDeleted }}</span>
</b-button>
<b-button
v-if="numItemsHidden"
v-b-tooltip.hover
title="Show hidden"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="setFilter('visible:false')">
<icon icon="eye-slash" />
<span>{{ numItemsHidden }}</span>
</b-button>
<b-button
v-b-tooltip.hover
:title="reloadButtonTitle"
:variant="reloadButtonVariant"
size="sm"
class="rounded-0 text-decoration-none"
@click="reloadContents()">
<span :class="reloadButtonCls" />
</b-button>
</b-button-group>
</div>
<b-button-group>
<ConfigProvider v-slot="{ config }">
Copy link
Member

@mvdbeek mvdbeek Mar 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the useConfig(true) composable here please ? The ConfigProvider triggers a new fetch every time.

<b-button
v-if="config && config.object_store_allows_id_selection"
:id="`history-storage-${history.id}`"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="showPreferredObjectStoreModal = true">
<icon icon="hdd" />
</b-button>
</ConfigProvider>
<ConfigProvider v-slot="{ config }">
<PreferredStorePopover
v-if="config && config.object_store_allows_id_selection"
:history-id="history.id"
:history-preferred-object-store-id="historyPreferredObjectStoreId"
:user="user">
</PreferredStorePopover>
</ConfigProvider>
<b-button-group>
<b-button
v-b-tooltip.hover
title="Show active"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="setFilter('')">
<span class="fa fa-map-marker" />
<span>{{ numItemsActive }}</span>
</b-button>
<b-button
v-if="numItemsDeleted"
v-b-tooltip.hover
title="Show deleted"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="setFilter('deleted:true')">
<icon icon="trash" />
<span>{{ numItemsDeleted }}</span>
</b-button>
<b-button
v-if="numItemsHidden"
v-b-tooltip.hover
title="Show hidden"
variant="link"
size="sm"
class="rounded-0 text-decoration-none"
@click="setFilter('visible:false')">
<icon icon="eye-slash" />
<span>{{ numItemsHidden }}</span>
</b-button>
<b-button
v-b-tooltip.hover
:title="reloadButtonTitle"
:variant="reloadButtonVariant"
size="sm"
class="rounded-0 text-decoration-none"
@click="reloadContents()">
<span :class="reloadButtonCls" />
</b-button>
</b-button-group>
<b-modal
v-model="showPreferredObjectStoreModal"
title="History Preferred Object Store"
modal-class="history-preferred-object-store-modal"
title-tag="h3"
size="sm"
hide-footer>
<SelectPreferredStore
:user-preferred-object-store-id="user.preferred_object_store_id"
:history="history"
@updated="onUpdatePreferredObjectStoreId" />
</b-modal>
</b-button-group>
</div>
</CurrentUser>
</template>

<script>
import prettyBytes from "pretty-bytes";
import { formatDistanceToNowStrict } from "date-fns";
import { usesDetailedHistoryMixin } from "./usesDetailedHistoryMixin.js";
import CurrentUser from "components/providers/CurrentUser";
import ConfigProvider from "components/providers/ConfigProvider";
import PreferredStorePopover from "./PreferredStorePopover";
import SelectPreferredStore from "./SelectPreferredStore";

export default {
components: {
ConfigProvider,
CurrentUser,
PreferredStorePopover,
SelectPreferredStore,
},
filters: {
niceFileSize(rawSize = 0) {
return prettyBytes(rawSize);
Expand All @@ -78,6 +123,8 @@ export default {
reloadButtonCls: "fa fa-sync",
reloadButtonTitle: "",
reloadButtonVariant: "link",
showPreferredObjectStoreModal: false,
historyPreferredObjectStoreId: this.history.preferred_object_store_id,
};
},
mounted() {
Expand Down Expand Up @@ -111,6 +158,13 @@ export default {
this.reloadButtonCls = "fa fa-sync";
}, 1000);
},
onUpdatePreferredObjectStoreId(preferredObjectStoreId) {
this.showPreferredObjectStoreModal = false;
// ideally this would be pushed back to the history object somehow
// and tracked there... but for now this is only component using
// this information.
this.historyPreferredObjectStoreId = preferredObjectStoreId;
},
},
};
</script>
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<template>
<b-popover :target="`history-storage-${historyId}`" triggers="hover" placement="bottomleft">
<template v-slot:title>Preferred Target Object Store</template>
<p v-if="historyPreferredObjectStoreId" class="history-preferred-object-store-inherited">
This target object store has been set at the history level.
</p>
<p v-else class="history-preferred-object-store-not-inherited">
This target object store has been inherited from your user preferences (set in User -> Preferences ->
Preferred Object Store). If that option is updated, this history will target that new default.
</p>
<ShowSelectedObjectStore
v-if="preferredObjectStoreId"
:preferred-object-store-id="preferredObjectStoreId"
for-what="Galaxy will default to storing this history's datasets in "></ShowSelectedObjectStore>
<div v-localize>
Change this preference object store target by clicking on the storage button in the history panel.
</div>
</b-popover>
</template>

<script>
import ShowSelectedObjectStore from "components/ObjectStore/ShowSelectedObjectStore";

export default {
components: {
ShowSelectedObjectStore,
},
props: {
historyId: {
type: String,
required: true,
},
historyPreferredObjectStoreId: {
type: String,
default: null,
},
user: { type: Object, required: true },
},
computed: {
preferredObjectStoreId() {
let id = this.historyPreferredObjectStoreId;
if (!id) {
id = this.user.preferred_object_store_id;
}
return id;
},
},
};
</script>
Loading