Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature(multi-bucket): add multi-bucket support to storage components #5562

Open
wants to merge 33 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
82099b3
initial commit to add 'bucket' property to storage components
jordanvn Aug 10, 2024
f45e688
chore: use StorageBucket type in StorageImagePathProps
jordanvn Aug 12, 2024
130914d
remove duplicate StorageBucket type declaration
jordanvn Aug 20, 2024
6759502
chore: update aws-amplify version to include multi-bucket support
jordanvn Aug 20, 2024
6d200a9
docs: include references to new 'bucket' prop and its usage
jordanvn Aug 20, 2024
77de102
Merge branch 'main' into multi-bucket-support
jordanvn Aug 20, 2024
5ea81d5
more explicitly clarifying that can be a string in docs example
jordanvn Aug 20, 2024
98b51db
Merge branch 'main' into multi-bucket-support
jordanvn Sep 23, 2024
79d1209
chore: changing reference of storage manager to file uploader
jordanvn Sep 23, 2024
9916524
chore: updating yarn.lock
jordanvn Sep 23, 2024
bb0c2f7
chore: undoing unnecessary linting changes
jordanvn Sep 23, 2024
0fab899
chore: moving yarn.lock from main branch parity
jordanvn Sep 24, 2024
77119df
Merge branch 'main' into multi-bucket-support
jordanvn Sep 24, 2024
cd12258
chore: updating yarn.lock to main
jordanvn Sep 24, 2024
4c6eae2
chore: add missing references to 'bucket'
jordanvn Sep 24, 2024
d36d03e
chore: adding tests and new example app
jordanvn Sep 26, 2024
f3953b6
Merge branch 'main' into multi-bucket-support
jordanvn Sep 26, 2024
fabf129
chore: add end of file line
jordanvn Sep 27, 2024
8cbbd05
chore: add changeset
jordanvn Sep 27, 2024
757226b
chore: setting more obviously fake bucket name as example
jordanvn Sep 30, 2024
f825df3
chore: adding link for setting up multi-bucket configuration to docs
jordanvn Sep 30, 2024
e96c3f3
chore: removing unnecessary type definitions
jordanvn Sep 30, 2024
a5c3d5e
chore: removing unnecessary type from Storage Image props
jordanvn Sep 30, 2024
68c0a84
Merge branch 'main' into multi-bucket-support
jordanvn Sep 30, 2024
21c0763
Merge branch 'main' into multi-bucket-support
jordanvn Oct 2, 2024
f6645f1
chore: adding bucket as omitted prop to gen1 props
jordanvn Oct 2, 2024
5787b6c
fix(tests): updating test data to fit expected behavior
jordanvn Oct 10, 2024
55a141a
Merge branch 'main' into multi-bucket-support
jordanvn Oct 10, 2024
2895e9b
Merge branch 'main' into multi-bucket-support
jordanvn Oct 11, 2024
cc79b45
Merge branch 'main' into multi-bucket-support
jordanvn Oct 11, 2024
23e7619
Merge branch 'main' into multi-bucket-support
jordanvn Dec 5, 2024
b2349b1
Merge branch 'main' into multi-bucket-support
calebpollman Jan 13, 2025
2674696
Merge branch 'main' into multi-bucket-support
jordanvn Jan 16, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ const getDeprecationMessage = ({

export const StorageImage = ({
accessLevel,
bucket,
className,
fallbackSrc,
identityId,
Expand Down Expand Up @@ -82,11 +83,20 @@ export const StorageImage = ({
onError,
options: {
accessLevel,
bucket,
targetIdentityId: identityId,
validateObjectExistence,
},
}),
[accessLevel, imgKey, identityId, onError, path, validateObjectExistence]
[
accessLevel,
bucket,
calebpollman marked this conversation as resolved.
Show resolved Hide resolved
imgKey,
identityId,
onError,
path,
validateObjectExistence,
]
);

const { url } = useGetUrl(input);
Expand Down
12 changes: 12 additions & 0 deletions packages/react-storage/src/components/StorageImage/types.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
import { StorageAccessLevel } from '@aws-amplify/core';
import { ImageProps } from '@aws-amplify/ui-react';

interface BucketInfo {
bucketName: string;
region: string;
}

// accepts either a 'friendly name' that the user has assigned
// or an object containing the region as well as the name generated by the backend
type StorageBucket = string | BucketInfo;

export interface StorageImageProps extends Omit<ImageProps, 'src'> {
// Use imgKey instead of key because key is a reserved keyword
// and cannot be accessed via props in React components
Expand All @@ -15,6 +24,7 @@ export interface StorageImageProps extends Omit<ImageProps, 'src'> {
* `accessLevel` will be replaced with `path` in a future major version of Amplify UI. See https://ui.docs.amplify.aws/react/connected-components/storage/storageimage#props
*/
accessLevel: StorageAccessLevel;
bucket?: StorageBucket;
Copy link
Contributor

@reesscot reesscot Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we bucket here if it's only supported in Gen2? (StorageImagePathProps)

Copy link
Member Author

@jordanvn jordanvn Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we shouldn't. This was a misunderstanding on my part. I've removed this

/**
* @deprecated
* `identityId` will be replaced with `path` in a future major version of Amplify UI. See https://ui.docs.amplify.aws/react/connected-components/storage/storageimage#props
Expand All @@ -36,6 +46,7 @@ export interface StorageImageProps extends Omit<ImageProps, 'src'> {

type OmittedPropKey =
| 'accessLevel'
| 'bucket'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we omitting bucket here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, think its related to my comment above.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This was an oversight on my part

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think the intention of the omission would be clearer if documented the same way as path

| 'imgKey'
| 'identityId'
| 'onStorageGetError'
Expand All @@ -46,6 +57,7 @@ export interface StorageImagePathProps
path: string | ((input: { identityId?: string }) => string);
imgKey?: never;
accessLevel?: never;
bucket?: StorageBucket;
identityId?: never;
onStorageGetError?: never;
}
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ const StorageManagerBase = React.forwardRef(function StorageManager(
acceptedFileTypes = [],
accessLevel,
autoUpload = true,
bucket,
components,
defaultFiles,
displayText: overrideDisplayText,
Expand Down Expand Up @@ -143,6 +144,7 @@ const StorageManagerBase = React.forwardRef(function StorageManager(

useUploadFiles({
accessLevel,
bucket,
files,
isResumable,
maxFileCount,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { isFunction } from '@aws-amplify/ui';

import { PathCallback, uploadFile } from '../../utils';
import { getInput } from '../../utils';
import { StorageBucket } from '../../types';
import { FileStatus } from '../../types';
import { StorageManagerProps } from '../../types';
import { UseStorageManager } from '../useStorageManager';
Expand All @@ -25,12 +26,14 @@ export interface UseUploadFilesProps
'setUploadingFile' | 'setUploadProgress' | 'setUploadSuccess' | 'files'
> {
accessLevel?: StorageManagerProps['accessLevel'];
bucket?: StorageBucket;
onProcessFileSuccess: (input: { id: string; processedKey: string }) => void;
path?: string | PathCallback;
}

export function useUploadFiles({
accessLevel,
bucket,
files,
isResumable,
maxFileCount,
Expand Down Expand Up @@ -73,6 +76,7 @@ export function useUploadFiles({

const input = getInput({
accessLevel,
bucket,
file,
key,
onProcessFileSuccess: handleProcessFileSuccess,
Expand Down Expand Up @@ -111,6 +115,7 @@ export function useUploadFiles({
}, [
files,
accessLevel,
bucket,
isResumable,
setUploadProgress,
setUploadingFile,
Expand Down
14 changes: 14 additions & 0 deletions packages/react-storage/src/components/StorageManager/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,15 @@ import {
} from './ui';
import { StorageManagerDisplayText, PathCallback, UploadTask } from './utils';

interface BucketInfo {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: should we share this type between components?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that makes sense to me. It didn't seem substantial enough to create a new shared types file for storage components, so I went with the definition in the Storage Manager types. There are already exported types and interfaces in that file, and none aside from props in Storage Image types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just import and extend FileUploaderProps in the assignment of StorageManagerProps and dedupe this file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting the removal of this file and providing its exports from FileUploader types? Or reducing StorageManagerProps to export interface StorageManagerProps extends FileUploaderProps {}?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keeping this file and doing the latter

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Thanks for the clarification

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated file to extend FileUploaderProps instead of duplicating type declaration

bucketName: string;
region: string;
}

// accepts either a 'friendly name' that the user has assigned
// or an object containing the region as well as the name generated by the backend
export type StorageBucket = string | BucketInfo;

export enum FileStatus {
ADDED = 'added',
QUEUED = 'queued',
Expand Down Expand Up @@ -68,6 +77,10 @@ export interface StorageManagerProps {
* Determines if the upload will automatically start after a file is selected, default value: true
*/
autoUpload?: boolean;
/**
* Designates the bucket to upload to, if the user has multiple buckets configured
*/
bucket?: StorageBucket;
/**
* Component overrides
*/
Expand Down Expand Up @@ -139,6 +152,7 @@ export interface StorageManagerPathProps
* - `PathCallback`: callback provided an input containing the current `identityId`,
* resolved value is prefixed to the file `key` for each file
*/
bucket?: StorageBucket;
path: string | PathCallback;
accessLevel?: never;
useAccelerateEndpoint?: boolean;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@ import { UploadDataWithPathInput, UploadDataInput } from 'aws-amplify/storage';

import { isString, isTypedFunction } from '@aws-amplify/ui';

import { ProcessFile } from '../types';
import { ProcessFile, StorageBucket } from '../types';
import { resolveFile } from './resolveFile';
import { PathCallback, PathInput } from './uploadFile';

export interface GetInputParams {
accessLevel: StorageAccessLevel | undefined;
bucket?: StorageBucket;
file: File;
key: string;
onProcessFileSuccess: (input: { processedKey: string }) => void;
Expand All @@ -21,6 +22,7 @@ export interface GetInputParams {

export const getInput = ({
accessLevel,
bucket,
file,
key,
onProcessFileSuccess,
Expand All @@ -44,7 +46,13 @@ export const getInput = ({
const contentType = file.type || 'binary/octet-stream';

// IMPORTANT: always pass `...rest` here for backwards compatibility
const options = { contentType, onProgress, useAccelerateEndpoint, ...rest };
const options = {
bucket,
contentType,
onProgress,
useAccelerateEndpoint,
...rest,
};

let inputResult: PathInput | UploadDataInput;
if (hasKeyInput) {
Expand Down
Loading