Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make shopify theme dev more resilient to HTTP errors with the Admin API #4606

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/silent-ghosts-hide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@shopify/theme': patch
---

Make `shopify theme dev` more resilient to HTTP errors with the Admin API
22 changes: 14 additions & 8 deletions packages/cli-kit/src/public/node/themes/api.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {restRequest} from '@shopify/cli-kit/node/api/admin'
import {AbortError} from '@shopify/cli-kit/node/error'

vi.mock('@shopify/cli-kit/node/api/admin')
vi.mock('@shopify/cli-kit/node/system')

const session = {token: 'token', storeFqdn: 'my-shop.myshopify.com'}

Expand Down Expand Up @@ -305,9 +306,14 @@ describe('deleteTheme', () => {
})

describe('request errors', () => {
const httpErrors = [401, 403, 500, 999]

httpErrors.forEach((httpError) => {
const httpResponses = [
{httpError: 401, expectedRetries: 3},
{httpError: 403, expectedRetries: 1},
{httpError: 500, expectedRetries: 3},
{httpError: 999, expectedRetries: 1},
]

httpResponses.forEach(({httpError, expectedRetries}) => {
test(`${httpError} errors`, async () => {
// Given
vi.mocked(restRequest).mockResolvedValue({
Expand All @@ -316,12 +322,12 @@ describe('request errors', () => {
headers: {},
})

await expect(async () => {
// When
return deleteTheme(1, session)
// When
const deletePromise = async () => deleteTheme(1, session)

// Then
}).rejects.toThrowError(AbortError)
// Then
await expect(deletePromise).rejects.toThrowError(AbortError)
expect(restRequest).toBeCalledTimes(expectedRetries)
})
})
})
Expand Down
45 changes: 43 additions & 2 deletions packages/cli-kit/src/public/node/themes/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import {
buildThemeAsset,
} from '@shopify/cli-kit/node/themes/factories'
import {Result, Checksum, Key, Theme, ThemeAsset} from '@shopify/cli-kit/node/themes/types'
import {outputDebug} from '@shopify/cli-kit/node/output'
import {sleep} from '@shopify/cli-kit/node/system'

export type ThemeParams = Partial<Pick<Theme, 'name' | 'role' | 'processing' | 'src'>>
export type AssetParams = Pick<ThemeAsset, 'key'> & Partial<Pick<ThemeAsset, 'value' | 'attachment'>>
Expand Down Expand Up @@ -108,6 +110,7 @@ async function request<T>(
session: AdminSession,
params?: T,
searchParams: {[name: string]: string} = {},
retries = 1,
): Promise<RestResponse> {
const response = await throttler.throttle(() => restRequest(method, path, session, params, searchParams))

Expand All @@ -128,13 +131,33 @@ async function request<T>(
case status === 403:
return handleForbiddenError(response, session)
case status === 401:
throw new AbortError(`[${status}] API request unauthorized error`)
// Retry 401 errors to be resilient to authentication errors.
return handleRetriableError({
Comment on lines 133 to +135
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't refresh the token, I'm not sure if this will do anything? I guess it doesn't hurt much but the token will be the same so I assume the API will keep returning 401?

Perhaps we should explore the option of "refresh on demand" when auth errors happen in addition to refreshing periodically in an interval? For example, something like session.refresh(). Or perhaps a job for AsyncLocalStorage if it's too hard to pass that logic down here 🤔

path,
retries,
retry: () => {
return request(method, path, session, params, searchParams, retries + 1)
},
fail: () => {
throw new AbortError(`[${status}] API request unauthorized error`)
},
})
case status === 422:
throw new AbortError(`[${status}] API request unprocessable content: ${errors(response)}`)
case status >= 400 && status <= 499:
throw new AbortError(`[${status}] API request client error`)
case status >= 500 && status <= 599:
throw new AbortError(`[${status}] API request server error`)
// Retry 500-family of errors as that may solve the issue (especially in 503 errors)
return handleRetriableError({
path,
retries,
retry: () => {
return request(method, path, session, params, searchParams, retries + 1)
},
fail: () => {
throw new AbortError(`[${status}] API request server error`)
},
})
default:
throw new AbortError(`[${status}] API request unexpected error`)
}
Expand Down Expand Up @@ -175,3 +198,21 @@ function errorMessage(response: RestResponse): string {

return ''
}

interface RetriableErrorOptions {
path: string
retries: number
retry: () => Promise<RestResponse>
fail: () => never
}

async function handleRetriableError({path, retries, retry, fail}: RetriableErrorOptions): Promise<RestResponse> {
if (retries >= 3) {
fail()
}

outputDebug(`[${retries}] Retrying '${path}' request...`)

await sleep(0.2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was there a decision that led you to using 0.2 seconds as our retry "cooldown"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should use different cooldowns depending on the error type... 0.2 seems sensible for something that is likely to keep failing, like a 401, so that we fail fast.
But perhaps a 503 could recover if we give it more time? Not sure really 🤔

At the same time, we are also adding another type of error handling in #4599 , so maybe it's good to fail fast here and let the consumer handle the situation...

return retry()
}
Loading