Confusing error message when testing scaffolded project

[aron]

Currently when generating the scaffolded project it just prints the stdout of the subprocess directly. This makes it look like the scaffold has failed rather than just the test. It took me a while to work out:

Output

replicate scaffold 3p2twllb5nsudqmtakaygg4dz4 --template=node
Cloning starter repo and installing dependencies...
Cloning into '3p2twllb5nsudqmtakaygg4dz4'...
remote: Enumerating objects: 22, done.
remote: Counting objects: 100% (22/22), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 22 (delta 10), reused 8 (delta 2), pack-reused 0
Receiving objects: 100% (22/22), 4.32 KiB | 4.32 MiB/s, done.
Resolving deltas: 100% (10/10), done.

added 2 packages in 1s
npm notice 
npm notice New major version of npm available! 9.8.1 -> 10.2.4
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.2.4
npm notice Run npm install -g [email protected] to update!
npm notice 
Writing new index.js...
Running example prediction...
Uh oh, an error occurred.
ApiError: Request to https://api.replicate.com/v1/predictions failed with status 402 Payment Required: {"title":"Monthly spend limit reached","detail":"You've hit your monthly spend limit. You can change or remove your limit at https://replicate.com/account/billing#limits.","status":402}
.
    at Replicate.request (/Users/Aron/Code/sandbox/3p2twllb5nsudqmtakaygg4dz4/node_modules/replicate/index.js:229:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Replicate.createPrediction (/Users/Aron/Code/sandbox/3p2twllb5nsudqmtakaygg4dz4/node_modules/replicate/lib/predictions.js:24:20)
    at async Replicate.run (/Users/Aron/Code/sandbox/3p2twllb5nsudqmtakaygg4dz4/node_modules/replicate/index.js:126:22)
    at async main (file:///Users/Aron/Code/sandbox/3p2twllb5nsudqmtakaygg4dz4/index.js:17:18) {
  request: Request {
    [Symbol(realm)]: { settingsObject: [Object] },
    [Symbol(state)]: {
      method: 'POST',
      localURLsOnly: false,
      unsafeRequest: false,
      body: [Object],
      client: [Object],
      reservedClient: null,
      replacesClientId: '',
      window: 'client',
      keepalive: false,
      serviceWorkers: 'all',
      initiator: '',
      destination: '',
      priority: null,
      origin: 'client',
      policyContainer: 'client',
      referrer: 'client',
      referrerPolicy: '',
      mode: 'cors',
      useCORSPreflightFlag: false,
      credentials: 'same-origin',
      useCredentials: false,
      cache: 'default',
      redirect: 'follow',
      integrity: '',
      cryptoGraphicsNonceMetadata: '',
      parserMetadata: '',
      reloadNavigation: false,
      historyNavigation: false,
      userActivation: false,
      taintedOrigin: false,
      redirectCount: 0,
      responseTainting: 'basic',
      preventNoCacheCacheControlHeaderModification: false,
      done: false,
      timingAllowFailed: false,
      headersList: [HeadersList],
      urlList: [Array],
      url: URL {}
    },
    [Symbol(signal)]: AbortSignal { aborted: false },
    [Symbol(headers)]: HeadersList {
      cookies: null,
      [Symbol(headers map)]: [Map],
      [Symbol(headers map sorted)]: null
    }
  },
  response: Response {
    [Symbol(realm)]: null,
    [Symbol(state)]: {
      aborted: false,
      rangeRequested: false,
      timingAllowPassed: true,
      requestIncludesCredentials: true,
      type: 'default',
      status: 402,
      timingInfo: [Object],
      cacheState: '',
      statusText: 'Payment Required',
      headersList: [HeadersList],
      urlList: [Array],
      body: [Object]
    },
    [Symbol(headers)]: HeadersList {
      cookies: null,
      [Symbol(headers map)]: [Map],
      [Symbol(headers map sorted)]: null
    }
  }
}

I think a simple place to start would be to have some form of delimiters.

--- START OF OUTPUT ---
...
--- END OF OUTPUT ---

But it might also be worth cleaning up the error messages of the template to make it more friendly too. If this sounds good 👍 and I'll implement it.

[mattt]

@aron Thanks for your feedback! I agree that delimiters would be a nice improvement.

Curious to know what you think about the scaffold command running the example in the first place. Did you find that unexpected?

[aron]

It was unexpected, honestly I didn't clock it was running a test (I assumed the issue was the replicate CLI tool failing to auth). I fixed the issue, and only realized it was the project failing when I started debugging the source of the error message.

When it did work it was nice to know everything was setup and working correctly. I think also if it's intentional is educational it would be nice to print the command that was run, and be clear it's not an example, it's the actual imported prediction:

Running generated prediction `node index.js`...
--- START OF OUTPUT ---
...
--- END OF OUTPUT ---
Successfully ran prediction! | Failed to run with exit code 1 (see above output for errors)