npm publish fails / `npm publish <folder>` publishes from cwd folder #504

unlight · 2022-07-31T14:10:57Z

I have configured semantic-release/npm with pkgRoot = './dist'
On CI/CD command is npx -w my-package semantic-release

CI run 28 Logs:

[1:20:01 PM] [semantic-release] [@semantic-release/git] › ℹ  Prepared Git release: my-package-v1.1.2
[1:20:03 PM] [semantic-release] › ✔  Created tag my-package-v1.1.2
[1:20:03 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/npm"
[1:20:03 PM] [semantic-release] [@semantic-release/npm] › ℹ  Publishing version 1.1.2 to npm registry on dist-tag latest
+ [email protected]
[1:20:04 PM] [semantic-release] [@semantic-release/npm] › ℹ  Published [email protected] to dist-tag @latest on https://registry.npmjs.org/

Note: [email protected] is a result of npm publish

CI run 29 Logs:

2022-07-31T13:06:50.035Z semantic-release:plugins options for @semantic-release/npm/verifyConditions: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.040Z semantic-release:plugins options for @semantic-release/npm/prepare: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.040Z semantic-release:plugins options for @semantic-release/npm/publish: {
  pkgRoot: '/root/packages/my-package/dist'
}
2022-07-31T13:06:50.041Z semantic-release:plugins options for @semantic-release/npm/addChannel: {
  pkgRoot: '/root/packages/my-package/dist'
}
[1:06:51 PM] [semantic-release] [@semantic-release/npm] › ℹ  Write version 1.1.2 to package.json in /root/packages/my-package/dist
v1.1.2
[1:06:52 PM] [semantic-release] › ✔  Completed step "prepare" of plugin "@semantic-release/npm"
[1:06:52 PM] [semantic-release] [@semantic-release/git] › ℹ  Found 1 file(s) to commit
2022-07-31T13:06:52.112Z semantic-release:git commited files: [ 'CHANGELOG.md' ]
[1:06:52 PM] [semantic-release] [@semantic-release/git] › ℹ  Prepared Git release: my-package-v1.1.2
[1:06:53 PM] [semantic-release] › ✔  Created tag my-package-v1.1.2
[1:06:53 PM] [semantic-release] › ℹ  Start step "publish" of plugin "@semantic-release/npm"
[1:06:53 PM] [semantic-release] [@semantic-release/npm] › ℹ  Publishing version 1.1.2 to npm registry on dist-tag latest
npm ERR! code E403
npm ERR! 403 403 Forbidden - PUT https://registry.npmjs.org/my-package - You cannot publish over the previously published versions: 0.0.0-dev.1.
  shortMessage: 'Command failed with exit code 1: npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/',
  command: 'npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/',
  escapedCommand: 'npm publish "/root/packages/my-package/dist" --userconfig "/tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc" --tag latest --registry "https://registry.npmjs.org/"',
  exitCode: 1,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: 'npm ERR! code E403\n' +
    'npm ERR! 403 403 Forbidden - PUT https://registry.npmjs.org/my-package - You cannot publish over the previously published versions: 0.0.0-dev.1.\n' +
    'npm ERR! 403 In most cases, you or one of your dependencies are requesting\n' +
    'npm ERR! 403 a package version that is forbidden by your security policy, or\n' +
    'npm ERR! 403 on a server you do not have access to.\n' +
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false,
  pluginName: '@semantic-release/npm'
}
Error: Command failed with exit code 1: npm publish /root/packages/my-package/dist --userconfig /tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc --tag latest --registry https://registry.npmjs.org/
##[error]Process completed with exit code 1.

As you can see from logs version 1.1.2 is correctly written to package.json in dist folder, but following command is failing

npm publish "/root/packages/my-package/dist" --userconfig "/tmp/6d5920eb12a6009b40553b60ba81d8e8/.npmrc" --tag latest --registry "https://registry.npmjs.org/"

It's trying to publish 0.0.0-dev.1 which is in package.json in /root/packages/my-package (equals to process.cwd())

But if I execute this command locally, it works without errors.

Corresponding npm publish command:

const result = execa(
  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd, env, preferLocal: true}
);

Looks like that execa has issues with cwd or preferLocal (or its combination).
Update: Some additional research results here in https://github.com/semantic-release/npm/issues/504#issuecomment-1201540336

The text was updated successfully, but these errors were encountered:

NullVoxPopuli · 2022-07-31T22:24:01Z

I just ran in to this as well.

my stack:

   at makeError (<repo>/node_modules/.pnpm/[email protected]/node_modules/execa/lib/error.js:60:11)
    at handlePromise (<repo>/node_modules/.pnpm/[email protected]/node_modules/execa/index.js:118:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async module.exports (<repo>/node_modules/.pnpm/@[email protected][email protected]/node_modules/@semantic-release/npm/lib/prepare.js:21:3)}

This fails when running not through execa as well:

❯ npm version 6.1.2 --userconfig /tmp/28e1f8c78e58a4699a9d64a8774e2e50/.npmrc --no-git-tag-version --allow-same-version
ember-statechart-component
v6.1.2
npm ERR! Cannot read properties of null (reading 'matches')

execa is on version 6 something -- could it be relateD?

travi · 2022-08-01T02:49:05Z

execa is on version 6 something -- could it be relateD?

the range defined for this project's dependency on execa does not allow v6, so something is misconfigured in your project. you will need to resolve that first before further investigation could be done within the context of semantic-release

unlight · 2022-08-01T18:08:57Z

I made some research.
I patched node_modules/@semantic-release/npm/lib/publish.js

const command = `npm publish ${basePath} --userconfig ${npmrc} --tag ${distTag} --registry ${registry}`;
require('child_process').execSync(command);

Same result, it maybe issue with npm itself (my version is 8.11.0), also I do no see any related bug fixes in changelog of execa (after v5)

I've tried another trick:

// npm publish --userconfig /tmp/xxx/.npmrc --tag latest --registry http://localhost:4873/ # {cwd: /root/packages/my-package/dist}
const result = execa(  'npm',
   ['publish', '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}
 );

And it works!

So it definitly something wrong with npm and npm publish <folder> command, npm publish for current folder works ok.

travi · 2022-08-01T19:09:09Z

But if I execute this command locally, it works without errors.

in your local test, did you execute directly from your dist directory, or with the same command that semantic-release is running and from the effective cwd?

i'm wondering if what you are describing is either a breaking change in npm or a bug that should be reported to them.

my version is 8.11.0

the npm plugin for semantic-release defines a direct dependency on npm, so the version installed because of that would have more impact on this situation than your globally installed version. you can determine which version is locally installed with the command npm ls npm. if you are using our recommendation of executing with npx instead of installing locally to your project, that would mean that the latest version within the defined range will be used.

unlight · 2022-08-01T19:26:57Z

But if I execute this command locally, it works without errors.

in your local test, did you execute directly from your dist directory, or with the same command that semantic-release is running and from the effective cwd?

same command that semantic-release is running and from the effective cwd

my version is 8.11.0

Yes, sorry, it's my global version. npm ls npm shows [email protected]

unlight · 2022-08-01T19:44:33Z

i'm wondering if what you are describing is either a breaking change in npm or a bug that should be reported to them.

In 1st comment I described behavior of underlying npm ([email protected])
In research global npm was used.
Same results.

Badisi · 2022-10-25T16:49:53Z

Just got the exact same issue yesterday and took me the whole day to figure it out.

This has nothing to do with execa and is more related to an npm issue while using workspaces.
You can see the detailed explanation and reproducible repo here : npm/cli#5745.

Nonetheless, this can be fixed in @semantic-release/npm by replacing cwd with cwd: basePath here: publish.js#L26.
As I don't see why the publish command should be executed from a different cwd anyway...

Badisi · 2022-11-01T19:35:45Z

@travi, would you accept a PR to fix that ? (ie. replace cwd with cwd: basePath)

travi · 2022-11-04T19:16:50Z

As I don't see why the publish command should be executed from a different cwd anyway...

are you considering the pkgRoot option in that statement?

This has nothing to do with execa and is more related to an npm issue while using workspaces.

@unlight @NullVoxPopuli are you using npm workspaces in the projects where you encountered this situation?

NullVoxPopuli · 2022-11-04T19:19:27Z

pnpm workspaces, but ya

Badisi · 2022-11-06T11:06:16Z

@travi, yes I'm considering it.

What I meant is:

Looking at the code here (publish.js#L26):

const basePath = pkgRoot ? path.resolve(cwd, pkgRoot) : cwd;
[...]
const result = execa(
  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd, env, preferLocal: true}
);

We can see that :

basePath is the final dist location (nb: it is taking pkgRoot into account)
execa is executed from cwd
npm publish command is given a basePath to publish

But why would execa be executed from a different path than basePath ?

A fix to this issue (also suggested by @unlight) would be to make sure that execa is executed directly from the dist path.

Something like:

  'npm',
  ['publish', basePath, '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}

or simply:

  'npm',
  ['publish', '--userconfig', npmrc, '--tag', distTag, '--registry', registry],
  {cwd: basePath, env, preferLocal: true}

…ublishes from cwd folder

Badisi · 2022-12-03T13:24:27Z

As it is a blocking issue for me, I have made a PR to fix it.

@travi, I would be very pleased if you could approve it. Thanks 🙏

donaldxdonald · 2023-04-07T03:26:13Z

any progress? 💦

Badisi · 2023-04-12T09:04:39Z

For anyone interested

As no action were taken here and it was blocking for me, I'm using this temporary fix :

package.json#L28

{
  "scripts": {
    "postinstall": "node patch-semantic-release-npm.js"
  }
}

patch-semantic-release-npm.js#L1-L24

/**
 *  TODO: this whole patch can be removed when https://github.com/semantic-release/npm/pull/531 is fixed
 */

const { readFileSync, writeFileSync, existsSync } = require('fs-extra');
const { green, red, gray } = require('@colors/colors/safe');
const { dirname, join } = require('path');

const filePath = join(dirname(require.resolve('@semantic-release/npm')), '/lib/publish.js');
if (existsSync(filePath)) {
  let data = readFileSync(filePath, { encoding: 'utf8' });

  if (!data.match(/cwd: basePath/gm)) {
    data = data.replace("'publish', basePath,", "'publish',");
    data = data.replace("cwd, env,", "cwd: basePath, env,");
    writeFileSync(filePath, data, { encoding: 'utf8' });
    console.log(green('success'), '@semantic-release/npm patched.');
  } else {
    console.log(green('success'), '@semantic-release/npm already patched.');
  }
} else {
  console.error(red('error'), 'cannot patch @semantic-release/npm.');
  console.error(gray(`"${filePath}" not found`));
}

unlight changed the title ~~npm publish through execa fails~~ npm publish through execa fails / npm publish <folder> publishes from cwd folder Aug 1, 2022

Badisi mentioned this issue Oct 25, 2022

npm publishing with wrong version tag dhoulb/multi-semantic-release#127

Open

unlight changed the title ~~npm publish through execa fails / npm publish <folder> publishes from cwd folder~~ npm publish fails / npm publish <folder> publishes from cwd folder Oct 25, 2022

Badisi added a commit to Badisi/npm that referenced this issue Dec 3, 2022

fix(semantic-release#504): npm publish fails / npm publish <folder> p…

5a58772

…ublishes from cwd folder

Badisi linked a pull request Dec 3, 2022 that will close this issue

fix(#504): npm publish fails / npm publish <folder> publishes from cwd folder #531

Open

greglbd mentioned this issue May 6, 2023

[bug] semantic-release-monorepo ignores pkgRoot pmowrer/semantic-release-monorepo#121

Open

This was referenced May 14, 2024

pkgRoot property not working #601

Open

[bug] ignores pkgRoot when used with semantic-release-monorepo #511

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

npm publish fails / `npm publish <folder>` publishes from cwd folder #504

npm publish fails / `npm publish <folder>` publishes from cwd folder #504

unlight commented Jul 31, 2022 •

edited

Loading

NullVoxPopuli commented Jul 31, 2022 •

edited

Loading

travi commented Aug 1, 2022 •

edited

Loading

unlight commented Aug 1, 2022 •

edited

Loading

travi commented Aug 1, 2022

unlight commented Aug 1, 2022 •

edited

Loading

unlight commented Aug 1, 2022

Badisi commented Oct 25, 2022 •

edited

Loading

Badisi commented Nov 1, 2022

travi commented Nov 4, 2022

NullVoxPopuli commented Nov 4, 2022

Badisi commented Nov 6, 2022

Badisi commented Dec 3, 2022

donaldxdonald commented Apr 7, 2023

Badisi commented Apr 12, 2023

npm publish fails / npm publish <folder> publishes from cwd folder #504

npm publish fails / npm publish <folder> publishes from cwd folder #504

Comments

unlight commented Jul 31, 2022 • edited Loading

NullVoxPopuli commented Jul 31, 2022 • edited Loading

travi commented Aug 1, 2022 • edited Loading

unlight commented Aug 1, 2022 • edited Loading

travi commented Aug 1, 2022

unlight commented Aug 1, 2022 • edited Loading

unlight commented Aug 1, 2022

Badisi commented Oct 25, 2022 • edited Loading

Badisi commented Nov 1, 2022

travi commented Nov 4, 2022

NullVoxPopuli commented Nov 4, 2022

Badisi commented Nov 6, 2022

Badisi commented Dec 3, 2022

donaldxdonald commented Apr 7, 2023

Badisi commented Apr 12, 2023

For anyone interested

npm publish fails / `npm publish <folder>` publishes from cwd folder #504

npm publish fails / `npm publish <folder>` publishes from cwd folder #504

unlight commented Jul 31, 2022 •

edited

Loading

NullVoxPopuli commented Jul 31, 2022 •

edited

Loading

travi commented Aug 1, 2022 •

edited

Loading

unlight commented Aug 1, 2022 •

edited

Loading

unlight commented Aug 1, 2022 •

edited

Loading

Badisi commented Oct 25, 2022 •

edited

Loading