Skip to content
This repository has been archived by the owner on Mar 11, 2024. It is now read-only.

Upgrade ipfs http client #12

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

wbt
Copy link

@wbt wbt commented Jun 22, 2022

NPM audit is reporting a high severity vulnerability in projects that use this package:

node-forge <=1.2.1
Severity: high
URL parsing in node-forge could lead to undesired behavior. - GHSA-gf8q-jrpm-jvxq
Improper Verification of Cryptographic Signature in node-forge - GHSA-cfm4-qjh2-4765

The import trace is:

-- @truffle/[email protected]
-- [email protected]
-- [email protected]
-- [email protected]
-- [email protected]
-- [email protected]

The issue has been fixed and the bottom three packages updated, but that's kind of irrelevant because peer-id was dropped from ipfs-core-types in this commit on 3/31/21, published in 0.4.0 in this commit 5/10/21. The version of ipfs-http-client released at the same time updates to use that version, and is version 50.0.0. Therefore, I think the minimum version of ipfs-http-client needed to address this particular vuln alert might be v50. That limits the need to update this package for breaking changes to those announced in v49 and v50:

(at v50): Minimum supported node version is 14
(at v50): all core api methods now have types, some method signatures have changed, named exports are now used by the http, grpc and ipfs client modules
(at v49): ipfs-repo upgrade requires repo migration to v10

That generic middle line in the changelog is not very helpful for figuring out what changed after 48.2.2 was published 01/22/21 and they stopped version-tagging that repo at v42. The three-dot syntax doesn't seem to work for comparison between commit hashes (or I'm using it incorrectly), but a workaround by tagging those commits in a fork might.

I'm putting this up for CI testing, hoping that the more limited proposed change set might reach acceptance (including any needed adaptations) more quickly than PR #8 which has been sitting open for two and a half months, to hopefully resolve the high-severity npm audit failure.

@eggplantzzz
Copy link
Contributor

Thanks @wbt! You will probably also need to regenerate the yarn.lock.

@wbt
Copy link
Author

wbt commented Jun 23, 2022

I did run yarn, but the yarn.lock file didn't show up as changed in the Git list.

@cds-amal
Copy link
Member

cds-amal commented Jun 23, 2022

Hi @wbt, thanks for all your effort to improve our dependencies. The way to update the dependencies on this monorepo is to run yarn bootstrap. Please note, you must use npm@6 otherwise your OS will run out of memory. Why? Well, with npm > 6, the transitive dep ursa-optional has a process bomb bug in its prepare/build script that will spawn processes until the OS has no more memory.

I'm hoping your update improves the situation 🤞, IIRC there's some looseness with Semver compliance.

@wbt
Copy link
Author

wbt commented Jun 23, 2022

Hmm, maybe it'd be better to have someone else run that. I can't get my node version manager to correctly downgrade with 6.x not being recognized as a valid version ("Invalid Version: libnpmversion-v3.0.5. Sorry.") The version manager seems to correctly show the right version, but using 'npm -v' directly reveals version 7.24.0, and it's unclear what version 'yarn bootstrap' would trigger. Also, it looks like that step might not be enough.

@wbt
Copy link
Author

wbt commented Jun 23, 2022

One of the breaking changes noted in the changelog is that the minimum Node version is now 14, suggesting that it's not likely to be straightforward to fix that Node 12.x test failure. Is there interest in dropping Node 12 support with a major-version bump, to address the high-severity vulnerability causing audits to fail?
You might even be able to be kinder to potential contributors by updating past Node 15 to get over that process bomb issue.
Or, maybe, you could try convincing any contacts in the IPFS core dev team to issue patches on older major-versions that address the transitive vulnerabilities.

@wbt
Copy link
Author

wbt commented Jun 24, 2022

So I found a note in your 5.5.16 release notes:

⏰ A friendly reminder for everyone that with Node 12 coming to end of life at the end of the month, we will be dropping support for it on May 31, 2022. The recommended Node versions are 14, 16, & 18 with NPM 6. 👩🏻‍💻

That suggests that the dropping of Node 12 support is part of the preannounced plan and removing that test from the matrix should be relatively noncontroversial.

wbt added a commit to wbt/preserves that referenced this pull request Jun 24, 2022
@cds-amal
Copy link
Member

Ok @wbt , I'll update the dependencies and let's see what happens

@cds-amal cds-amal self-assigned this Jun 30, 2022
@wbt
Copy link
Author

wbt commented Jun 30, 2022

OK, thanks. To help keep focus, I would like to make sure we're not holding up a fix on a high-severity vulnerability for Node 18 support (which is certainly nice to have, but probably shouldn't be a blocker on this).

@eggplantzzz
Copy link
Contributor

And just a note @wbt; even though we announced dropping Node 12 support we are kind of waiting on Truffle to be more fully compatible with Node 18 to fully go through with it. We want to give users a big enough time window to upgrade.

@cds-amal
Copy link
Member

cds-amal commented Jul 11, 2022

Hrm, I'm not sure what's going on here. According to the bootstrap output, the yarn.lock file was updated; however, CI shows the yarncheck job as failing. This is unexpected because yarn bootstrap should have resolved dependencies! There are also Typescript failures during bootstrap.

yarn bootstrap

Output from yarn boostrap

$ yarn bootstrap
yarn run v1.22.19
$ npx lerna bootstrap
lerna notice cli v4.0.0
lerna info versioning independent
lerna info bootstrap root only
[1/4] Resolving packages...
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipfs-core-types > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > [email protected]: This module has been superseded by @ipld/dag-pb and multiformats
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > [email protected]: This module has been superseded by @ipld/dag-cbor and multiformats
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipld-raw > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipfs-core-utils > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipld-dag-pb > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipld-dag-cbor > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipld-raw > [email protected]: This module has been superseded by the multiformats module
warning workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs-http-client > ipld-dag-pb > interface-ipld-format > [email protected]: This module has been superseded by the multiformats module
[2/4] Fetching packages...
[3/4] Linking dependencies...
warning "@typescript-eslint/eslint-plugin > [email protected]" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > [email protected]" has unmet peer dependency "go-ipfs@*".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > [email protected]" has unmet peer dependency "ipfs-client@*".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > @textile/hub > @textile/buckets > @improbable-eng/[email protected]" has unmet peer dependency "google-protobuf@^3.2.0".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > @textile/hub > @textile/hub-filecoin > @improbable-eng/[email protected]" has unmet peer dependency "google-protobuf@^3.2.0".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs > ipfs-cli > [email protected]" has unmet peer dependency "electron-webrtc@^0.3.0".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs > ipfs-cli > [email protected]" has unmet peer dependency "wrtc@^0.4.6".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-buckets > ipfs > ipfs-core > [email protected]" has unmet peer dependency "abort-controller@*".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-filecoin > @ganache/filecoin > ipfs > ipfs-cli > [email protected]" has unmet peer dependency "electron-webrtc@^0.3.0".
warning "workspace-aggregator-b943d70c-c989-468f-9c20-6f3606e0be9b > @truffle/preserve-to-filecoin > @ganache/filecoin > ipfs > ipfs-cli > [email protected]" has unmet peer dependency "wrtc@^0.4.6".
[4/4] Building fresh packages...
success Saved lockfile.
$ lerna run prepare --stream --concurrency=1 && husky install
lerna notice cli v4.0.0
lerna info versioning independent
lerna info Executing command in 5 packages: "yarn run prepare"
@truffle/preserve: $ yarn build && yarn docs
@truffle/preserve: $ tsc
@truffle/preserve: $ typedoc --options ./docs/typedoc.json
@truffle/preserve: Error: ../preserve-to-buckets/lib/ipfs-adapter.ts:6:37 - error TS2344: Type 'typeof import("/home/amal/work/preserves/packages/preserve-to-ipfs/node_modules/ipfs-http-client/dist/src/index")' does not satisfy the constraint '(...args: any) => any'.
@truffle/preserve: 6 export type IpfsClient = ReturnType<typeof createIpfsHttpClient>;
@truffle/preserve:                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@truffle/preserve: Error: ../preserve-to-ipfs/lib/connect.ts:20:16 - error TS2349: This expression is not callable.
@truffle/preserve:   Type 'typeof import("/home/amal/work/preserves/packages/preserve-to-ipfs/node_modules/ipfs-http-client/dist/src/index")' has no call signatures.
@truffle/preserve: 20   const ipfs = createIpfsClient({ url });
@truffle/preserve:                   ~~~~~~~~~~~~~~~~
@truffle/preserve: Error: ../preserve-to-ipfs/lib/ipfs-adapter.ts:6:37 - error TS2344: Type 'typeof import("/home/amal/work/preserves/packages/preserve-to-ipfs/node_modules/ipfs-http-client/dist/src/index")' does not satisfy the constraint '(...args: any) => any'.
@truffle/preserve:   Type 'typeof import("/home/amal/work/preserves/packages/preserve-to-ipfs/node_modules/ipfs-http-client/dist/src/index")' provides no match for the signature '(...args: any): any'.
@truffle/preserve: 6 export type IpfsClient = ReturnType<typeof createIpfsHttpClient>;
@truffle/preserve:                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@truffle/preserve: error Command failed with exit code 3.
@truffle/preserve: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
@truffle/preserve: error Command failed with exit code 3.
@truffle/preserve: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
lerna ERR! yarn run prepare exited 3 in '@truffle/preserve'
error Command failed with exit code 3.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
lerna ERR! yarn install --mutex network:42424 --non-interactive --ignore-engines exited 3 in 'root'
lerna ERR! yarn install --mutex network:42424 --non-interactive --ignore-engines exited 3 in 'root'
error Command failed with exit code 3.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

yarncheck

diff of `yarn.lock`

$ yarn install --ignore-scripts --ignore-engines
yarn install v1.22.19
[1/4] Resolving packages...
warning workspace-aggregator-e0af79f4-f7bb-40a6-abff-d096ec7dd73e > @truffle/preserve-to-filecoin > @trufflesuite/filecoin.js > @zondax/filecoin-signing-tools > [email protected]: Critical security vulnerability fixed in v0.21.1. For more information, see https://github.com/axios/axios/pull/3410
[2/4] Fetching packages...
[3/4] Linking dependencies...
-- SNIP --
[4/4] Building fresh packages...
warning Ignored scripts due to flag.
success Saved lockfile.
Done in 7.21s.

$ git diff | cat
diff --git i/yarn.lock w/yarn.lock
index edb9d29..0ad75d9 100644
--- i/yarn.lock
+++ w/yarn.lock
@@ -2946,12 +2946,12 @@ aws4@^1.8.0:
   resolved "https://registry.yarnpkg.com/aws4/-/aws4-1.11.0.tgz#d61f46d83b2519250e2784daf5b09479a8b41c59"
   integrity sha512-xh1Rl34h6Fi1DC2WWKfxUTVqRsNnr6LsKz2+hfwDxQJWmrx8+c7ylaqBMcHfl1U1r2dsifOvKX3LQuLNZ+XSvA==

[email protected]:
-  version "0.26.1"
-  resolved "https://registry.yarnpkg.com/axios/-/axios-0.26.1.tgz#1ede41c51fcf51bbbd6fd43669caaa4f0495aaa9"
-  integrity sha512-fPwcX4EvnSHuInCMItEhAGnaSEXRBjtzh9fOtsE6E1G6p7vl7edEeZe11QHf18+6+9gR5PbKV/sGKNaD8YaMeA==
+axios@^0.20.0:
+  version "0.20.0"
+  resolved "https://registry.yarnpkg.com/axios/-/axios-0.20.0.tgz#057ba30f04884694993a8cd07fa394cff11c50bd"
+  integrity sha512-ANA4rr2BDcmmAQLOKft2fufrtuvlqR+cXNNinUmvfeSNCOF98PZL+7M/v1zIdGo7OLjEA9J2gXJL+j4zGsl0bA==
   dependencies:
-    follow-redirects "^1.14.8"
+    follow-redirects "^1.10.0"

 babel-jest@^26.6.3:
   version "26.6.3"
@@ -5359,10 +5359,10 @@ fnv1a@^1.0.1:
   resolved "https://registry.yarnpkg.com/fnv1a/-/fnv1a-1.1.1.tgz#4e01d51bae60735d00e54ffde02581fe2e74f465"
   integrity sha512-S2HviLR9UyNbt8R+vU6YeQtL8RliPwez9DQEVba5MAvN3Od+RSgKUSL2+qveOMt3owIeBukKoRu2enoOck5uag==

-follow-redirects@^1.14.8:
-  version "1.14.9"
-  resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.14.9.tgz#dd4ea157de7bfaf9ea9b3fbd85aa16951f78d8d7"
-  integrity sha512-MQDfihBQYMcyy5dhRDJUHcw7lb2Pv/TuE6xP1vyraLukNDHKbDxDNaOE3NbCAdKQApno+GPRyo1YAp89yCjK4w==
+follow-redirects@^1.10.0:
+  version "1.15.1"
+  resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.15.1.tgz#0ca6a452306c9b276e4d3127483e29575e207ad5"
+  integrity sha512-yLAMQs+k0b2m7cVxpS1VKJVvoz7SS9Td1zss3XRwXj+ZDH00RJgnuLx7E44wx02kQLrdM3aOOy+FpzS7+8OizA==

 for-in@^1.0.2:
   version "1.0.2"

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants