[BUG] Yarn lockfile validation fails when using a symlink "yarn link" local package

[adamdmharvey]

Important: Bug reports should be directed to [email protected] for faster response and prioritization. Bug reports may still be created here but they may not get immediate attention and may be closed in favor of internally tracked tickets.

Describe the bug
The Backstage CNCF project uses Node.js and Yarn for dependency management. It is a monorepo, and contains two local packages which are not published and are referenced locally by the project using the yarn link feature.

These two packages keep failing a Lockfile validation during a fossa analyze.

It's specifically this package.json dependency reference:

https://github.com/backstage/backstage/blob/5623a818b957b0ef3d44434cd4b8afbf9ae43f7d/packages/techdocs-cli/package.json#L50

Which then scans against these values in the yarn.lock:

"techdocs-cli-embedded-app@link:../techdocs-cli-embedded-app::locator=%40techdocs%2Fcli%40workspace%3Apackages%2Ftechdocs-cli":
  version: 0.0.0-use.local
  resolution: "techdocs-cli-embedded-app@link:../techdocs-cli-embedded-app::locator=%40techdocs%2Fcli%40workspace%3Apackages%2Ftechdocs-cli"
  languageName: node
  linkType: soft

[...snip...]

"@techdocs/cli@workspace:*, @techdocs/cli@workspace:packages/techdocs-cli":
  version: 0.0.0-use.local
  resolution: "@techdocs/cli@workspace:packages/techdocs-cli"
  dependencies:
    "@backstage/backend-common": "workspace:^"
    "@backstage/catalog-model": "workspace:^"
    "@backstage/cli": "workspace:^"
    "@backstage/cli-common": "workspace:^"
    "@backstage/config": "workspace:^"
    "@backstage/plugin-techdocs-node": "workspace:^"
    "@frsource/cypress-plugin-visual-regression-diff": ^3.2.8
    "@types/commander": ^2.12.2
    "@types/dockerode": ^3.3.0
    "@types/fs-extra": ^9.0.6
    "@types/http-proxy": ^1.17.4
    "@types/node": ^16.11.26
    "@types/serve-handler": ^6.1.0
    "@types/webpack-env": ^1.15.3
    commander: ^9.1.0
    cypress: ^10.0.0
    dockerode: ^3.3.1
    find-process: ^1.4.5
    fs-extra: ^10.0.1
    global-agent: ^3.0.0
    http-proxy: ^1.18.1
    nodemon: ^2.0.2
    react-dev-utils: ^12.0.0-next.60
    serve-handler: ^6.1.3
    techdocs-cli-embedded-app: "link:../techdocs-cli-embedded-app"
    ts-node: ^10.0.0
    winston: ^3.2.1
  bin:
    techdocs-cli: bin/techdocs-cli
  languageName: unknown
  linkType: soft

The resulting summary output is:

Scan Summary
------------
fossa-cli version 3.8.6 (revision b2657cb78351 compiled with ghc-9.0)
fossa endpoint server version: 4.12.12

7 projects scanned;  6 skipped,  0 succeeded,  1 failed,  0 analysis warnings

* yarn project in "/Users/adaharve/Documents/github/backstage-backstage/": failed
* yarn project in "/Users/adaharve/Documents/github/backstage-backstage/cypress/": skipped (exclusion filters)
* yarn project in "/Users/adaharve/Documents/github/backstage-backstage/microsite/": skipped (exclusion filters)
* npm project in "/Users/adaharve/Documents/github/backstage-backstage/packages/cli/asset-types/": skipped (exclusion filters)
* npm project in "/Users/adaharve/Documents/github/backstage-backstage/packages/create-app/templates/default-app/examples/template/content/": skipped (exclusion filters)
* npm project in "/Users/adaharve/Documents/github/backstage-backstage/packages/eslint-plugin/src/__fixtures__/monorepo/": skipped (exclusion filters)
* yarn project in "/Users/adaharve/Documents/github/backstage-backstage/storybook/": skipped (exclusion filters)
-
----------
yarn project in "/Users/adaharve/Documents/github/backstage-backstage/": failed

>>> Relevant errors

  Error

    Couldn't find package for descriptor: Descriptor {descriptorScope = Nothing, descriptorName = "techdocs-cli-embedded-app", descriptorRange = "link:../techdocs-cli-embedded-app"}

    Traceback:
      - Validating lockfile
      - Lockfile V2 analysis
      - Project Analysis: YarnProjectType

  Error

    Couldn't find package for descriptor: Descriptor {descriptorScope = Nothing, descriptorName = "example-app", descriptorRange = "link:../app"}

    Traceback:
      - Validating lockfile
      - Lockfile V2 analysis
      - Project Analysis: YarnProjectType

To Reproduce
Steps to reproduce the behavior:

1. Fork https://github.com/backstage/backstage
2. Clone your fork
3. Use your FOSSA API Key
4. Use this config file:

version: 3

project:
  link: https://backstage.io
  url: github.com/FORKUSER/backstage

targets:
  only:
    - type: yarn
      path: .
    - type: npm
      path: packages/
    - type: yarn
      path: packages/
    - type: yarn
      path: plugins/
    - type: npm
      path: plugins/
  exclude:
    - type: yarn
      path: packages/app
    - type: yarn
      path: packages/backend

6. Run a fossa analyze --debug against the repo
7. Result is a failure on the monorepo for validating the lock file.

Expected behavior
Lockfile validation should succeed.

Debug bundle
fossa.debug.json.gz

Additional context
This was potentially previously reported here in Nov 2022 but the issue was closed: #1111

The Backstage project is working to enable FOSSA scanning to support it's CNCF CLO Compliance via backstage/backstage#18821.

I wasn't sure if there's some sort of exlusion we could add, but it seems it's just the core validation of the lockfile itself which fails regardless if we exclude the relevant packages in the target or via paths.

[github-actions]

Thank you @adamdmharvey for creating this issue. If this is in regards to a defect, product question or feature request: you should use our support portal at https://support.fossa.com to file a request, as you would receive more immediate support.

[adamdmharvey]

(As per issue template, I also e-mailed support w/ reference to this GH issue.)

[adamdmharvey]

Cross ref: https://support.fossa.com/hc/en-us/requests/6722

Note using the FOSSA UI, and importing my personal fork of the Backstage project, the full analysis (minus my exclusions) seemed to all validate successfully. So it's almost like the CLI is acting differently from the built-system through the site?

e.g., for FOSSA folks who may have access: https://app.fossa.com/projects/git%2Bgithub.com%2Fadamdmharvey%2Fbackstage/refs/branch/master/5623a818b957b0ef3d44434cd4b8afbf9ae43f7d