sf project generate manifest returns inconsistent results

[jakubmplatek]

Note
Before you submit your issue, make sure that:

• You're using the latest version of Salesforce CLI.
• You've searched both open and closed issues for related posts.
• You've used the doctor command to diagnose common issues.
• You understand that GitHub Issues don't adhere to any agreement or SLA.
  • If you require immediate assistance, use official channels such as Salesforce Customer Support.

Summary

I'm building automation to run scheduled snapshots of all metadata in my orgs, using sf project generate manifest --from-org command as a base for generating batches of retrieve calls.
I noticed some unexpected changes listed between runs, and found that that initial package manifest generated is inconsistent: it will randomly return a different count of members for a type, or skip a type entirely.

I've tested it by running the generate manifest command in a loop against the same environment and then comparing the generated packages, and found individual runs completely missing some types, including:
ApexClass,MatchingRule,CustomObject,ValidationRule,WebLink,AppMenu,CustomField,CompactLayout,CustomPermission,ListView,CustomMetadata

Tested on different instances, both in prod and sandbox orgs, and in different sf projects, all using the latest cli (2.69.14)

Steps To Reproduce

IMPORTANT
Provide a repository that's configured to reproduce the issue. If you are unable to provide a repo, please explain why not. The more info we have from the start, the faster we can resolve your issue.
We may close your issue if you don't include proper instructions.

• Generate a project with sf project generate or fork dreamhouse-lwc.
• Provide detailed step-by-step instructions on how to reproduce the issue.

authenticate with a salesforce org and run the command a few times to generate multiple manifests (date used to autogenerate a different filename each time)
sf project generate manifest --from-org <alias> --name package_$( date +%s ).xml
compare the contents of the generated package manifests - some of them will have whole metadata types missing, or some members within a metadata type missing

note: the issue is intermittent, so it might take 5-10 runs to see significant differences

Tip

use sf doctor --create-issue to automatically fill the required information

logs generated by doctor:
logs.zip

Expected result

while testing in an org without active users (i.e. no changes to metadata), I expect each package manifest to have strictly equal contents

Actual result

components listed differ between generated manifests

Additional information

System Information

Windows 10, Git for Windows bash

{
    "architecture": "win32-x64",
    "cliVersion": "@salesforce/cli/2.69.14",
    "nodeVersion": "node-v20.11.0",
    "osVersion": "Windows_NT 10.0.19045",
    "rootPath": "C:\\Users\\jplatek\\AppData\\Roaming\\npm\\node_modules\\@salesforce\\cli",
    "shell": "C:\\Program Files\\Git\\usr\\bin\\bash.exe",
    "pluginVersions": [
      "@copado/copado-cli 3.3.0 (user)",
      "@oclif/plugin-autocomplete 3.2.11 (core)",
      "@oclif/plugin-commands 4.1.12 (core)",
      "@oclif/plugin-help 6.2.18 (core)",
      "@oclif/plugin-not-found 3.2.29 (core)",
      "@oclif/plugin-plugins 5.4.19 (core)",
      "@oclif/plugin-search 1.2.16 (core)",
      "@oclif/plugin-update 4.6.14 (core)",
      "@oclif/plugin-version 2.2.16 (core)",
      "@oclif/plugin-warn-if-update-available 3.1.24 (core)",
      "@oclif/plugin-which 3.2.19 (core)",
      "@salesforce/cli 2.69.14 (core)",
      "apex 3.6.6 (core)",
      "api 1.3.2 (core)",
      "auth 3.6.78 (core)",
      "data 3.12.3 (core)",
      "deploy-retrieve 3.15.17 (core)",
      "info 3.4.25 (core)",
      "limits 3.3.42 (core)",
      "marketplace 1.3.7 (core)",
      "org 5.2.7 (core)",
      "packaging 2.9.5 (core)",
      "schema 3.3.43 (core)",
      "settings 2.4.8 (core)",
      "sobject 1.4.46 (core)",
      "telemetry 3.6.26 (core)",
      "templates 56.3.33 (core)",
      "trust 3.7.45 (core)",
      "user 3.6.4 (core)",
      "@salesforce/sfdx-scanner 4.7.0 (user)",
      "sfdmu 4.35.1 (user)",
      "sfdx-git-delta 5.42.1 (user)"
    ]
  }

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.