[LFS] Smudge error: Error downloading

[JanuszL]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

It is related to #1462 and the #5202.
As I understand conda makes a mirror copy of the git repo, fetches relevant LFS objects, and then uses, this local copy of the repo to do the full clone. However, if the ref that is about to be built doesn't have LFS objects the default branch of the repo uses we get Smudge error: Error downloading. I believe it happens because the copy from the local repo mirror will checkout the default branch that may refer to different LFS objects than conda fetches during mirroring only for selected ref.
A simple repro outside of the conda env (but the process is the same):

docker run --rm -ti ubuntu:22.04
apt update && apt install -y git-lfs
git clone --mirror https://github.com/NVIDIA/nvImageCodec
cd nvImageCodec
git lfs fetch origin v0.2.0
git clone /nvImageCodec.git/ /opt/test

Downloading example/assets/images/Weimaraner.bmp (1.6 MB)
Error downloading object: example/assets/images/Weimaraner.bmp (0750c51): Smudge error: Error downloading example/assets/images/Weimaraner.bmp (0750c51191ae72e771e3f1f85c60707121f87af2fe67bbf74b6d30820f0b75f6): batch request: missing protocol: "/nvImageCodec.git/info/lfs"

Errors logged to /opt/test/.git/lfs/logs/20240809T143311.386877822.log
Use `git lfs logs last` to view the log.

The same I see in the conda build process.
Maybe it should just (as described here):

git init 
and then use

git fetch url-to-repo branchname:refs/remotes/origin/branchname

Conda Info

No response

Conda Config

No response

Conda list

No response

Additional Context

No response

[JanuszL]

Related to #1462.