Cloning fails due to invalid path

[lukas-j94]

Hi, I tried to clone the repository with git clone --recursive https://github.com/dcaribou/transfermarkt-datasets.git, but I get the following message:

Cloning into 'transfermarkt-datasets'...
remote: Enumerating objects: 1467, done.
remote: Counting objects: 100% (605/605), done.
remote: Compressing objects: 100% (303/303), done.
remote: Total 1467 (delta 354), reused 461 (delta 288), pack-reused 862
Receiving objects: 100% (1467/1467), 2.94 MiB | 10.73 MiB/s, done.
Resolving deltas: 100% (697/697), done.
error: invalid path 'streamlit/pages/03_📈_analysis:_player_value.py'
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Even though is says clone succeeded, the transfermarkt-datasets folder is actually empty apart from the .git folder.

Any idea what could be the issue here? Any help would be much appreciated!

[dcaribou]

Hi @lukas-j94. May I ask on which system are you? Mac, Windows, Linux...

[lukas-j94]

I am on Windows

[dcaribou]

I think it has to do with the special character in the file name streamlit/pages/03_📈_analysis:_player_value.py. Can you try running

git config core.protectNTFS false

Source here

[lukas-j94]

it worked now, thanks! I followed those steps which are mentioned in the source you provided:

1. Clone the Git repository locally;
2. 'cd' into the local Git repository folder that has just been cloned;
3. Run git reset
4. Run git config core.protectNTFS false
5. Run git checkout (just git checkout, no * at the end of command).

I have set up the environment and tried to run python 1_acquire.py local --asset all --season 2022, but it gives this error log:

Unhandled error in Deferred:
2022-09-15 16:43:39 [twisted] CRITICAL: Unhandled error in Deferred:

Traceback (most recent call last):
File "C:\Users\lukas.jansen\PycharmProjects\transfermarkt-datasets\1_acquire.py", line 188, in
arguments.func(**vars(arguments))
File "C:\Users\lukas.jansen\PycharmProjects\transfermarkt-datasets\1_acquire.py", line 123, in acquire_on_local
crawl()
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\twisted\internet\defer.py", line 1905, in unwindGenerator
return _cancellableInlineCallbacks(gen)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\twisted\internet\defer.py", line 1815, in _cancellableInlineCallbacks
_inlineCallbacks(None, gen, status)
--- ---
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\twisted\internet\defer.py", line 1660, in _inlineCallbacks
result = current_context.run(gen.send, result)
File "C:\Users\lukas.jansen\PycharmProjects\transfermarkt-datasets\1_acquire.py", line 120, in crawl
yield runner.crawl(asset_obj.name, parents=asset_obj.parent.file_full_path)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\crawler.py", line 204, in crawl
crawler = self.create_crawler(crawler_or_spidercls)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\crawler.py", line 237, in create_crawler
return self._create_crawler(crawler_or_spidercls)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\crawler.py", line 241, in _create_crawler
spidercls = self.spider_loader.load(spidercls)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\spiderloader.py", line 77, in load
raise KeyError(f"Spider not found: {spider_name}")
builtins.KeyError: 'Spider not found: games'
result = current_context.run(gen.send, result)
File "C:\Users\lukas.jansen\PycharmProjects\transfermarkt-datasets\1_acquire.py", line 120, in crawl
yield runner.crawl(asset_obj.name, parents=asset_obj.parent.file_full_path)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\crawler.py", line 204, in crawl
crawler = self.create_crawler(crawler_or_spidercls)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\crawler.py", line 237, in create_crawler
return self._create_crawler(crawler_or_spidercls)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\crawler.py", line 241, in _create_crawler
spidercls = self.spider_loader.load(spidercls)
File "C:\Users\lukas.jansen\Anaconda3\envs\transfermarkt-datasets\lib\site-packages\scrapy\spiderloader.py", line 77, in load
raise KeyError(f"Spider not found: {spider_name}")
KeyError: 'Spider not found: games'

[dcaribou]

Did you use the --recursive option at clone? There should be a folder transfermarkt-scraper in the project with some files in it

[lukas-j94]

Yes, I used the --recursive option, but for some reason it did not work so I cloned the transfermarkt-scraper manually inside the transfermarkt-scraper folder. Then I had another folder called transfermarkt-scraper inside the transfermarkt-scraper folder. It seems like that caused the issue, I now moved the files out of that lower-level folder and it all works.

Thanks a lot for your support, much appreciated!

[dcaribou]

No worries @lukas-j94, happy to help