Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

递归读取chache文件夹的时候非常缓慢 Reading the cache folder recursively is very slow. #24

Open
sunnyhigh521 opened this issue Dec 15, 2024 · 1 comment

Comments

@sunnyhigh521
Copy link

在处理数据的时候有一行代码卡了很久,我检查了一下程序发现卡在了这里。请问有什么解决方案吗?
When processing data, a line of code took a long time, and upon checking the program, I found that it was stuck here. What solutions do you suggest?
1734180112083

@extr15
Copy link

extr15 commented Dec 17, 2024

There are so many '*.gz' files under deep directory, thus the process is too slow.
Actually you can read cache_pluto_1M/metadata/cache_pluto_1M_metadata_node_0.csv directly to get the candidate dirs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants