We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Once the testing is done (#9), we create a release (#23), and start the crawl per this issue.
The countries to crawl are:
The top 525 sites for each country are listed in this repo.
In addition to each countries' top 525 sites we also crawl the United States top 525 list for each country as a general list.
This will lead to 19*525 = 9,975 crawled sites in total.
@atlasharry identified the Google Clould VMs for each country:
@atlasharry will take the lead on the crawl and organize how others can help (however it makes sense).
The text was updated successfully, but these errors were encountered:
SebastianZimmeck
atlasharry
PattonYin
No branches or pull requests
Once the testing is done (#9), we create a release (#23), and start the crawl per this issue.
1. Countries
The countries to crawl are:
2. Sites
The top 525 sites for each country are listed in this repo.
In addition to each countries' top 525 sites we also crawl the United States top 525 list for each country as a general list.
This will lead to 19*525 = 9,975 crawled sites in total.
3. Google Cloud VM
@atlasharry identified the Google Clould VMs for each country:
@atlasharry will take the lead on the crawl and organize how others can help (however it makes sense).
The text was updated successfully, but these errors were encountered: