A Flake8 plugin to catch common issues on Scrapy spiders.
| Code | Meaning |
|---|---|
| SCP01 | There are URLs in start_urls whose netloc is not in allowed_domains |
| SCP02 | There are URLs in allowed_domains |
| SCP03 | Usage of urljoin(response.url, '/foo') instead of response.urljoin('/foo') |
| SCP04 | Usage of Selector(response) in callback |
This is a work in progress, so new issues will be added to this list.
To run this in your project, please make sure you have flake8 installed first:
$ pip install flake8
And then install flake8-scrapy:
$ pip install flake8-scrapy
Now, all you have to do is run it on your project:
$ flake8
And Flake8 will run the checks defined in this plugin.