Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added optional source url restrictions to Rule #1

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Added optional source url restrictions to Rule #1

wants to merge 3 commits into from

Commits on Oct 27, 2015

  1. Added optional source url restrictions to Rule 

    - added Rule args allow_sources and deny_sources (paralleling LinkExtractor's allow & deny)
- added Rule method source_allowed to check if url is allowed/denied
    @fraserharris
    fraserharris committed Oct 27, 2015
    Configuration menu
    Copy the full SHA
    78c25c3 View commit details
    Browse the repository at this point in the history

  2. CrawlSpider skips restricted rules

    @fraserharris
    fraserharris committed Oct 27, 2015
    Configuration menu
    Copy the full SHA
    288e98c View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2015

  1. Added tests for allow_sources and deny_sources

    @fraserharris
    fraserharris committed Oct 28, 2015
    Configuration menu
    Copy the full SHA
    424f7aa View commit details
    Browse the repository at this point in the history