Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternatives to Cloudflare for bot protection #143

Open
m-i-l opened this issue Jan 31, 2024 · 0 comments
Open

Alternatives to Cloudflare for bot protection #143

m-i-l opened this issue Jan 31, 2024 · 0 comments

Comments

@m-i-l
Copy link
Contributor

m-i-l commented Jan 31, 2024

As per https://blog.searchmysite.net/posts/almost-all-searches-on-my-independent-search-engine-are-now-from-seo-spam-bots/ and https://blog.searchmysite.net/posts/an-update-on-the-automated-seo-searches-issue/ , and in common with other search engines, searchmysite.net goes through phases where it is periodically hit with 100s of 1000s of searches for "scraping footprints" which come from botnets which are difficult to identify and block. So I have put searchmysite.net behind Cloudflare to try to mitigate.

The problem is firstly that Cloudflare introduces privacy concerns, e.g. https://fedi.at/@digitalRightsNinja/111844521305855609 .

And secondly, putting it behind Cloudflare doesn't really solve the problem, because Cloudflare is really for blocking good bots (which aren't the problem but even if they were they're relatively easy to deal with without Cloudflare) and not the bad bots.

There have been suggestions for alternatives, e.g. Fastly and Deflect.ca. Having a quick look they all seem to be paid for (I'm using the free Cloudflare) and I'm not sure any of them will be any better at identifying these botnets (since the whole point of the botnets is to evade detection), but making a note to investigate further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant