Filter IP addresses from known scanners

[mlodic]

We should periodically download this batch of data: https://raw.githubusercontent.com/stamparm/maltrail/master/trails/static/mass_scanner.txt
and add those IP to whitelists to reduce number of false positives

[MatheuslFavaretto]

If you want to filter out IP addresses from known scanners, you can use a tool or a database that maintains a list of these IP addresses. One popular tool for this purpose is Fail2Ban, which is an open-source intrusion prevention software that can detect and block attempts to access your system from known malicious IP addresses.

Fail2Ban uses a set of rules, known as filters, to scan log files and block access from IP addresses that match these rules. You can configure Fail2Ban to use a filter that includes a list of known scanner IP addresses, which will help prevent these scanners from accessing your system.

Another option is to use a database of known scanner IP addresses to filter them out using your own custom code. There are several databases available that maintain lists of IP addresses associated with malicious activity, such as the OpenBL and Blocklist.de. You can use these databases to create your own filter that blocks IP addresses from known scanners.

In addition, you can use the Django middleware to filter IP addresses from known scanners. The middleware is a way to add functionality to the request/response processing pipeline in Django. You can write custom middleware that checks the IP address of each request and blocks access from known scanner IP addresses.

Here's an example of how you can use middleware to filter IP addresses from known scanners:

# Create a list of known scanner IP addresses
known_scanner_ips = ['1.2.3.4', '5.6.7.8', '9.10.11.12']

# Define custom middleware
class FilterScannerMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        # Get the IP address of the request
        ip_address = request.META.get('REMOTE_ADDR')
        
        # Check if the IP address is in the list of known scanner IPs
        if ip_address in known_scanner_ips:
            return HttpResponse('Access Denied')

        # If the IP address is not in the list of known scanner IPs, allow the request to proceed
        response = self.get_response(request)
        return response

You can add this middleware to your Django settings file to enable it. The middleware will then check the IP address of each request and block access from any IP addresses in the list of known scanner IPs.

[mlodic]

hey, thanks for your interest. Anyway I think we should consider the context of this issue that seems to be misunderstood considering what you have written. This is a Threat Intel Platform and we are talking about the IP addresses stored by the DB, not the ones who connect to the application. We don't want to stop external IP addresses. We just want to categorize them correctly.
So what you have suggested, while it is super useful in other contexts, does not apply to this one.