Adding matching type URL to warninglists

[HugeekMcGill]

All MISP Warninglists are not created equal. MISP warninglists have their characterization defined by specifying types in their “matching_attributes” --> Ex: "matching_attributes": ["domain", "hostname", "url" ],

The issue we are seeing is that a domain can also be an URL but some warninglists are missing the “matching_attributes” “URL” so the IoC will not be blocked if the type is “URL”.

Like in the below example, if the attribute (IoC) type is URL and the warninglisst doesn’t have the “matching_attributes” “URL”, the warninglists its ignored by MISP.

• The attribute (IoC) “0f3kjf7t0dbj.wpeproxy.com” with type “domain” and “URL” is part of a warninglist with “domain” and “URL” matching_attributes. This causes the warninglist to block both attributes since it’s a protected domain and URL.
  --> "matching_attributes": ["domain", "hostname", "url"]

• The attribute (IoC) “fs.microsoft.com” with type “domain” and “URL” is part of LIST OF KNOWN WINDOWS 10 CONNECTION ENDPOINTS with only the “domain” attributes. This causes the warninglists to block only the domain attributes since it’s a protected domain. The URL IoC will not be blocked since the warninglists haven’t detected the “URL” type.
  --> "matching_attributes": ["domain", "hostname", "domain|ip"]

[adulau]

Good catch indeed. Maybe we should update first the generator script to make it persistent. I'll have a look. Thanks a lot.

[HugeekMcGill]

@adulau, tell me where to check if you need help on patching the generators. I'm also investigation this week at other issues in the warning-list items.