Refactored recordedfuture.py added rfpoller class, fixed connection i…

[recordedfuture-simonhornestedt]

Description

• Refactored the code and created RFPoller class, it is an adaption of the existing ft/csv.py that was used previously. But with functionality specific to the recorded future miners Much of the duplicated code that existed in IPRiskList, DomainRiskList and MasterRiskList have been moved to the RFPoller for more readable code.

• Implemented the ignore_regex conf option to function properly on the response from the API

• Removed the decode_gzip conf option since this is handled by requests

• Fixed an issue of Connection Broken: IncompleteRead errors that occured when streaming the response to a DictReader by adding a cStringIO as a sort of buffer.

• Fixed a faulty param sent ("output_format" -> "format") in the request to the API

Types of changes

• Bug fix (non-breaking change which fixes an issue)

Checklist

• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes if appropriate.
• All new and existing tests passed.

[welcome-to-palo-alto-networks]

🎉 Thanks for opening this pull request! We really appreciate contributors like you! 🙌