fix(docx): PAPP-35228 Make the parser smart enough to extract IOCs from docx files with embedded HTML

[phantom-jacob]

Pull Request Type

Bugfix: A customer reported that not all link targets were being extracted from their docx files. Upon further examination, it became apparent that the docx files for which the parser connector was failing to extract all IOCs were built in an uncommon way: rather than text (and links) being written directly into the docx file, the document had embedded an HTML page. Under the hood, Microsoft stores that HTML separately from the text of a docx file, so we were simply not parsing it at all.

Release Notes

PAPP-35228 Fix: Extract Indicators of Compromise from docx files with embedded HTML

What is the current behavior?

The parser connector does not attempt to parse HTML embedded in docx files when running the extract ioc action

What is the new behavior?

Now, the connector does parse both the core document and embedded HTML sections of docx files

Other information

• While I was in the connector, I added type hints to all the functions defined within. I also took advantaged of TypedDict and dataclass to make the connector code significantly clearer
• I had to move parameter validation for the extract ioc action into a separate function, as the pre-commit hook got upset at the complexity of the original function
• I added a new dependency, python-docx. I confirmed it has an MIT license. Related to this change, the defusedxml dependency was dropped, as it was only used for manual parsing of docx files. I figured the interface provided by python-docx was much more ergonomic and less error prone
• I removed the simplejson dependency as the standard library json package is the same thing
• The only real functional changes in this PR are in the _docx_to_text() function of parser_methods.py