Presidio Image Redactor - improve scalability and design #1049
Replies: 2 comments 5 replies
-
|
Hello! I'm trying to get a better understanding of how the image processing flow happens from the beginning up to the class RecognizerBase(ABC):
"""A class representing an abstract visual objects recognizer.
"""
@abstractmethod
def recognize(self, image: object) -> List[RecognizerResult]:
"""Recognize visual objects
:param image: PIL Image/numpy array to be processed
:return: List of the recognized objects
"""
...
class RecognizerResult:
"""Represent the results of analysing the image by recognizer.
"""
entity_type: str
reconizer_name: str
text: Optional[str] = None
bbox: Bbox
polygon: Optional[Polygon] = NoneIn the # For this the result obtained by TesseractOCR...
RecognizerResult:
entity_type: "text"
reconizer_name: TesseractOCR
text: "My name is Jhon Doe, My phone number is 212-555-5555"
bbox: Bbox
# we get two PII entities by presidio-analyzer...
[
type: PERSON, start: 11, end: 19, score: 0.85,
type: PHONE_NUMBER, start: 40, end: 52, score: 0.75
]
# Now we need to somehow map them back to the boxes/polygonsAnd depending on the type of recognizer, the mapping logic may vary (for example, it will be different for the TesseractOCR and QR codes). One of the options is to create a separate class for recognizers who deal with text data class TextRecognizer(RecognizerBase, ABC):
@abstractmethod
def map_pii_to_boxes(
result: List[RecognizerResult], pii: List[PresidioResults]
) -> List[RecognizerResult]:
...Or something like that. Do you guys see the image processing flow roughly in the same direction? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @vpvpvpvp, thanks for reviewing this! I agree that the mapping is not that straightforward and dependent on the recognizer itself. Having all text recognizers expose another method for mapping is a good idea. We would need to let every result know from which recognizer it came from (we do a similar thing in |
Beta Was this translation helpful? Give feedback.
-
|
Alternatively we can couple the text with its bounding box and send it to |
Beta Was this translation helpful? Give feedback.
-
|
Or, it is possible just to make class TextRecognizer(RecognizerBase):
def __init__(self, presidio_analyzer: AnalyzerEngine):
self.presidio_analyzer = presidio_analyzer
....Regarding how to map the result back to the recognizer it came from. The simplest thing is to just store a field like recognizer/recognizer_name in the |
Beta Was this translation helpful? Give feedback.
-
|
Yes, that's a good idea! Thanks |
Beta Was this translation helpful? Give feedback.
-
|
@vJenny @shanepeckham if you have some time, I'd be very interested to get your input on this design, especially to see if it helps in any way to integrate your obfuscation code sometime in the future. |
Beta Was this translation helpful? Give feedback.
-
Context / Problem Statement
The Presidio Image Redactor package was developed as beta as part of the effort on Presidio V2 in January 2021. It features a simple OCR pipeline which extracts text, parses it, and sends it to the Presidio Analyzer package. Once PII is identified, bounding boxes are matched with
RecognizerResultobjects and those bounding boxes are redacted.Two significant contributions to the package, one focusing on DICOM and the other on QR scanning, in addition to limited accuracy and extensibility due to the existing design, express the need to improve the package's performance and generalizability to new use cases, starting from DICOM and going to non-textual PII objects such as faces, or semi-textual objects like QR codes or license plate numbers.
This ADR proposes an architecture change to
presidio-image-redactorto make it more similar to the structure ofpresidio-analyzerandpresidio-anonymizerthat are used in the text version of Presidio.This is the proposed high-level flow:
flowchart LR style A fill:#bbf,color:#fff style B fill:#bbf,color:#fff style C fill:#bbf,color:#fff style User fill:#bbf,color:#fff User subgraph ImageAnalyzerEngine subgraph TextInImageRecognizers OCRTextRecognizer QRTextRecognizer ... end subgraph PiiObjectsRecognizers FacesRecognizer LicensePlateRecognizer .... end TextInImageRecognizers --> match_pii_to_boxes PiiObjectsRecognizers --> match_pii_to_boxes end User -- "PII in text" --> TextInImageRecognizers User -- "PII in visual objects" --> PiiObjectsRecognizers match_pii_to_boxes --> A["return raw results"] match_pii_to_boxes --> B["return redacted image"] match_pii_to_boxes --> C["overlay detected PII on image"]And in more detail:
stateDiagram-v2 FileReader state ImageAnalyzerEngine { state TextInImageRecognizers { QRTextRecognizer OCRBase --> TesseractOcr OCRBase --> AzureFormsRecognizer OCRBase --> AzureOCR OCRBase --> ... TesseractOcr --> OCRTextRecognizer AzureOCR --> OCRTextRecognizer TesseractOcr --> DicomTextRecognizer AzureOCR --> DicomTextRecognizer AzureFormsRecognizer --> FRPiiRecognizer QRTextRecognizer --> presidio_analyzer FRPiiRecognizer --> presidio_analyzer DicomTextRecognizer --> presidio_analyzer OCRTextRecognizer --> presidio_analyzer } state PiiObjectRecognizers { FacesRecognizer LicensePlatesRecognizer CustomPiiObjectRecognizer .... } TextInImageRecognizers --> match_pii_to_boxes PiiObjectRecognizers --> match_pii_to_boxes } FileReader --> ImageAnalyzerEngine state ImageAnonymizerEngine { redact_image highlight_bounding_boxes } ImageAnalyzerEngine --> ImageAnonymizerEngine redact_image --> return_redacted_image highlight_bounding_boxes --> return_image_with_highlights ImageAnalyzerEngine --> return_pii_locationsLike the structure of the
AnalyzerEngineobject, the newImageAnalyzerEnginewould contain a list of recognizers, one for each type of detection logic.The recognizer would contain the logic for:
The output of the
ImageAnalyzerEnginewould then be inputted into theImageAnonymizerEngine, that would expose oprators such asredactorvalidate. Users would be able to create additional types of operators.Consequences
Links
Beta Was this translation helpful? Give feedback.
All reactions