Datamodel docs #7

0ssigeno · 2024-10-21T08:55:19Z

No description provided.

mlodic · 2024-10-21T14:30:55Z

docs/IntelOwl/contribute.md

+```python3
+{"query_status": "no_results"}
+```
+meaning that we can provide use the following code to consider only _real_ results:


Suggested change

meaning that we can provide use the following code to consider only _real_ results:

meaning that we can use the following code to consider only _real_ results:

mlodic · 2024-10-21T14:31:35Z

docs/IntelOwl/contribute.md

+If you specify a path that is not present in the `DataModel`, an error will be added to the job.
+If you specify a path that is not present in the `AnalyzerConfig`, a warning will be added to the job.
+### Analyzer._do_create_data_model
+This is a function that every `Analyzer` can override: this functions returns a boolean and, if `False`, the datamodel will not be created.


Suggested change

This is a function that every `Analyzer` can override: this functions returns a boolean and, if `False`, the datamodel will not be created.

This is a function that every `Analyzer` can override: this function returns a boolean and, if `False`, the datamodel will not be created.

mlodic · 2024-10-21T14:32:14Z

docs/IntelOwl/contribute.md

+### Analyzer._do_create_data_model
+This is a function that every `Analyzer` can override: this functions returns a boolean and, if `False`, the datamodel will not be created.
+This can be used if the `Analyzers` can succeed without retrieving useful results.
+Let's use as an example `UrlHaus`: if the domain analyzed is not present in its database, the result will be


Suggested change

Let's use as an example `UrlHaus`: if the domain analyzed is not present in its database, the result will be

Let's use `UrlHaus` as an example : if the domain analyzed is not present in its database, the result will be

mlodic · 2024-10-21T14:33:28Z

docs/IntelOwl/contribute.md

+If you specify a path that is not present in the `AnalyzerConfig`, a warning will be added to the job.
+### Analyzer._do_create_data_model
+This is a function that every `Analyzer` can override: this functions returns a boolean and, if `False`, the datamodel will not be created.
+This can be used if the `Analyzers` can succeed without retrieving useful results.


Suggested change

This can be used if the `Analyzers` can succeed without retrieving useful results.

This can be useful when a specific `Analyzer` succeeds without retrieving useful results.

mlodic · 2024-10-21T14:34:12Z

docs/IntelOwl/contribute.md

+```
+
+### Analyzer._create_data_model_mtm
+This is a function that every `Analyzer` can override: this functions returns a dictionary where the values are the objects that will be added in a many to many relationship in the datamodel, and the keys the names of the fields.


Suggested change

This is a function that every `Analyzer` can override: this functions returns a dictionary where the values are the objects that will be added in a many to many relationship in the datamodel, and the keys the names of the fields.

This is a function that every `Analyzer` can override: this function returns a dictionary where the values are the objects that will be added in a many to many relationship in the datamodel, and the keys the names of the fields.

this function returns a dictionary where the values are the objects that will be added in a many to many relationship in the datamodel, and the keys the names of the fields.

This is the technical explanation and is understandable. I would like to see another sentence with a more generic use-case explanation, less technical. In practice, you did it for all the cases but this one.

Example This mean that you can use it for more articulate data transformation to parse the `AnalyzerReport` into a `DataModel`. or This can be used if the `Analyzers` can succeed without retrieving useful results.

mlodic · 2024-10-21T14:38:00Z

docs/IntelOwl/contribute.md

+Here we are creating many `Signature` objects (using the signatures that matched the sample analyzed) and adding them to the `signatures` field.
+
+### Analyzer._update_data_model
+This is the last function that you can override in the `Analyzer` class: this functions returns nothing, and is called after every other check.


Suggested change

This is the last function that you can override in the `Analyzer` class: this functions returns nothing, and is called after every other check.

This is the last function that you can override in the `Analyzer` class: this function returns nothing, and is called after every other check.

mlodic · 2024-10-21T14:40:32Z

docs/IntelOwl/contribute.md

+      self.report: AnalyzerReport
+      if self.report.report["isWhitelisted"]:
+          evaluation = (
+              self.report.data_model_class.EVALUATIONS.FALSE_POSITIVE.value


Suggested change

self.report.data_model_class.EVALUATIONS.FALSE_POSITIVE.value

self.report.data_model_class.EVALUATIONS.TRUSTED.value

mlodic · 2024-10-21T14:40:45Z

docs/IntelOwl/contribute.md

+      data_model.evaluation = evaluation
+```
+We are setting the field `evaluation` depending on some logic that we constructed, using the data inside the report.
+If the IP has some report but is whitelisted then we set the `evaluation` to `false positive`, otherwise to `malicious`.


Suggested change

If the IP has some report but is whitelisted then we set the `evaluation` to `false positive`, otherwise to `malicious`.

If the IP address has been reported by some AbuseIPDB users but, at the same time, is whitelisted by AbuseIPDB, then we set its `evaluation` to `trusted`. On the contrary, if it's not whitelisted, we set it as `malicious`.

mlodic · 2024-10-21T14:51:07Z

docs/IntelOwl/contribute.md

+In version XXX of IntelOwl, a new plugin has been added: the `DataModel`.
+Its main functionality is to model an `Analyzer` result to a set of prearranged keys, allowing users to easily search, evaluate and use the analyzer result.
+The author of an `AnalyzerConfig` is able to decide mapping between each field of the `AnalyzerReport` and the corresponding in the `DataModel`.


The basic description of each Plugin should be put in the "Usage section" so I expect these 3 rows there while I would name this overall section "How to create/customize the Data Model" or something similar to align it with the rest of the existing documentation

Datamodel docs

0fa533a

0ssigeno requested a review from mlodic October 21, 2024 08:55

mlodic requested changes Oct 21, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Datamodel docs #7

Datamodel docs #7

0ssigeno commented Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

mlodic Oct 21, 2024

	meaning that we can provide use the following code to consider only _real_ results:
	meaning that we can use the following code to consider only _real_ results:

	This is a function that every `Analyzer` can override: this functions returns a boolean and, if `False`, the datamodel will not be created.
	This is a function that every `Analyzer` can override: this function returns a boolean and, if `False`, the datamodel will not be created.

	Let's use as an example `UrlHaus`: if the domain analyzed is not present in its database, the result will be
	Let's use `UrlHaus` as an example : if the domain analyzed is not present in its database, the result will be

	This can be used if the `Analyzers` can succeed without retrieving useful results.
	This can be useful when a specific `Analyzer` succeeds without retrieving useful results.

	This is the last function that you can override in the `Analyzer` class: this functions returns nothing, and is called after every other check.
	This is the last function that you can override in the `Analyzer` class: this function returns nothing, and is called after every other check.

	self.report.data_model_class.EVALUATIONS.FALSE_POSITIVE.value
	self.report.data_model_class.EVALUATIONS.TRUSTED.value

	If the IP has some report but is whitelisted then we set the `evaluation` to `false positive`, otherwise to `malicious`.
	If the IP address has been reported by some AbuseIPDB users but, at the same time, is whitelisted by AbuseIPDB, then we set its `evaluation` to `trusted`. On the contrary, if it's not whitelisted, we set it as `malicious`.

Datamodel docs #7

Are you sure you want to change the base?

Datamodel docs #7

Conversation

0ssigeno commented Oct 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment