diff --git a/docs/xdr/usecases/use_your_own_cti.md b/docs/xdr/usecases/use_your_own_cti.md new file mode 100644 index 0000000000..1ee81e6ead --- /dev/null +++ b/docs/xdr/usecases/use_your_own_cti.md @@ -0,0 +1,216 @@ +# Use your own CTI + +[IOC Collections](../features/detect/ioccollections.md) can be used to use your own CTI inside Sekoia.io's XDR. + +This section will show how to programmatically add your CTI inside the platform and how to take profit of it. + +## Prerequisites + +- A Sekoia.io XDR license +- An API Key with that contains at least the following permissions: + * INTHREAT_READ_IOC_COLLECTIONS + * INTHREAT_WRITE_IOC_COLLECTIONS + +## Detect threats using your CTI + +By creating custom CTI rules for your IOC collections it is possible to detect threats based on your own indicators. To setup this kind of rules follow [the dedicated section in the IOC Collections documentation](../features/detect/ioccollections.md#detect-threats-using-ioc-collections). + +## Push CTI + +### Python requirements + +The code samples provided below need the following python library: + +- requests +- websockets + +In order to install them the command `pip install requests websockets` can be used. + +### Create an IOC collection + +The first step is to create an IOC collection. It can be done manually following [the dedicated documentation](../features/detect/ioccollections.md#create-an-ioc-collection) or automatically relying on the REST endpoint: + +```py +import requests + +api_key = "{Your API Key}" # (1)! +headers = { + "authorization": f"Bearer {api_key}" +} +payload = {"name": "My IOC Collection"} +res = requests.post( + "https://api.sekoia.io/v2/inthreat/ioc-collections", + headers=headers, + json=payload, +) +res.raise_for_status() +print(f"Collection created with id {res.json()['id']}") +``` + +1. Replace the content of this string with your API Key + +!!! tip + When creating an IOC Collection it is also possible to specify: + + - A description using the `description` field + - For Multi-tenant users it is possible to specify: + - For which sub-community the collection must be created for by providing the `community_uuid` attribute + - If the collection must be made available to sub-communities by setting `available_for_sub_communities` to `true`. + +### Import indicators from text + +The first way to import indicators is to do it providing a raw text format where each line contains one indicator. +The first part of the snippet is about creating the import task, while the second part waits until the import finishes. + +```py +import json + +import requests +from websockets.sync.client import connect + + +api_key = "{Your API Key}" # (1)! +collection_id = "ioc-collection--xxx" # (2)! + +# First we create the import task +headers = { + "authorization": f"Bearer {api_key}" +} +observables_text = """ +microsoftsvc.com +178.236.246.57 +d30bb9df615a8d1661f843d426ff40eb,4344e695b5f65917dc68f241ecde4b99cf25d930 +""" # (3)! +payload = { + "indicators": observables_text, + "x_ic_related_threat_refs": ["intrusion-set--202fc7e0-1580-4ad1-83d8-5f98be4fe1a2"], # (4)! +} # (5)! +res = requests.post( + f"https://api.sekoia.io/v2/inthreat/ioc-collections/{collection_id}/indicators/text", + headers=headers, + json=payload +) +res.raise_for_status() +task_id = res.json()["task_id"] + +# Then we wait for the import to finish +headers = {"Cookie": f"access_token_cookie={api_key}"} +with connect("wss://app.sekoia.io/live/", additional_headers=headers) as websocket: # (6)! + while True: + raw_message = websocket.recv() + message = json.loads(raw_message) + if message.get("type") == "ioc-collection" and message["attributes"].get("task_id") == task_id: # (7)! + imported = message["attributes"]["imported"] + total = message["attributes"]["total"] + print(f"Imported {imported}/{total}") + if imported == total: + break +print("Import finished") +``` + +1. Replace the content of this string with your API Key +2. Replace the content of this string with the id of the collection the IOC must be added to +3. One IOC per line. Hashes belonging to the same IOC can be added on the same line separated by a comma +4. It is possible to link the created indicators to several threats by providing their ids inside the `x_ic_related_threat_refs` field. +5. When importing indicators various attributes can be provided like the validity dates, a description, the kill chain phases, ... +To get information about them check [the documentation about this endpoint](https://docs.sekoia.io/cti/develop/rest_api/intelligence/#tag/IOC-Collections/operation/post_collection_indicators_text_resource) +6. Connect to `LiveAPI` using a websocket to be notified in real time of events happening in the community +7. Keep only messages that are about our task + + +### Import indicators from file + +In this sample we will import IOC from a file. The formats currently supported along with their mime types are: + +- CSV: `text/csv`, `application/csv` or `text/plain` +- XLS: `application/vnd.ms-excel` or `application/octet-stream` +- XLSX: `application/zip` or `application/vnd.openxmlformats-officedocument.spreadsheetml.sheet` + +The first step consists of uploading the file to obtain a preview. In the response of this requests we will obtain: + +- A file handle that must be used to process the file +- A preview of the first lines of the file +- The detected mapping: Which column contains the indicator, the validity dates, the kill chain, the threat, ... +- Whether the first line has been detected to be ignored or not + +Once the preview has been generated, if everything seems right the file can be processed. +The last step is to wait for the file to be completely processed. Depending on the number of indicators it can take a bit of time. + +```py +import json + +import requests +from websockets.sync.client import connect + + +api_key = "{Your API Key}" # (1)! +collection_id = "ioc-collection--xxx" # (2)! + +# First we generate a preview of the file +headers = { + "authorization": f"Bearer {api_key}" +} +files = {"file": ("file", open("path/to/file.csv", "rb"), "text/csv")} # (3)! +res = requests.post( + f"https://api.sekoia.io/v2/inthreat/ioc-collections/{collection_id}/indicators/files/preview", + headers=headers, + files=files, +) +res.raise_for_status() +content = response.json() + +# Then we ask the API to process the file +# The API guessed the mapping and whether the first line must be ignored or not, +# but in this case we assume we know it and provide it manually. +# Update this mapping depending on your files (one type per column). +payload = { + "file_handle": content["file_handle"], # (4)! + "mapping": [ + "observable", # (5)! + "valid_from", + "valid_until", + null, # (6)! + "kill_chain_phases", # (7)! + "x_ic_related_threat_refs", # (8)! + ], + "ignore_first_line": True, # (9)! +} # (10)! + +res = requests.post( + f"https://api.sekoia.io/v2/inthreat/ioc-collections/{collection_id}/indicators/files/process", + headers=headers, + json=payload, +) +res.raise_for_status() +task_id = res.json()["task_id"] + +# Then we wait for the import to finish +headers = {"Cookie": f"access_token_cookie={api_key}"} +with connect("wss://app.sekoia.io/live/", additional_headers=headers) as websocket: # (11)! + while True: + raw_message = websocket.recv() + message = json.loads(raw_message) + if message.get("type") == "ioc-collection" and message["attributes"].get("task_id") == task_id: # (12)! + imported = message["attributes"]["imported"] + total = message["attributes"]["total"] + print(f"Imported {imported}/{total}") + if imported == total: + break +print("Import finished") +``` + +1. Replace the content of this string with your API Key +2. Replace the content of this string with the id of the collection the IOC must be added to +3. Update this line with the path to the file to import and the appropriate mime type +4. The file handle must be used to process the file in the next call +5. The column containing the indicator value is named `observable` +6. Columns not linked to any attribute must be set to `null` +7. The kill chain phase names must match a tactic from MITRE ATT&CK or a phase from the cyber kill chain. +Several values can be specified separated by commas. +8. The name of the threats must be known inside the intelligence center. +Several values can be specified separated by commas. i.e. `Phishing,APT-29` +9. If the first line contains the columns names we can ignore it when processing the file +10. When importing indicators various attributes can be provided like the validity dates, a description, the kill chain phases, ... +To get information about them check [the documentation about this endpoint](https://docs.sekoia.io/cti/develop/rest_api/intelligence/#tag/IOC-Collections/operation/post_collection_indicators_text_resource) +11. Connect to `LiveAPI` using a websocket to be notified in real time of events happening in the community +12. Keep only messages that are about our task diff --git a/mkdocs.yml b/mkdocs.yml index 3892851034..9e416b49cb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -105,6 +105,7 @@ nav: - Implement a blocklist in Sekoia.io: xdr/usecases/playbook/implement_blocklist.md - Synchronize Alerts with an external tool: xdr/usecases/playbook/synchronize_alerts.md - Send notifications to a Webhook using a playbook: xdr/usecases/playbook/notifications_using_playbooks.md + - Use your own CTI in Sekoia.io: xdr/usecases/use_your_own_cti.md - FAQ: - General: xdr/FAQ.md - Alerts: xdr/FAQ/Alerts_qa.md @@ -186,7 +187,7 @@ nav: - Expiration Rules: tip/features/produce/expiration_rules.md - Monitor: - Dashboards: tip/features/monitor/dashboard.md - - Threat Landscape: cti/features/monitor/threat_landscape.md + - Threat Landscape: tip/features/monitor/threat_landscape.md - External Integrations: - Overview: tip/features/integrations/index.md - API: tip/features/integrations/api.md @@ -199,7 +200,7 @@ nav: - PaloAlto Cortex XSOAR: tip/features/integrations/paloalto_xsoar.md - Automate: - Playbooks: tip/features/automate/index.md - - Manage accounts: xdr/features/automate/manage-accounts.md + - Manage accounts: tip/features/automate/manage-accounts.md - Navigate playbooks: tip/features/automate/navigate-playbooks.md - Build playbooks: tip/features/automate/build-playbooks.md - Triggers: tip/features/automate/triggers.md