CrowdStrike API Logs Not Parsing

[chadstout1939]

Version

2.4.110

Installation Method

Network installation on Ubuntu (unsupported)

Description

other (please provide detail below)

Installation Type

Distributed

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

16

RAM

128

Storage for /

500GB

Storage for /nsm

8TB

Network Traffic Collection

span port

Network Traffic Speeds

Less than 1Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Hello all!

I have connected my SO distributed deployment to CrowdStrike and I am ingesting CrowdStrike Alert and Host data via the API just fine. However, 'message' is not being parsed. After looking around at other posts, I found a few helpful commands:

• Running 'so-elasticsearch-pipeline-status logs-crowdstrike.host-1.38.0 | head' on the Manager and Search nodes shows that the pipeline has not been used on either. The same for 'logs-crowdstrike.alert-1.38.0'.
• Running 'so-logstash-pipeline-stats search' from the Search node doesn't show any issues, same for Manager node using 'manager'.
• I checked '/opt/so/saltstack/default/salt/elasticsearch/files/ingest' for the CrowdStrike ingest files, and I did not find any.
• The logs I do see, show valid data with no 'error.message' fields. The 'message' is just a large JSON blob of text.
• The integration says it should be using the respective ingest pipelines: logs-crowdstrike.alert-1.38.0 and logs-crowdstrike.host-1.38.0
• In Kibana -> Stack Management -> Index Management -> .ds-logs-crowdstrike.alert-default-2025.01.10-000001 -> Mappings, there is only the fields that I am finding in the SOC Dashboard.

It's like the ingested data is just skipping the ingest pipelines for CrowdStrike Alert and Host. I'm not sure where else to look and what else to try.

Thank you for any help or pointers.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[reyesj2]

can you share an example log from the crowdstrike integration? The whole log after its been ingested into Security Onion. You can left-click -> clipboard -> copy full event as JSON

If you go into Kibana -> stack management -> Index management you should be able to search for so-logs-crowdstrike.host and see

The path you mentioned /salt/elasticsearch/files/ingest is not used in this case, because the ingest pipelines are bundled with the Elastic integration itself not created by Security Onion

Another thing to check is that the index template (managed by Security Onion) was correctly applied / used.
I don't have crowdstrike data, but in my case I am showing zeek data, on the far right you can see index template: so-zeek.

For your crowdstrike data you should see so-logs-crowdstrike.host (the .host part can change depending on the subcomponent. looks like crowdstrike has host, alert, falcon, fdr).

[chadstout1939]

Thank you for your reply!!

Here is the sanitized log:

{
"@timestamp": "2025-01-13T16:04:37.133Z",
"agent.ephemeral_id": "UID",
"agent.id": "UID",
"agent.name": "FleetServer",
"agent.type": "filebeat",
"agent.version": "8.14.3",
"data_stream.dataset": "crowdstrike.host",
"data_stream.namespace": "default",
"data_stream.type": "logs",
"ecs.version": "8.0.0",
"elastic_agent.id": "UID",
"elastic_agent.snapshot": false,
"elastic_agent.version": "8.14.3",
"event.agent_id_status": "verified",
"event.dataset": "crowdstrike.host",
"event.ingested": "2025-01-13T16:04:47Z",
"input.type": "cel",
"message": "{"agent_load_flags":"1","agent_local_time":"2025-01-10T11:20:31.349Z","agent_version":"7.19.18913.0","bios_manufacturer":"Dell Inc.","bios_version":"1.1.1","build_number":"22631","chassis_type":"10","chassis_type_desc":"Laptop","cid":"a472733519164692ac2ad3d0f078f14e","config_id_base":"65994767","config_id_build":"18913","config_id_platform":"3","connection_ip":"192.168.1.66","connection_mac_address":"50-2f-9b-01-99-60","cpu_signature":"526017","cpu_vendor":"0","default_gateway_ip":"192.168.1.1","device_id":"UID","device_policies":{"content-update":{"applied":true,"applied_date":"2024-12-02T19:56:03.326467951Z","assigned_date":"2024-12-02T19:56:03.326467951Z","policy_id":"UID","policy_type":"content-update","settings_hash":"10534386584412607255"},"device_control":{"applied":true,"applied_date":"2025-01-07T16:57:50.512630287Z","assigned_date":"2025-01-07T16:54:19.137685909Z","policy_id":"UID","policy_type":"device-control","settings_hash":"85cf9e33"},"firewall":{"applied":true,"applied_date":"2024-06-20T13:13:52.997102397Z","assigned_date":"2024-06-20T13:11:50.731429438Z","policy_id":"UID","policy_type":"firewall","rule_set_id":"UID","settings_hash":"61a5496e"},"global_config":{"applied":true,"applied_date":"2025-01-12T17:46:53.884976028Z","assigned_date":"2025-01-12T17:40:09.281073487Z","policy_id":"UID","policy_type":"globalconfig","settings_hash":"addfc919"},"host-retention":{"applied":true,"applied_date":"2024-08-14T11:57:51.25048488Z","assigned_date":"2024-08-14T11:55:22.350794824Z","policy_id":"UID","policy_type":"host-retention","settings_hash":"fce339fc6cbb4dae36929fe363a81368f4a7e3f2c3a3d62ff7e3ef202ee48df5"},"prevention":{"applied":true,"applied_date":"2024-12-05T00:16:29.63392876Z","assigned_date":"2024-12-05T00:09:53.829341172Z","policy_id":"UID","policy_type":"prevention","rule_groups":["UID"],"settings_hash":"754b047b"},"remote_response":{"applied":true,"applied_date":"2024-06-17T01:11:19.559030112Z","assigned_date":"2024-06-17T01:10:49.489635409Z","policy_id":"UID","policy_type":"remote-response","settings_hash":"25ab5e6e"},"sensor_update":{"applied":true,"applied_date":"2025-01-07T13:47:13.14943548Z","assigned_date":"2025-01-07T13:45:16.227257148Z","policy_id":"UID","policy_type":"sensor-update","settings_hash":"tagged|1;101","uninstall_protection":"ENABLED"},"system-tray":{"applied":true,"applied_date":"2024-06-17T01:17:45.402885047Z","assigned_date":"2024-06-17T01:17:16.820812553Z","policy_id":"UID","policy_type":"system-tray","settings_hash":"64111ecabb0f2fff301bef6007dd2792f26df4fe37d52475b301883d5d05a856"}},"external_ip":"256.256.256.256","filesystem_containment_status":"normal","first_seen":"2020-10-29T16:50:24Z","group_hash":"08e57070457a711638a6517597d17c1c82e4f4f826ea246a63130204835a24c0","groups":["UID","UID"],"hostname":"HOSTNAME","kernel_version":"10.0.22631.4601","last_login_timestamp":"2024-12-12T20:31:34Z","last_login_user":"user.name","last_login_user_sid":"S-1-5-21-UID","last_reboot":"2025-01-10T18:19:55Z","last_seen":"2025-01-13T15:59:02Z","local_ip":"192.168.1.66","mac_address":"50-2f-9b-01-99-60","machine_domain":"comoso.com","major_version":"10","meta":{"version":"17376","version_string":"1:4615273930"},"minor_version":"0","modified_timestamp":"2025-01-13T15:59:27Z","os_build":"22631","os_product_name":"Windows 11 Pro","os_version":"Windows 11","ou":["OU","OU","OU"],"platform_id":"0","platform_name":"Windows","pointer_size":"8","policies":[{"applied":true,"applied_date":"2024-12-05T00:16:29.63392876Z","assigned_date":"2024-12-05T00:09:53.829341172Z","policy_id":"UID","policy_type":"prevention","rule_groups":["f66eeca33387486ebed2134a75559a40"],"settings_hash":"754b047b"}],"product_type":"1","product_type_desc":"Workstation","provision_status":"Provisioned","reduced_functionality_mode":"no","rtr_state":"enabled","serial_number":"SERIAL","service_pack_minor":"0","site_name":"SITE-NAME","status":"normal","system_manufacturer":"Dell Inc.","system_product_name":"Computer Model","tags":[]}",
"tags": [
"forwarded",
"crowdstrike-host"
],
"soc_id": "flFpYJQBkNFFTi-kFXCp",
"soc_score": 2.0000336,
"soc_type": "",
"soc_timestamp": "2025-01-13T16:04:37.133Z",
"soc_source": "sorr-siem:.ds-logs-crowdstrike.host-default-2025.01.10-000001"
}

I checked the Index Templates and did not find a "so-logs-crowdstrike.host" or "so-logs-crowdstrike.alert". Just "logs-crowdstrike.host" and "logs-crowdstrike.alert". I'm not sure if this picture will go through properly or not.

The index template applied to the data streams for "logs-crowdstrike.alert-default" and "logs-crowdstrike.host-default" are "so-logs".

[reyesj2]

Ahh, looks like I was looking at my development instance of Security Onion. Yes, the host index template is missing. It is going to be fixed in the .120 release. My recommendation would be to wait until .120 so you can soup and then verify your logs are correctly parsed. If not please re-comment here and we can look closer. Sorry for the additional confusion!

[chadstout1939]

No worries! I will wait for the .120 release and report back! Thank you!!