Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for Sigma Correlation Event count #1357

Merged
merged 5 commits into from
Jun 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG-Japanese.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- [Mimikatz Use](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/builtin/win_alert_mimikatz_keywords.yml)
- デフォルトでは、適用可能なルールを持つ`.evtx`ファイルのみ読み込む。たとえば、さまざまなイベントログのディレクトリをスキャンしている場合でも、 `Channel: Security` を探すルールのみを有効にした場合、Hayabusaは`Security`以外のすべてのイベントログを無視します。ベンチマークでは、通常のスキャンで約10%、単一のルールでスキャンする場合は最大60%以上のパフォーマンス向上が得られる。チャネルに関係なくすべての`.evtx`ファイルを読み込みたい場合は、`csv-timeline` と `json-timeline` の `-a、--scan-all-evtx-files` オプションでこのフィルタリングをオフにすることができる。(#1318) (@fukusuket)
- 注意: チャンネルフィルタリングは .evtx ファイルにのみ適用され、`-J, --json-input`オプションを使用してイベントログをJSONファイルから読み込む際に`-A`または`-a`を指定するとエラーが発生する。(#1345) (@fukusuket)
- Sigma CorrelationのEvent Countに対応した。 (#1337) (@fukusuket)

**改善:**

Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
- [Mimikatz Use](https://github.com/SigmaHQ/sigma/blob/master/rules/windows/builtin/win_alert_mimikatz_keywords.yml)
- By default now, `.evtx` files that have applicable rules will be loaded. So for example, if you are scanning a directory of various event logs but only enable a rule that is looking for `Channel: Security` then Hayabusa will ignore all non-security event logs. In our benchmarks, this gives a speed benefit of around 10% with normal scans and up to 60%+ performance increase when scanning with a single rule. If you want to load all `.evtx` files regardless of channel, then you can turn off this filtering with the `-a, --scan-all-evtx-files` option in `csv-timeline` and `json-timeline`. (#1318) (@fukusuket)
- Note: Channel filtering only works with .evtx files and you will receive an error if you try to load event logs from a JSON file with `-J, --json-input` and also specify `-A` or `-a`. (#1345) (@fukusuket)
- Support for Sigma Correlation's Event Count. (#1337) (@fukusuket)

**Enhancements:**

Expand Down
56 changes: 30 additions & 26 deletions src/detections/detection.rs
Original file line number Diff line number Diff line change
@@ -1,38 +1,39 @@
extern crate csv;

use crate::detections::configs::Action;
use crate::detections::utils::{create_recordinfos, format_time, write_color_buffer};
use crate::options::profile::Profile::{
self, Channel, Computer, EventID, EvtxFile, Level, MitreTactics, MitreTags, OtherTags,
Provider, RecordID, RecoveredRecord, RenderedMessage, RuleAuthor, RuleCreationDate, RuleFile,
RuleID, RuleModifiedDate, RuleTitle, SrcASN, SrcCity, SrcCountry, Status, TgtASN, TgtCity,
TgtCountry, Timestamp,
};
use std::default::Default;
use std::fmt::Write;
use std::path::Path;
use std::sync::Arc;

use chrono::{TimeZone, Utc};
use compact_str::CompactString;
use hashbrown::HashMap;
use itertools::Itertools;
use nested::Nested;
use num_format::{Locale, ToFormattedString};
use std::default::Default;
use serde_json::Value;
use termcolor::{BufferWriter, Color, ColorChoice};
use tokio::{runtime::Runtime, spawn, task::JoinHandle};
use yaml_rust::Yaml;

use crate::detections::configs::Action;
use crate::detections::configs::STORED_EKEY_ALIAS;
use crate::detections::field_data_map::FieldDataMapKey;
use crate::detections::message::{AlertMessage, DetectInfo, ERROR_LOG_STACK, TAGS_CONFIG};
use crate::detections::rule::correlation_parser::parse_correlation_rules;
use crate::detections::rule::{self, AggResult, RuleNode};
use crate::detections::utils::{create_recordinfos, format_time, write_color_buffer};
use crate::detections::utils::{get_serde_number_to_string, make_ascii_titlecase};
use crate::filter;
use crate::options::htmlreport;
use crate::options::pivot::insert_pivot_keyword;
use crate::options::profile::Profile::{
self, Channel, Computer, EventID, EvtxFile, Level, MitreTactics, MitreTags, OtherTags,
Provider, RecordID, RecoveredRecord, RenderedMessage, RuleAuthor, RuleCreationDate, RuleFile,
RuleID, RuleModifiedDate, RuleTitle, SrcASN, SrcCity, SrcCountry, Status, TgtASN, TgtCity,
TgtCountry, Timestamp,
};
use crate::yaml::ParseYaml;
use hashbrown::HashMap;
use serde_json::Value;
use std::fmt::Write;
use std::path::Path;

use crate::detections::configs::STORED_EKEY_ALIAS;
use crate::detections::field_data_map::FieldDataMapKey;
use std::sync::Arc;
use tokio::{runtime::Runtime, spawn, task::JoinHandle};

use super::configs::{
EventKeyAliasConfig, StoredStatic, GEOIP_DB_PARSER, GEOIP_DB_YAML, GEOIP_FILTER, STORED_STATIC,
Expand Down Expand Up @@ -134,12 +135,13 @@ impl Detection {
None
};
// parse rule files
let ret = rulefile_loader
let mut ret = rulefile_loader
.files
.into_iter()
.map(|rule_file_tuple| rule::create_rule(rule_file_tuple.0, rule_file_tuple.1))
.filter_map(return_if_success)
.collect();
ret = parse_correlation_rules(ret, stored_static, &mut parseerror_count);
if !(stored_static.logon_summary_flag
|| stored_static.search_flag
|| stored_static.metrics_flag
Expand Down Expand Up @@ -1190,6 +1192,15 @@ impl Detection {

#[cfg(test)]
mod tests {
use std::path::Path;

use chrono::TimeZone;
use chrono::Utc;
use compact_str::CompactString;
use serde_json::Value;
use yaml_rust::Yaml;
use yaml_rust::YamlLoader;

use crate::detections;
use crate::detections::configs::load_eventkey_alias;
use crate::detections::configs::Action;
Expand All @@ -1209,13 +1220,6 @@ mod tests {
use crate::detections::utils;
use crate::filter;
use crate::options::profile::Profile;
use chrono::TimeZone;
use chrono::Utc;
use compact_str::CompactString;
use serde_json::Value;
use std::path::Path;
use yaml_rust::Yaml;
use yaml_rust::YamlLoader;

fn create_dummy_stored_static() -> StoredStatic {
StoredStatic::create_static_data(Some(Config {
Expand Down
Loading
Loading