v1.2.0

v.1.2.0 of BinaryAlert is here! This version simplifies and hardens the BinaryAlert architecture, adds support for scanning UPX-packed binaries and PDFs, and adds more configuration options to customize your deployment.

Change Summary

• Adds support for scanning UPX packing and PDFs
  • ./manage.py live_test now includes examples of both
• YARA rule source configuration is now much more expressive
• YARA match results reported to SNS now include the substring in the data that triggered the match
• Replace dispatcher Lambda function with SQS => Lambda event source mappings
• Improved retroactive analysis:
  • Replace batcher Lambda function with S3 inventory
  • Replace the ./manage.py analyze_all command with retro_fast (to read the latest inventory) and retro_slow (to enumerate the bucket directly)
  • An ongoing retroactive analysis can be stopped with ./manage.py purge_queue
• Dynamo, S3, and SQS resources are all server-side encrypted
• Improved support and documentation for using BinaryAlert to scan your existing S3 buckets.
• New advanced configuration options (terraform/terraform.tfvars):
  • You can add external S3 and KMS resources to the config file to grant permissions
  • The "BinaryAlert" cost tag added to all applicable AWS resources is now configurable
  • You can configure a second SNS topic to alert when a file does not match any YARA rules (thanks @goochi1!)
  • You can configure the retention for each SQS queue and set concurrency limits for the Lambda functions that poll them
• Upgraded requirements and dependencies
  • cbapi v1.3.4 => v1.3.6
  • terraform/aws-provider v1.5 => v1.30
  • yara-python v3.7.0 => 3.8.0

For the complete list of changes and issues closed, see the associated milestone.

Upgrading From v1.1.0

• The rule sources and Terraform variable files have changed their formats somewhat. We recommend cloning the new version of BinaryAlert and then manually copying over your existing config.
• 2 of the Lambda functions will be destroyed (dispatcher and batcher) - that's by design!
• WARNING: In order to encrypt the Dynamo table, it must be destroyed and re-created.
  • If you want the history of YARA matches from the table, export them before upgrading.
  • Alternatively, remove the server_side_encryption directive in terraform/dynamo.tf to keep the table the way it is.
  • Once the table is recreated, BinaryAlert will have no history of YARA matches, meaning it will think every YARA match is brand new. Your first retro scan will alert on every matching file.

v1.1.0

v1.1.0 adds support for archive analysis via yextend! Hundreds of different types of archives (.docx, .rar, .zip, etc) are now natively extracted and scanned with your YARA rules.

Change Summary

• Bundles yextend for YARA analysis of archives
• The YARA analyzer can now be configured to analyze any bucket
• The shred utility is used to destroy files in /tmp after downloading them from S3
• Adds YARA rules for CobaltStrike and the backdoored version of CCleaner
• Upgraded Dependencies
  • cbapi v1.3.2 => v1.3.4
  • terraform v0.10.X => v0.11.X
  • terraform/aws-provider 0.1.4 => 1.5.X
  • yara-python v3.6.3 => v3.7.0
• CLI Changes
  • Adds --version flag
  • Adds destroy command
  • Creates a new Lambda version for every function on every deploy
  • live_test now uploads an archive in addition a text file
• Documentation Updates
  • Provides a least-privilege IAM policy for deploying BinaryAlert
  • Explains how to install openssl development libraries prior to installing YARA
  • Adds a credits page

For the complete list of changes and issues closed, see the associated milestone.

Upgrading From v1.0.0

Upgrading is quite easy and can happen on top of your existing deploy:

1. git checkout v1.1.0
2. Install the new (upgraded) requirements in your virtualenv:

source venv/bin/activate
pip install -r requirements.txt

3. Upgrade terraform to v0.11+
4. Redeploy: ./manage.py deploy
5. Wait a few minutes for the changes to propagate, and then ./manage.py live_test

Note that the SNS alert no longer includes the YARA RuleTags in the MatchedRules section.

v1.0.0

BinaryAlert's first official release is here! To get started, visit https://binaryalert.io/getting-started.html

Change Summary

• Adds official documentation at binaryalert.io
• Adds support for the hash YARA module
• Adds CarbonBlack integration
• Adds cb_copy_all (CarbonBlack copy), configure, and compile_rules CLI commands
• Upgraded to Terraform v0.10
  • Changes now require user approval before a deploy continues
• Adds a BinaryAlert CloudWatch dashboard
• Adds and enforces Python type annotations
• Test coverage increased to 96%
• General code cleanup
• [Bug Fix] S3 filenames are now correctly url-decoded (e.g. supports filenames with spaces)

For a complete list of changes, see the associated milestone.

Upgrading

If you are upgrading from BinaryAlert v0.10:

• You will need to upgrade Terraform to v0.10.4+
• The LambdaVersion table column has been renamed to AnalyzerVersion.
  • This will force a new resource; Terraform will destroy the old DynamoDB table.
  • If you want to keep the old table, remove the state from Terraform before applying changes: terraform state rm aws_dynamodb_table.binaryalert_yara_matches

0.10

Change Summary

Resolves: #3, #7, #23, #24

• Adds ~50 new YARA rules and reorganizes the rules folder structure
• Adds new live_test feature to CLI
• SNS alert format changed to be more friendly for StreamAlert/PagerDuty integration
• Test coverage is improved, measured, and enforced
• [Bug Fix] Batcher no longer errors when scanning an empty bucket

New Feature: Live Test

You can now quickly test the end-to-end functionality of a live BinaryAlert deployment with a single command: python3 manage.py live_test

This will upload a harmless test file which should trigger a YARA match alert. The live test will verify that the match was correctly identified and saved in DynamoDB.

New YARA Match Alert Format

When a file matches a YARA rule, the SNS output format has changed from

{
    'FileInfo': { ... },
    'MatchedRules': [
        {
            'RuleFile': 'rules.yara',
            'RuleName': 'my_rule_name,
            ...
        }
    ]
}

to

{
    'FileInfo': { ... },
    'NumMatchedRules': 2,
    'MatchedRules': {
        'Rule1': {
            'RuleFile': 'rules.yara',
            'RuleName': 'my_rule_name,
            ...
        },
        'Rule2': { ... }
    ]
}

The new format is easy to integrate with StreamAlert and renders better in PagerDuty

Initial Open-Source Release

0.9

Initial commit