eventExport - json.loads(chunk) fails when payload (chunk) contains character ' in values

[trizzosk]

Hi,
I wanted to utilize the stream API to feed all data to internal SIEM system, however I do have issues.

Script ses-api-samples/icdm/python/eventExport.py is failing when chunk contains illegal characters (e.g. '...) json values. Seems like incorrect escaping of chars...

Line 48:

          stream_response = json.loads(chunk)

If you pass following json (Note: the json string is chunk printed to console and has been stripped here becasue the rest of the data is OK for parsing):

{
    "actor":
    {
        "session_id":0,
        "app_name":"PowerShell_C:\\\\WINDOWS\\\\system32\\\\WindowsPowerShell\\\\v1.0\\\\PowerShell.exe_10.0.18362.1",
        "start_time":"2023-01-11T09:45:28.220Z",
        "cmd_line":"\\"C:\\\\WINDOWS\\\\system32\\\\WindowsPowerShell\\\\v1.0\\\\PowerShell.exe\\" -NoLogo -Noninteractive -NoProfile  -ExecutionPolicy Bypass \\"& \'C:\\\\WINDOWS\\\\CCM\\\\SystemTemp\\\\77647c33-683e-4733-b958-d8b018d5d73e.ps1\'\\""
    },
    "data":"\\nfunction ExceptionHandler\\n{\\n    $errorMessage = $_.Exception.Message\\n\\t$logName = $args[0]\\n\\t$sourceName = $args[1]\\n\\t$categoryID = $args[2]\\n    $returnDescription = \\"Compliance Result = Non-Compliant\\"\\n    echo \\"Exception occurred in the PowerShell script with error message - $errorMessage\\" | Out-File $logFile -Append\\n    echo \\"Compliance Result = Non-Compliant\\" | Out-File $logFile -Append\\n    Write-Output \\"Non-Compliant\\"\\n    \\n    WriteErrorEventLog $propertyName 100 $errorMessage $returnDescription $logName $sourceName $categoryID\\n    [System.Environment]::Exit(1)\\n}\\n\\nfunction WriteErrorEventLog\\n{\\n    $propertyName = $args[0]\\n    $returnCode = $args[1]\\n    $message = $args[2]\\n    $returnDescription = $args[3]\\n\\t$logName = $args[4]\\n\\t$sourceName = $args[5]\\n\\t$categoryID = $args[6]\\n    \\n    $entryType = \\"Error\\"\\n\\n    WriteEventLog $entryType $propertyName $returnCode $message $returnDescription $logName $sourceName $categoryID\\n}\\n\\nfunction WriteInfoEventLog\\n{\\n    $propertyName = $args[0]\\n    $returnCode = $args[1]\\n    $message = $args[2]\\n    $returnDescription = $args[3]\\n\\t$logName = $args[4]\\n\\t$sourceName = $args[5]\\n\\t$categoryID = $args[6]\\n    $entryType = \\"Information\\"\\n\\n    WriteEventLog $entryType $propertyName $returnCode $message $returnDescription $logName $sourceName $categoryID\\n}\\n\\nfunction WriteWarningEventLog\\n{\\n    $propertyName = $args[0]\\n    $returnCode = $args[1]\\n    $message = $args[2]\\n    $returnDescription = $args[3]\\n\\t$logName = $args[4]\\n\\t$sourceName = $args[5]\\n\\t$categoryID = $args[6]\\n    $entryType = \\"Warning\\"\\n\\n    WriteEventLog $entryType $propertyName $returnCode $message $returnDescription $logName $sourceName $categoryID\\n}\\n\\nfunction WriteEventLog\\n{\\n    $entryType = $args[0]\\n    $propertyName = $args[1]\\n    $returnCode = $args[2]\\n    $message = $args[3]\\n    $returnDescription = $args[4]\\t\\n    $logName = $args[5]\\n    $sourceName = $args[6]\\n    $categoryID = $args[7] # 1 => Discovery, 2 => Remediation\\n\\n    if ($categoryID -eq 1) {\\n        $categoryName = \\"Discovery\\"\\n    } else {\\n        $categoryName = \\"Remediation\\"\\n    }\\n\\n     # empty value is being passed from scripts. Hence adding check\\n    if(($returnCode -eq \\"\\") -Or ($returnCode -eq $null)){\\n\\t   $returnCode = 100\\n\\t}\\n\\n    $completeDescription = \\"Property Name=$propertyName, Execution Phase=$categoryName, Description=$message, $returnDescription\\"\\n\\n    if ([System.Diagnostics.EventLog]::Exists($logName) -eq $false) {\\n        echo \\"$logName Exists is false. Creating new Event Log\\" | Out-File $logFile -Append\\n        New-EventLog -LogName $logName -Source $sourceName\\n    } else {\\n        if ([System.Diagnostics.EventLog]::SourceExists($sourceName) -eq $false) {\\n            echo \\"$sourceName Exists is false. Creating new Event Log\\" | Out-File $logFile -Append\\n            New-EventLog -LogName $logName -Source $sourceName\\n        }\\n    }\\n    $enc = [system.Text.Encoding]::UTF8\\n\\t$data = $enc.GetBytes($propertyName) \\t\\n    Write-EventLog -LogName $logName -Source $sourceName -EntryType $entryType -EventId $returnCode -Message $completeDescription -Category $categoryID -RawData $data\\n}\\n# BCU Discovery Template\\nfunction CreateRegistryKeyForNonEnforcement\\n{\\n    $policyGUID = $args[0]\\n    $policyVersion = $args[1]\\n    \\n    $path = \\"HKLM:\\\\SOFTWARE\\\\HP\\\\MIK\\\\Compliance\\\\\\" + $policyGUID + \\"\\\\\\" + $policyVersion\\n    If ((Test-Path $path) -eq $False) {\\n        New-Item -Path $path -Force | Out-Null\\n    }\\n}\\n\\nfunction CheckRegistryPropertyKeyForNonEnforcement\\n{\\n    $policyGUID = $args[0]\\n    $policyVersion = $args[1]\\n    $propertyName = $args[2]\\n    \\n    $path = \\"HKLM:\\\\SOFTWARE\\\\HP\\\\MIK\\\\Compliance\\\\\\" + $policyGUID + \\"\\\\\\" + $policyVersion\\n    \\n    CreateRegistryKeyForNonEnforcement $policyGUID $policyVersion\\n\\n    $value = (Get-ItemProperty -Path $path).$propertyName\\n    if ($value -eq $null) {\\n        Write-Output $False\\n    } else {\\n        Write-Output $True\\n    }\\n}\\n\\nfunction GetProviderPropertyObject\\n{\\n    $providerNamespace = $args[0]\\n    $providerClassName = $args[1]\\n    $xmlPayload = $args[2]\\n\\n    invoke-wmimethod -path $providerClassName -namespace $providerNamespace -name get -argumentlist $xmlPayload, [ref]$out\\n}\\n\\ntry {\\n    # General arguments \\n    $logFolder = $Env:ProgramData + \'\\\\HP\\\\HP MIK\\\\Logs\' + \'\\\\BIOS Password - BIOS Configuration\'\\n    $configurationItemName = \\"BIOS Password - BIOS Configuration\\"\\n    \\n    # WMI Provider related arguments\\n    $providerNamespace = \\"root\\\\HP\\\\InstrumentedServices\\\\v1\\"\\n    $providerClassName = \\"HP_BIOSConfig\\"\\n    \\n    # WMI Property related arguments\\n    $propertyName = \\"_NumLockAtBoot\\"\\n    $expectedPropertyValue = \\"Enable\\"\\n    $xmlPayload = \\"<?xml version=\'1.0\' encoding=\'utf-8\'?>\\n<si:get xmlns:si=\'http://frameworks.hp.com/siam\' xmlns:xsi=\'http://www.w3.org/2001/XMLSchema-instance\' xmlns:schemaLocation=\'http://frameworks.hp.com/siam.xsd\'>\\n  <si:header />\\n  <si:security>\\n    <si:blob />\\n  </si:security>\\n  <si:values>\\n    <si:get_value name=\'_NumLockAtBoot\' />\\n  </si:values>\\n</si:get>\\"\\n    \\n    # Non-Enforcement related arguments\\n    $propertyEnforcement = \\"True\\"\\n    $propertyAccessMode = \\"RD_WR\\"\\n    $policyGUID = \\"5103e037-f537-4570-9e01-ad9045c08399\\"\\n    $",
    "attacks":
    [
        {
            "technique_uid":"T1059",
            "sub_technique_uid":"T1059.001",
            "technique_name":"Command and Scripting Interpreter",
            "sub_technique_name":"PowerShell",
            "tactic_ids":[2],
            "tactic_uids":["TA0002"]
        },
        {
            "technique_uid":"T1064",
            "technique_name":"Scripting",
            "tactic_ids":[5,2],
            "tactic_uids":["TA0005","TA0002"]
        }
    ],
    "analysis":"{}",
    "time":"2023-01-11T09:45:28.531Z",
    "log_time":"2023-01-11T09:46:02.717Z",
}

The issue here is the data key which is not correctly formatted - but the problem is that this is 1:1 from the stream API.... what shall I do?