Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt to Zeek "files" log losing conn_uids+tx_hosts+rx_hosts and gaining uid+id #2981

Merged
merged 1 commit into from
Jan 23, 2024
Merged

Adapt to Zeek "files" log losing conn_uids+tx_hosts+rx_hosts and gaining uid+id #2981

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions apps/zui/src/plugins/brimcap/zeek/correlations.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ export function activateZeekCorrelations() {
return zedScript`
from ${session.poolName}
| md5==${getMd5()}
| count() by tx_hosts
| count() by tx_host:=id.resp_h
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do the := assignment here because if I leave it as just id.resp_h then only id appears as the column header in the table and no values under it, I guess since id is a record type and I assume the table is only prepared to render primitive values, and primitive values are all we want to show here anyway. So I'm kinda conjuring up tx_host and rx_host as fake field names just for presentation purposes, but that feels defensible.

| sort -r
| head 5`
},
Expand All @@ -41,7 +41,7 @@ export function activateZeekCorrelations() {
return zedScript`
from ${session.poolName}
| md5==${getMd5()}
| count() by rx_hosts
| count() by rx_host:=id.orig_h
| sort -r
| head 5`
},
Expand Down
3 changes: 2 additions & 1 deletion apps/zui/src/plugins/brimcap/zeek/queries.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ export function uidQuery(pool: string, uid: string) {
}

export function uidFilter(uid: string) {
return zedScript`uid==${uid} or ${uid} in conn_uids or ${uid} in uids or referenced_file.uid==${uid}`
return zedScript`uid==${uid} or ${uid} in uids or referenced_file.uid==${uid}`
}

export function communityConnFilter(data: CommunityConnArgs) {
Expand All @@ -25,6 +25,7 @@ export function findConnLog(pool: string, uid: string) {
| (` +
uidFilter(uid) +
`)
| _path=="conn"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jameskerr: I'll definitely appreciate your close review here. The function name findConnLog() and how I see it being used makes it pretty clear that we only ever want it to find true conn records, so I think adding what I've done here is safe and desirable. But please double check me!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me!

| is(ts, <time>)
| is(duration, <duration>)
| is(uid, <string>)
Expand Down
1 change: 0 additions & 1 deletion apps/zui/src/plugins/brimcap/zeek/util.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ export function findUid(value: zed.Value) {
}

const specialUids = {
files: "conn_uids",
dhcp: "uids",
}
if (value.has("_path")) {
Expand Down
1 change: 0 additions & 1 deletion apps/zui/src/ppl/detail/models/Correlation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import {get} from "lodash"
import * as zed from "@brimdata/zed-js"

const specialUids = {
files: "conn_uids",
dhcp: "uids",
}

Expand Down
17 changes: 6 additions & 11 deletions apps/zui/src/ppl/zeek/descriptions.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,19 +244,14 @@ export default {
desc: "An identifier associated with a single file.",
},
{
name: "tx_hosts",
type: "table",
desc: "If this file was transferred over a network connection this should show the host or hosts that the data sourced from.",
},
{
name: "rx_hosts",
type: "table",
desc: "If this file was transferred over a network connection this should show the host or hosts that the data traveled to.",
name: "uid",
type: "string",
desc: "Unique ID for the connection.",
},
{
name: "conn_uids",
type: "table",
desc: "Connection UIDs over which the file was transferred.",
name: "id",
type: "record conn_id",
desc: "The connection's 4-tuple of endpoint addresses/ports.",
},
{
name: "source",
Expand Down
Loading