diff --git a/CHANGELOG.md b/CHANGELOG.md index edd4c90c14..d0f61b1038 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to this project will be documented in this file based on the * Reintroduce a streamlined `user_agent` field set. #240, #262 * Add `geo.name` for ad hoc location names. #248 * Add `event.timezone` to allow for proper interpretation of incomplete timestamps. #258 +* Add fields `source.address`, `destination.address`, `client.address`, and + `server.address`. #247 ### Improvements * Improved the definition of the file fields #196 diff --git a/README.md b/README.md index b37241ad99..5904b3885c 100644 --- a/README.md +++ b/README.md @@ -107,11 +107,12 @@ Examples: In the case of Beats for logs, the agent.name is filebeat. For APM, it ## Client fields -A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjuction with server fields. Client fields are generally not populated for packet-level events. +A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events. | Field | Description | Level | Type | Example | |---|---|---|---|---| +| client.address | Some event client addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.
Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword | | | client.ip | IP address of the client.
Can be one or multiple IPv4 or IPv6 addresses. | core | ip | | | client.port | Port of the client. | core | long | | | client.mac | MAC address of the client. | core | keyword | | @@ -161,6 +162,7 @@ Destination fields describe details about the destination of a packet/event. Des | Field | Description | Level | Type | Example | |---|---|---|---|---| +| destination.address | Some event destination addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.
Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword | | | destination.ip | IP address of the destination.
Can be one or multiple IPv4 or IPv6 addresses. | core | ip | | | destination.port | Port of the destination. | core | long | | | destination.mac | MAC address of the destination. | core | keyword | | @@ -414,6 +416,7 @@ A Server is defined as the responder in a network connection for events regardin | Field | Description | Level | Type | Example | |---|---|---|---|---| +| server.address | Some event server addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.
Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword | | | server.ip | IP address of the server.
Can be one or multiple IPv4 or IPv6 addresses. | core | ip | | | server.port | Port of the server. | core | long | | | server.mac | MAC address of the server. | core | keyword | | @@ -444,6 +447,7 @@ Source fields describe details about the source of a packet/event. Source fields | Field | Description | Level | Type | Example | |---|---|---|---|---| +| source.address | Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket. You should always store the raw address in the `.address` field.
Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword | | | source.ip | IP address of the source.
Can be one or multiple IPv4 or IPv6 addresses. | core | ip | | | source.port | Port of the source. | core | long | | | source.mac | MAC address of the source. | core | keyword | | diff --git a/fields.yml b/fields.yml index 963c889925..89be3062ad 100644 --- a/fields.yml +++ b/fields.yml @@ -123,10 +123,21 @@ title: Client group: 2 description: > - A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjuction with server fields. Client fields are generally not populated for packet-level events. + A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events. type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event client addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip @@ -292,6 +303,17 @@ type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event destination addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip @@ -1258,6 +1280,17 @@ type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event server addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip @@ -1388,6 +1421,17 @@ type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event source addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip diff --git a/schema.csv b/schema.csv index c9288c08b2..129bcaa299 100644 --- a/schema.csv +++ b/schema.csv @@ -8,6 +8,7 @@ agent.id,keyword,core,8a4f500d agent.name,keyword,core,foo agent.type,keyword,core,filebeat agent.version,keyword,core,6.0.0-rc2 +client.address,keyword,extended, client.bytes,long,core,184 client.domain,keyword,core, client.ip,ip,core, @@ -27,6 +28,7 @@ container.image.tag,keyword,extended, container.labels,object,extended, container.name,keyword,extended, container.runtime,keyword,extended,docker +destination.address,keyword,extended, destination.bytes,long,core,184 destination.domain,keyword,core, destination.ip,ip,core, @@ -128,6 +130,7 @@ process.thread.id,long,extended,4242 process.title,keyword,extended, process.working_directory,keyword,extended,/home/alice related.ip,ip,extended, +server.address,keyword,extended, server.bytes,long,core,184 server.domain,keyword,core, server.ip,ip,core, @@ -140,6 +143,7 @@ service.name,keyword,core,elasticsearch-metrics service.state,keyword,core, service.type,keyword,core,elasticsearch service.version,keyword,core,3.2.4 +source.address,keyword,extended, source.bytes,long,core,184 source.domain,keyword,core, source.ip,ip,core, diff --git a/schemas/client.yml b/schemas/client.yml index e81a741d9d..5aa49f4924 100644 --- a/schemas/client.yml +++ b/schemas/client.yml @@ -3,10 +3,21 @@ title: Client group: 2 description: > - A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjuction with server fields. Client fields are generally not populated for packet-level events. + A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields. Client fields are generally not populated for packet-level events. type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event client addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip diff --git a/schemas/destination.yml b/schemas/destination.yml index a03e5a69bb..73c52d78f2 100644 --- a/schemas/destination.yml +++ b/schemas/destination.yml @@ -8,6 +8,17 @@ type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event destination addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip diff --git a/schemas/server.yml b/schemas/server.yml index 504b07300e..66246327b3 100644 --- a/schemas/server.yml +++ b/schemas/server.yml @@ -7,6 +7,17 @@ type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event server addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip diff --git a/schemas/source.yml b/schemas/source.yml index a9e184c6bd..1efff6fe1c 100644 --- a/schemas/source.yml +++ b/schemas/source.yml @@ -8,6 +8,17 @@ type: group fields: + - name: address + level: extended + type: keyword + description: > + Some event source addresses are defined ambiguously. The event will + sometimes list an IP, a domain or a unix socket. You should always + store the raw address in the `.address` field. + + Then it should be duplicated to `.ip` or `.domain`, depending on which + one it is. + - name: ip level: core type: ip diff --git a/template.json b/template.json index 6e19e02521..1cbd8a1b12 100644 --- a/template.json +++ b/template.json @@ -49,6 +49,10 @@ }, "client": { "properties": { + "address": { + "ignore_above": 1024, + "type": "keyword" + }, "bytes": { "type": "long" }, @@ -148,6 +152,10 @@ }, "destination": { "properties": { + "address": { + "ignore_above": 1024, + "type": "keyword" + }, "bytes": { "type": "long" }, @@ -613,6 +621,10 @@ }, "server": { "properties": { + "address": { + "ignore_above": 1024, + "type": "keyword" + }, "bytes": { "type": "long" }, @@ -665,6 +677,10 @@ }, "source": { "properties": { + "address": { + "ignore_above": 1024, + "type": "keyword" + }, "bytes": { "type": "long" },