Introduce source.address, destination.address, client.address a…

…nd `server.address`. (elastic#247)

CHANGELOG.md

@@ -38,6 +38,8 @@ All notable changes to this project will be documented in this file based on the
 * Reintroduce a streamlined `user_agent` field set. #240, #262
 * Add `geo.name` for ad hoc location names. #248
 * Add `event.timezone` to allow for proper interpretation of incomplete timestamps. #258
+* Add fields `source.address`, `destination.address`, `client.address`, and
+  `server.address`. #247
 
 ### Improvements
 * Improved the definition of the file fields #196

README.md

@@ -107,11 +107,12 @@ Examples: In the case of Beats for logs, the agent.name is filebeat. For APM, it
 
 ## <a name="client"></a> Client fields
 
-A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjuction with server fields.  Client fields are generally not populated for packet-level events. 
+A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields.  Client fields are generally not populated for packet-level events.
 
 
 | Field  | Description  | Level  | Type  | Example  |
 |---|---|---|---|---|
+| <a name="client.address"></a>client.address | Some event client addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket.  You should always store the raw address in the `.address` field.<br/>Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword |  |
 | <a name="client.ip"></a>client.ip | IP address of the client.<br/>Can be one or multiple IPv4 or IPv6 addresses. | core | ip |  |
 | <a name="client.port"></a>client.port | Port of the client. | core | long |  |
 | <a name="client.mac"></a>client.mac | MAC address of the client. | core | keyword |  |
@@ -161,6 +162,7 @@ Destination fields describe details about the destination of a packet/event. Des
 
 | Field  | Description  | Level  | Type  | Example  |
 |---|---|---|---|---|
+| <a name="destination.address"></a>destination.address | Some event destination addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket.  You should always store the raw address in the `.address` field.<br/>Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword |  |
 | <a name="destination.ip"></a>destination.ip | IP address of the destination.<br/>Can be one or multiple IPv4 or IPv6 addresses. | core | ip |  |
 | <a name="destination.port"></a>destination.port | Port of the destination. | core | long |  |
 | <a name="destination.mac"></a>destination.mac | MAC address of the destination. | core | keyword |  |
@@ -414,6 +416,7 @@ A Server is defined as the responder in a network connection for events regardin
 
 | Field  | Description  | Level  | Type  | Example  |
 |---|---|---|---|---|
+| <a name="server.address"></a>server.address | Some event server addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket.  You should always store the raw address in the `.address` field.<br/>Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword |  |
 | <a name="server.ip"></a>server.ip | IP address of the server.<br/>Can be one or multiple IPv4 or IPv6 addresses. | core | ip |  |
 | <a name="server.port"></a>server.port | Port of the server. | core | long |  |
 | <a name="server.mac"></a>server.mac | MAC address of the server. | core | keyword |  |
@@ -444,6 +447,7 @@ Source fields describe details about the source of a packet/event. Source fields
 
 | Field  | Description  | Level  | Type  | Example  |
 |---|---|---|---|---|
+| <a name="source.address"></a>source.address | Some event source addresses are defined ambiguously. The event will sometimes list an IP, a domain or a unix socket.  You should always store the raw address in the `.address` field.<br/>Then it should be duplicated to `.ip` or `.domain`, depending on which one it is. | extended | keyword |  |
 | <a name="source.ip"></a>source.ip | IP address of the source.<br/>Can be one or multiple IPv4 or IPv6 addresses. | core | ip |  |
 | <a name="source.port"></a>source.port | Port of the source. | core | long |  |
 | <a name="source.mac"></a>source.mac | MAC address of the source. | core | keyword |  |

fields.yml

@@ -123,10 +123,21 @@
       title: Client
       group: 2
       description: >
-        A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjuction with server fields.  Client fields are generally not populated for packet-level events. 
+        A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields.  Client fields are generally not populated for packet-level events.
       type: group
       fields:
 
+        - name: address
+          level: extended
+          type: keyword
+          description: >
+            Some event client addresses are defined ambiguously. The event will
+            sometimes list an IP, a domain or a unix socket.  You should always
+            store the raw address in the `.address` field.
+    
+            Then it should be duplicated to `.ip` or `.domain`, depending on which
+            one it is.
+
         - name: ip
           level: core
           type: ip
@@ -292,6 +303,17 @@
       type: group
       fields:
 
+        - name: address
+          level: extended
+          type: keyword
+          description: >
+            Some event destination addresses are defined ambiguously. The event will
+            sometimes list an IP, a domain or a unix socket.  You should always
+            store the raw address in the `.address` field.
+    
+            Then it should be duplicated to `.ip` or `.domain`, depending on which
+            one it is.
+
         - name: ip
           level: core
           type: ip
@@ -1258,6 +1280,17 @@
       type: group
       fields:
 
+        - name: address
+          level: extended
+          type: keyword
+          description: >
+            Some event server addresses are defined ambiguously. The event will
+            sometimes list an IP, a domain or a unix socket.  You should always
+            store the raw address in the `.address` field.
+    
+            Then it should be duplicated to `.ip` or `.domain`, depending on which
+            one it is.
+
         - name: ip
           level: core
           type: ip
@@ -1388,6 +1421,17 @@
       type: group
       fields:
 
+        - name: address
+          level: extended
+          type: keyword
+          description: >
+            Some event source addresses are defined ambiguously. The event will
+            sometimes list an IP, a domain or a unix socket.  You should always
+            store the raw address in the `.address` field.
+    
+            Then it should be duplicated to `.ip` or `.domain`, depending on which
+            one it is.
+
         - name: ip
           level: core
           type: ip

schema.csv

@@ -8,6 +8,7 @@ agent.id,keyword,core,8a4f500d
 agent.name,keyword,core,foo
 agent.type,keyword,core,filebeat
 agent.version,keyword,core,6.0.0-rc2
+client.address,keyword,extended,
 client.bytes,long,core,184
 client.domain,keyword,core,
 client.ip,ip,core,
@@ -27,6 +28,7 @@ container.image.tag,keyword,extended,
 container.labels,object,extended,
 container.name,keyword,extended,
 container.runtime,keyword,extended,docker
+destination.address,keyword,extended,
 destination.bytes,long,core,184
 destination.domain,keyword,core,
 destination.ip,ip,core,
@@ -128,6 +130,7 @@ process.thread.id,long,extended,4242
 process.title,keyword,extended,
 process.working_directory,keyword,extended,/home/alice
 related.ip,ip,extended,
+server.address,keyword,extended,
 server.bytes,long,core,184
 server.domain,keyword,core,
 server.ip,ip,core,
@@ -140,6 +143,7 @@ service.name,keyword,core,elasticsearch-metrics
 service.state,keyword,core,
 service.type,keyword,core,elasticsearch
 service.version,keyword,core,3.2.4
+source.address,keyword,extended,
 source.bytes,long,core,184
 source.domain,keyword,core,
 source.ip,ip,core,

schemas/client.yml

@@ -3,10 +3,21 @@
   title: Client
   group: 2
   description: >
-    A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjuction with server fields.  Client fields are generally not populated for packet-level events. 
+    A client is defined as the initiator of a network connection for events regarding sessions, connections, or bidirectional flow records. For TCP events, the client is the initiator of the TCP connection that sends the SYN packet(s). For other protocols, the client is generally the initiator or requestor in the network transaction. Some systems use the term "originator" to refer the client in TCP connections. The client fields describe details about the system acting as the client in the network event. Client fields are usually populated in conjunction with server fields.  Client fields are generally not populated for packet-level events.
   type: group
   fields:
 
+    - name: address
+      level: extended
+      type: keyword
+      description: >
+        Some event client addresses are defined ambiguously. The event will
+        sometimes list an IP, a domain or a unix socket.  You should always
+        store the raw address in the `.address` field.
+
+        Then it should be duplicated to `.ip` or `.domain`, depending on which
+        one it is.
+
     - name: ip
       level: core
       type: ip

schemas/destination.yml

@@ -8,6 +8,17 @@
   type: group
   fields:
 
+    - name: address
+      level: extended
+      type: keyword
+      description: >
+        Some event destination addresses are defined ambiguously. The event will
+        sometimes list an IP, a domain or a unix socket.  You should always
+        store the raw address in the `.address` field.
+
+        Then it should be duplicated to `.ip` or `.domain`, depending on which
+        one it is.
+
     - name: ip
       level: core
       type: ip

schemas/server.yml

@@ -7,6 +7,17 @@
   type: group
   fields:
 
+    - name: address
+      level: extended
+      type: keyword
+      description: >
+        Some event server addresses are defined ambiguously. The event will
+        sometimes list an IP, a domain or a unix socket.  You should always
+        store the raw address in the `.address` field.
+
+        Then it should be duplicated to `.ip` or `.domain`, depending on which
+        one it is.
+
     - name: ip
       level: core
       type: ip

schemas/source.yml

@@ -8,6 +8,17 @@
   type: group
   fields:
 
+    - name: address
+      level: extended
+      type: keyword
+      description: >
+        Some event source addresses are defined ambiguously. The event will
+        sometimes list an IP, a domain or a unix socket.  You should always
+        store the raw address in the `.address` field.
+
+        Then it should be duplicated to `.ip` or `.domain`, depending on which
+        one it is.
+
     - name: ip
       level: core
       type: ip

template.json

@@ -49,6 +49,10 @@
         },
         "client": {
           "properties": {
+            "address": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "bytes": {
               "type": "long"
             },
@@ -148,6 +152,10 @@
         },
         "destination": {
           "properties": {
+            "address": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "bytes": {
               "type": "long"
             },
@@ -613,6 +621,10 @@
         },
         "server": {
           "properties": {
+            "address": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "bytes": {
               "type": "long"
             },
@@ -665,6 +677,10 @@
         },
         "source": {
           "properties": {
+            "address": {
+              "ignore_above": 1024,
+              "type": "keyword"
+            },
             "bytes": {
               "type": "long"
             },