Add common pipeline and mapping issues and mitigations

docs/en/observability/logs-troubleshooting.asciidoc

@@ -1,11 +1,19 @@
 [[logs-troubleshooting]]
 = Troubleshoot logs
 
+Use this page to find possible solutions for errors your encountering with your logs. This troubleshooting page is broken into the following sections:
+
+- <<logs-onboarding-troubleshooting>>
+- <<logs-common-mapping-troubleshooting, Mapping and pipeline issues>>
+
+[[logs-onboarding-troubleshooting]]
+== Common onboarding issues
+
 This section provides possible solutions for errors you might encounter while onboarding your logs.
 
 [discrete]
 [[logs-troubleshooting-insufficient-priv]]
-== User does not have permissions to create API key
+=== User does not have permissions to create API key
 
 If you don't have the required privileges to create an API key, you'll see the following error message:
 
@@ -21,7 +29,7 @@ to the role of the authenticated user.
 
 [discrete]
 [[logs-troubleshooting-insufficient-priv-solution]]
-=== Solution
+==== Solution
 
 You need to either:
 
@@ -30,7 +38,7 @@ You need to either:
 
 [discrete]
 [[logs-troubleshooting-API-key-failed]]
-== Failed to create API key
+=== Failed to create API key
 
 If you don't have the privileges to create `savedObjects` in {kib}, you'll see the following error message:
 
@@ -43,14 +51,14 @@ Something went wrong: Unable to create observability-onboarding-state
 
 [discrete]
 [[logs-troubleshooting-API-key-failed-solution]]
-=== Solution
+==== Solution
 
 You need an administrator to give you the `Saved Objects Management` {kib} privilege to generate the required `observability-onboarding-state` flow state.
 Once you have the necessary privileges, restart the onboarding flow.
 
 [discrete]
 [[logs-troubleshooting-kib-not-accessible]]
-== {kib} not accessible from host
+=== {kib} not accessible from host
 
 If {kib} is not accessible from the host, you'll see the following error message after pasting the *Install the {agent}* instructions into the host:
 
@@ -61,7 +69,7 @@ Failed to connect to {host} port {port} after 0 ms: Connection refused
 
 [discrete]
 [[logs-troubleshooting-kib-not-accessible-solution]]
-=== Solution
+==== Solution
 
 The host needs access to {kib}. Port `443` must be open and the deployment's {es} endpoint must be reachable. Locate your project's endpoint from *Help menu (image:images/help-icon.png[]) → Connection details*.
 
@@ -73,7 +81,7 @@ curl https://your-endpoint.elastic.cloud
 
 [discrete]
 [[logs-troubleshooting-download-agent]]
-== Download {agent} failed
+=== Download {agent} failed
 
 If the host was able to download the installation script but cannot connect to the public artifact repository, you'll see the following error message:
 
@@ -86,7 +94,7 @@ Failed to download Elastic Agent, see script for error.
 
 [discrete]
 [[logs-troubleshooting-download-agent-solution]]
-=== Solutions
+==== Solutions
 
 * If the combination of the {agent} version and operating system architecture is not available, you'll see the following error message:
 +
@@ -108,7 +116,7 @@ To fix this, delete previous downloads and restart the onboarding.
 
 [discrete]
 [[logs-troubleshooting-install-agent]]
-== Install {agent} failed
+=== Install {agent} failed
 
 If an {agent} already exists on your host, you'll see the following error message:
 
@@ -121,19 +129,93 @@ Failed to install Elastic Agent, see script for error.
 
 [discrete]
 [[logs-troubleshooting-install-agent-solution]]
-=== Solution
+==== Solution
 You can uninstall the current {agent} using the `elastic-agent uninstall` command, and run the script again.
 
 WARNING: Uninstalling the current {agent} removes the entire current setup, including the existing configuration.
 
 [discrete]
 [[logs-troubleshooting-wait-for-logs]]
-== Waiting for Logs to be shipped... step never completes
+=== Waiting for Logs to be shipped... step never completes
 
 If the *Waiting for Logs to be shipped...* step never completes, logs are not being shipped to {es}, and there is most likely an issue with your {agent} configuration.
 
 [discrete]
 [[logs-troubleshooting-wait-for-logs-solution]]
-=== Solution
+==== Solution
+
+Inspect the {agent} logs for errors. See the {fleet-guide}/debug-standalone-agents.html#inspect-standalone-agent-logs[Debug standalone {agent}s] documentation for more on finding errors in {agent} logs.
+
+[[logs-common-mapping-troubleshooting]]
+== Mapping and pipeline issues
+
+This section provides possible solutions for mapping and pipeline issues you might encounter with your logs.
+
+[[logs-mapping-troubleshooting-keyword-limit]]
+=== Keyword fields are too long
+The `keyword` field limit is 32,766 bytes. When indexing a document, if your `keyword` field length exceeds this limit, you'll see an error similar to the following:
+
+[source, plaintext]
+----
+max_bytes_length_exceeded_exception: bytes can be at most 32766 in length
+----
+
+[discrete]
+[[logs-mapping-troubleshooting-keyword-limit-solution]]
+==== Solution
+Avoid this error using one of the following options:
+
+*Stop indexing the field:* If you don't need the `keyword` field for aggregation or search, set `"index":false` in the index template to stop indexing the field.
+
+*Increase `ignore_above`:* To index any `keyword` fields under the field limit, but not those over the limit, set the `ignore_above` setting in the index template.
+
+NOTE: The value for `ignore_above` is the _character count_, but Lucene counts bytes.
+If you use UTF-8 text with many non-ASCII characters, you may want to set the limit to `32766 / 4 = 8191` since UTF-8 characters may occupy at most 4 bytes.
+
+Refer to the {ref}/ignore-above.html[`ignore_above`] docs for more information.
+
+*Convert the `keyword` field to a `text` field:* To continue indexing the field while avoiding length limits, you can convert the `keyword` field to a `text` field. To do this:
+
+. Create a new index with the `text` field data type.
+. Reindex from the `_source` field of the source index using the {ref}/docs-reindex.html[`_reindex` API].
+
+[discrete]
+[[logs-mapping-troubleshooting-date-mismatch]]
+=== Date format mismatch
+If the format of the `date` field in your document doesn't match the format set in your index template, you'll see an error similar to the following:
+
+[source, plaintext]
+----
+failed to parse field [date] of type [date] in document with id 'KGcZb3cBqhj6kAxank_x'.
+----
+
+[discrete]
+[[logs-mapping-troubleshooting-date-solution]]
+==== Solution
+Avoid this error by adding the format of the mismatched date to your index template.
+Multiple formats can be specified by separating them with `||` as a separator.
+Each format will be tried in turn until a matching format is found.
+For example:
+
+[source,console,id=date-format-example]
+----
+PUT my-index-000001
+{
+  "mappings": {
+    "properties": {
+      "date": {
+        "type":   "date",
+        "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
+      }
+    }
+  }
+}
+----
+
+Refer to the {ref}/date.html[`date` field type] docs for more information.
+
+[[logs-mapping-troubleshooting-grok-mismatch]]
+=== Grok or dissect pattern mismatch
 
-Inspect the {agent} logs for errors. See the {fleet-guide}/debug-standalone-agents.html#inspect-standalone-agent-logs[Debug standalone {agent}s] documentation for more on finding errors in {agent} logs.
+[[logs-mapping-troubleshooting-string-mismatch]]
+=== String/number or Float/int mismatch