Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet] Fields with dynamic types are silently ignored for some types #168823

Closed
jsoriano opened this issue Oct 13, 2023 · 1 comment · Fixed by #168842
Closed

[Fleet] Fields with dynamic types are silently ignored for some types #168823

jsoriano opened this issue Oct 13, 2023 · 1 comment · Fixed by #168842
Assignees
@jsoriano
Labels
bug Fixes for quality problems that affect the customer experience Team:Fleet Team label for Observability Data Collection Fleet team

Comments

@jsoriano
Copy link
Member

Kibana version: Probably all.

Describe the bug:

Dynamic mapping definitions for some types are silently ignored and don't generate any mapping.

For example this field definition doesn't generate any mapping, nor produces any failure:

- name: numeric_labels
  type: object
  object_type: scaled_float
  dynamic: true

If the object_type is changed to other types as double, the mapping is generated.

The "compact" syntax is also ignored:

- name: numeric_labels.*
  type: scaled_float
  dynamic: true

Steps to reproduce:

  1. Define a field with dynamic mapping in a package with the scaled_float type.
  2. Install this package.
  3. Check that there is no mapping defined for this field.

Expected behavior:

The field mapping is installed, or if it is incorrect, a failure is produced.

Any additional context:

@kyungeunni identified that the issue could be around the following code, where there is some special handling for some types, but not for all of them: https://github.com/elastic/kibana/blob/c000a25f41474df052d6fefe187916455e515e24/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/template.ts#L284C26-L290

Thanks @kyungeunni for identifying this issue!
@jsoriano jsoriano added bug Fixes for quality problems that affect the customer experience Team:Fleet Team label for Observability Data Collection Fleet team labels Oct 13, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@jsoriano jsoriano self-assigned this Oct 13, 2023
@jsoriano jsoriano mentioned this issue Oct 13, 2023
Merged
2 tasks
jsoriano added a commit that referenced this issue Oct 17, 2023
@jsoriano
[Fleet] Add support for additional types for dynamic mappings (#168842)
d2165f3 
Add support for fields that generate dynamic mappings of the following
types:
- match_only_text
- scaled_float
- aggregate_metric_double
- half_float
- ip
- flattened
- integer (mapped as long as in other cases)
- group (child fields are installed as dynamic mappings)

Default `match_matching_type` has been corrected to provide always a
valid value. For example for floats it was using `float`, what is invalid, for
this case it is changed to `double`.

When the type is not known, an error is raised now instead of silently
ignoring the field definition, this helps package developers discover earlier
if their mappings are not well supported.

Tested with the `install_all_packages` and no current package fails
because of this.

Fix #168823
@tommyers-elastic tommyers-elastic mentioned this issue Oct 17, 2023
Merged
19 tasks
dej611 pushed a commit to dej611/kibana that referenced this issue Oct 17, 2023
@jsoriano @dej611
[Fleet] Add support for additional types for dynamic mappings (elasti…
ca8500d 
…c#168842)

Add support for fields that generate dynamic mappings of the following
types:
- match_only_text
- scaled_float
- aggregate_metric_double
- half_float
- ip
- flattened
- integer (mapped as long as in other cases)
- group (child fields are installed as dynamic mappings)

Default `match_matching_type` has been corrected to provide always a
valid value. For example for floats it was using `float`, what is invalid, for
this case it is changed to `double`.

When the type is not known, an error is raised now instead of silently
ignoring the field definition, this helps package developers discover earlier
if their mappings are not well supported.

Tested with the `install_all_packages` and no current package fails
because of this.

Fix elastic#168823
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsoriano jsoriano

Labels
bug Fixes for quality problems that affect the customer experience Team:Fleet Team label for Observability Data Collection Fleet team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Fleet] Add support for additional types for dynamic mappings jsoriano/kibana
2 participants
@jsoriano @elasticmachine