Skip to content

Commit

Permalink
[Fleet] Create intermediate objects when using dynamic mappings (#169981
Browse files Browse the repository at this point in the history
)

For `dynamic: "runtime"`, and possibly other configurations different to
`dynamic: true`, intermediate objects need to exist for dynamic
mappings, otherwise ingestion can fail with `Missing intermediate object`
errors.

This pull request includes these intermediate objects, and avoids the
creation of static properties with wildcards, that is probably not what
is expected for these mappings.

Intermediate objects are included as static properties for parts of the
name before the wildcard, and as dynamic templates when the full path
has wildcards.

In the modified test files are examples of all of these.

This is more an issue since the following recent fixes, that create the
actual dynamic mappings for some cases where only empty objects were
created before.
* elastic/elastic-package#1492
* #168842
  • Loading branch information
jsoriano authored Nov 6, 2023
1 parent 09ff2c4 commit d398606
Show file tree
Hide file tree
Showing 3 changed files with 207 additions and 55 deletions.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -1151,7 +1151,12 @@ describe('EPM template', () => {
runtime: true
`;
const runtimeFieldMapping = {
properties: {},
properties: {
labels: {
type: 'object',
dynamic: true,
},
},
dynamic_templates: [
{
'labels.*': {
Expand All @@ -1177,7 +1182,12 @@ describe('EPM template', () => {
object_type: scaled_float
`;
const runtimeFieldMapping = {
properties: {},
properties: {
numeric_labels: {
type: 'object',
dynamic: true,
},
},
dynamic_templates: [
{
numeric_labels: {
Expand Down Expand Up @@ -1205,7 +1215,12 @@ describe('EPM template', () => {
default_metric: "max"
`;
const runtimeFieldMapping = {
properties: {},
properties: {
aggregate: {
type: 'object',
dynamic: true,
},
},
dynamic_templates: [
{
'aggregate.*': {
Expand All @@ -1226,7 +1241,7 @@ describe('EPM template', () => {
expect(mappings).toEqual(runtimeFieldMapping);
});

it('tests processing groub sub fields in a dynamic template', () => {
it('tests processing group sub fields in a dynamic template', () => {
const textWithRuntimeFieldsLiteralYml = `
- name: group.*.network
type: group
Expand All @@ -1236,7 +1251,12 @@ describe('EPM template', () => {
metric_type: counter
`;
const runtimeFieldMapping = {
properties: {},
properties: {
group: {
type: 'object',
dynamic: true,
},
},
dynamic_templates: [
{
'group.*.network.bytes': {
Expand All @@ -1248,6 +1268,26 @@ describe('EPM template', () => {
},
},
},
{
'group.*.network': {
path_match: 'group.*.network',
match_mapping_type: 'object',
mapping: {
type: 'object',
dynamic: true,
},
},
},
{
'group.*': {
path_match: 'group.*',
match_mapping_type: 'object',
mapping: {
type: 'object',
dynamic: true,
},
},
},
],
};
const fields: Field[] = safeLoad(textWithRuntimeFieldsLiteralYml);
Expand Down
Loading

0 comments on commit d398606

Please sign in to comment.