Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup spread operators in reduce calls #157471

Merged
merged 18 commits into from
May 22, 2023

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented May 12, 2023

Summary

The spread operator is costly and put pressure on GC. It should be avoided when possible, especially in loops.

This PR adapts a lot of reduce calls in the codebase to remove the usages of the diabolic spread operator, when possible.

Note: the PR is not fully exhaustive. I focused on the server-side, as we're more directly impacted than on browser-side code regarding performances.

Removing ... usages in kittens.reduce()

For reduce loops, the spread operator can usually easily be replaced:

- setting a value on the accum object and returning it

BAD

  return this.toArray().reduce(
      (acc, renderer) => ({
        ...acc,
        [renderer.name]: renderer,
      }),
      {} as Record<string, ExpressionRenderer>
    );

GOOD

  return this.toArray().reduce((acc, renderer) => {
      acc[renderer.name] = renderer;
      return acc;
    }, {} as Record<string, ExpressionRenderer>);

- assigning values to the accum object and returning it

BAD

  const allAggs: Record<string, any> = fieldAggRequests.reduce(
      (aggs: Record<string, any>, fieldAggRequest: unknown | null) => {
        return fieldAggRequest ? { ...aggs, ...(fieldAggRequest as Record<string, any>) } : aggs;
      },
      {}
    );

GOOD

  const allAggs = fieldAggRequests.reduce<Record<string, any>>(
      (aggs: Record<string, any>, fieldAggRequest: unknown | null) => {
        if (fieldAggRequest) {
          Object.assign(aggs, fieldAggRequest);
        }
        return aggs;
      },
      {}
    );

- pushing items to the accum list and returning it

BAD

  const charsFound = charToArray.reduce(
    (acc, char) => (value.includes(char) ? [...acc, char] : acc),
    [] as string[]
  );

GOOD

  const charsFound = charToArray.reduce((acc, char) => {
    if (value.includes(char)) {
      acc.push(char);
    }
    return acc;
  }, [] as string[]);

Questions

Are you sure all the changes in this are strictly better for runtime performances?

Yes, yes I am.

How much better?

Likely not much.

Are you planning on analyzing the perf gain?

Nope.

So why did you do it?

I got tired of seeing badly used spread operators in my team's owned code, and I had some extra time during on-week, so I spent a few hours adapting the usages in all our runtime/production codebase.

Was it fun?

Take your best guess.

@pgayvallet pgayvallet changed the title Cleanup spread operations in reduce operations Cleanup spread operators in reduce calls May 12, 2023
@pgayvallet pgayvallet added chore performance release_note:skip Skip the PR/issue when compiling release notes v8.9.0 labels May 15, 2023
@pgayvallet pgayvallet marked this pull request as ready for review May 15, 2023 15:07
@pgayvallet pgayvallet requested review from a team as code owners May 15, 2023 15:07
Copy link
Contributor

@jpdjere jpdjere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detection Rules Managemente LGTM 👍

Copy link
Contributor

@darnautov darnautov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ML changes LGTM

Copy link
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maximpn
Copy link
Contributor

maximpn commented May 17, 2023

@pgayvallet it's a nice idea to improve performance by these changes 👍

I'm curious if there are plans to have some protection against spread operator usage in the code as it can return while the codebase will continue evolving.

On top if that it's interesting if this type of changes can be automated with GhatGPT or any other tool.

Copy link
Contributor

@mattkime mattkime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am proud that my review was not needed.

Code check only.

@pgayvallet
Copy link
Contributor Author

@maximpn

I'm curious if there are plans to have some protection against spread operator usage in the code as it can return while the codebase will continue evolving.

I spent some time searching for the proper es-lint rule, but I couldn't find something suiting exactly what I was looking for (meaning, avoid spread only when unnecessary). AFAIK it's either all or nothing, which is far from being good enough for the kind of precise banning / validation I was looking for.

On top if that it's interesting if this type of changes can be automated with GhatGPT or any other tool.

ChatGPT? Last time I asked him development questions, I'm afraid he failed my interview 😅 .

Any other tool? Maybe. I did not spent time looking for one to be honest.

Copy link
Contributor

@miltonhultgren miltonhultgren left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚢

[] as string[]
);
return indexMaps.reduce((indexNames, indexMap) => {
indexNames.push(...Object.keys(indexMap));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why not replace the spread here with concat?
Is the creation of the new array potentially more expensive than the spread of the currentValue?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean

return indexNames.concat(Object.keys(indexMap));

instead of

indexNames.push(...Object.keys(indexMap));
return indexNames;

right?

Now, yes, I think that the allocation/creation of the new array via concat should be more expensive than pushing the spread of keys (even if said spread creates a new array too, its length / size is smaller than the one returned by concat).

But I have absolutely no numbers or benchmarks to support my assumptions here, so I may also just be wrong. All I know for sure is that both are better than what the code was previously 😄

miltonhultgren

This comment was marked as duplicate.

@pgayvallet
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine kibanamachine requested a review from a team as a code owner May 22, 2023 07:42
@pgayvallet pgayvallet enabled auto-merge (squash) May 22, 2023 07:43
Copy link
Contributor

@e40pud e40pud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detection engine code LGTM!

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Fleet Cypress Tests / View agents list Bulk actions should allow to bulk upgrade agents and cancel that upgrade

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
alerting 183.7KB 183.7KB -4.0B
canvas 993.9KB 993.9KB -7.0B
enterpriseSearch 2.4MB 2.4MB -12.0B
expressionPartitionVis 31.0KB 31.0KB -4.0B
expressionXY 120.0KB 120.0KB -3.0B
infra 2.0MB 2.0MB -3.0B
maps 2.7MB 2.7MB +37.0B
snapshotRestore 266.8KB 266.8KB +24.0B
total +28.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
alerting 49.2KB 49.2KB -4.0B
cases 165.9KB 165.9KB +3.0B
data 404.1KB 404.1KB +5.0B
esUiShared 152.4KB 152.4KB -8.0B
eventAnnotation 19.5KB 19.5KB -4.0B
expressions 98.0KB 98.0KB -4.0B
fleet 129.6KB 129.6KB -8.0B
infra 104.3KB 104.3KB +12.0B
ml 71.5KB 71.5KB +5.0B
uiActions 20.1KB 20.1KB +4.0B
total +1.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
enterpriseSearch 19 21 +2
securitySolution 400 404 +4
total +6

Total ESLint disabled count

id before after diff
enterpriseSearch 20 22 +2
securitySolution 480 484 +4
total +6

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pgayvallet pgayvallet merged commit 8453fe8 into elastic:main May 22, 2023
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label May 22, 2023
delanni pushed a commit to delanni/kibana that referenced this pull request May 25, 2023
## Summary

The spread operator is costly and put pressure on GC. It should be
avoided when possible, especially in loops.

This PR adapts a lot of `reduce` calls in the codebase to remove the
usages of the diabolic spread operator, when possible.

Note: the PR is not fully exhaustive. I focused on the server-side, as
we're more directly impacted than on browser-side code regarding
performances.

## Removing `...` usages in `kittens.reduce()`

For `reduce` loops, the spread operator can usually easily be replaced:

#### - setting a value on the accum object and returning it

#### BAD
```ts
  return this.toArray().reduce(
      (acc, renderer) => ({
        ...acc,
        [renderer.name]: renderer,
      }),
      {} as Record<string, ExpressionRenderer>
    );
```

#### GOOD
```ts
  return this.toArray().reduce((acc, renderer) => {
      acc[renderer.name] = renderer;
      return acc;
    }, {} as Record<string, ExpressionRenderer>);
```


#### - assigning values to the accum object and returning it 

#### BAD
```ts
  const allAggs: Record<string, any> = fieldAggRequests.reduce(
      (aggs: Record<string, any>, fieldAggRequest: unknown | null) => {
        return fieldAggRequest ? { ...aggs, ...(fieldAggRequest as Record<string, any>) } : aggs;
      },
      {}
    );
```

#### GOOD
```ts
  const allAggs = fieldAggRequests.reduce<Record<string, any>>(
      (aggs: Record<string, any>, fieldAggRequest: unknown | null) => {
        if (fieldAggRequest) {
          Object.assign(aggs, fieldAggRequest);
        }
        return aggs;
      },
      {}
    );
```

#### - pushing items to the accum list and returning it 

#### BAD
```ts
  const charsFound = charToArray.reduce(
    (acc, char) => (value.includes(char) ? [...acc, char] : acc),
    [] as string[]
  );
```

#### GOOD
```ts
  const charsFound = charToArray.reduce((acc, char) => {
    if (value.includes(char)) {
      acc.push(char);
    }
    return acc;
  }, [] as string[]);
```

## Questions

#### Are you sure all the changes in this are strictly better for
runtime performances?

Yes, yes I am.

#### How much better?

Likely not much.

#### Are you planning on analyzing the perf gain?

Nope.

#### So why did you do it?

I got tired of seeing badly used spread operators in my team's owned
code, and I had some extra time during on-week, so I spent a few hours
adapting the usages in all our runtime/production codebase.

#### Was it fun?

Take your best guess.

---------

Co-authored-by: Kibana Machine <[email protected]>
@delanni
Copy link
Contributor

delanni commented Oct 17, 2023

Are you planning on analyzing the perf gain?
Nope.

Yet:
#168812

🤨 How should we trust anything anymore?

@pgayvallet
Copy link
Contributor Author

pgayvallet commented Oct 17, 2023

@delanni right, I apologize. I will close the other PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting chore Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) performance release_note:skip Skip the PR/issue when compiling release notes Team:APM All issues that need APM UI Team support Team:Fleet Team label for Observability Data Collection Fleet team Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v8.9.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.