Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kedro-telemetry: Improve performance by switching to after_command_run #709

Closed
ankatiyar opened this issue May 30, 2024 · 1 comment
Closed
Assignees

Comments

@ankatiyar
Copy link
Contributor

ankatiyar commented May 30, 2024

Description

Follow up on kedro-org/kedro#1476

From profiling various Kedro CLI commands, it turns out that a good chunk of the time is spent sending information to Heap in the before_command_run hook. This issue is to improve things on kedro-telemetry side.

image

Context

kedro-org/kedro#1476 (comment)
kedro-org/kedro#1476 (comment)

Possible Implementations

  • Move to after_command_run hook, but this does not get executed if an exception occurs during the kedro run etc. The hook could be changed on the Kedro side to get executed even if the command fails.
  • Try asynchronous sending of Heap data but this might be tricky since it's still an open request on pluggy support async hook calls pytest-dev/pluggy#320

Also to consider

There's some project hooks executed by kedro-telemetry too - we could figure out how to improve performance by not having to send this data twice

@merelcht merelcht moved this to To Do in Kedro Framework Jun 20, 2024
@lrcouto lrcouto moved this from To Do to In Progress in Kedro Framework Jul 15, 2024
@ankatiyar ankatiyar changed the title kedro-telemetry: Improve performance kedro-telemetry: Improve performance by switching to after_command_run Jul 16, 2024
@ankatiyar
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

No branches or pull requests

3 participants