Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimize /runOpenApi to allow pre-dereferencing OpenAPI Spec #750

Merged
merged 3 commits into from
Mar 9, 2024

Conversation

snowe2010
Copy link
Contributor

@snowe2010 snowe2010 commented Mar 8, 2024

This PR fixes an issue where the openApi plugin would try to dereference the
OpenAPI spec for each plugin when the plugin is run rather than on
initialization.

I did not attempt to fix the issue for every plugin, I simply wanted to start with
the one that was affecting me the most, the Github plugin. But this PR
adds the capability to allow any OpenAPI plugin to now use a pre-dereferenced
OpenAPI.Document instead of de-referencing during runtime.

Before:

/plugins averaged 19.472ms before the change
/runPluginQuery averaged 6867.030ms before the change

GET /node-service/api/plugins 200 17.112 ms - 31024
POST /node-service/api/runPluginQuery 200 6622.313 ms - 156023
GET /node-service/api/plugins 200 17.698 ms - 31024
POST /node-service/api/runPluginQuery 200 6632.372 ms - 156023
GET /node-service/api/plugins 200 25.301 ms - 31024
POST /node-service/api/runPluginQuery 200 7055.684 ms - 156023
GET /node-service/api/plugins 200 17.178 ms - 31024
POST /node-service/api/runPluginQuery 200 6835.205 ms - 156023
GET /node-service/api/plugins 200 16.735 ms - 31024
POST /node-service/api/runPluginQuery 200 7005.931 ms - 156023
GET /node-service/api/plugins 200 22.808 ms - 31024
POST /node-service/api/runPluginQuery 200 7050.673 ms - 156023

After:

/plugins averaged 20.302ms after the change
/runPluginQuery averaged 1533.787ms after the change

GET /node-service/api/plugins 200 77.839 ms - 31024
POST /node-service/api/runPluginQuery 200 1824.782 ms - 156023
GET /node-service/api/plugins 200 6.849 ms - 31024
POST /node-service/api/runPluginQuery 200 1525.905 ms - 156023
GET /node-service/api/plugins 200 15.647 ms - 31024
POST /node-service/api/runPluginQuery 200 1400.096 ms - 156023
GET /node-service/api/plugins 200 3.265 ms - 31024
POST /node-service/api/runPluginQuery 200 1498.696 ms - 156023
GET /node-service/api/plugins 200 9.287 ms - 31024
POST /node-service/api/runPluginQuery 200 1625.815 ms - 156023
GET /node-service/api/plugins 200 8.925 ms - 31024
POST /node-service/api/runPluginQuery 200 1427.427 ms - 156023

That's a 77.67% decrease in runtime for the /runPluginQuery, but a 4.27% increase for the /plugins query.
Without the 77ms outlier the results are much nicer:

(6.849 ms + 15.647 ms + 3.265 ms + 9.287 ms + 8.925 ms) / 5 = ≈ 8.795 ms
[(19.472 - 8.795) / 19.472] * 100 = ≈ 54.84 %

54.84% decrease in runtime for the /plugins query and a 77.67% decrease in runtime for the /runPluginQuery

allow passing in an already dereferenced OpenAPI Document for plugins
that have large openapi specs like github. This takes the runtime for
Github from over 10s to ~1s on an M1 Max.

Adjust the github plugin to use this new capability by dereferencing on
plugin initialization, passing the derefed document to the runOpenApi
function.
Copy link

netlify bot commented Mar 8, 2024

👷 Deploy request for lowcoder-cloud pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit 275cf68

@snowe2010 snowe2010 changed the title optimize runOpenApi optimize /runOpenApi to allow pre-dereferencing OpenAPI Spec Mar 8, 2024
Copy link
Contributor

@FalkWolsky FalkWolsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for this truly helpful contribution!

@FalkWolsky FalkWolsky merged commit ffa595a into lowcoder-org:main Mar 9, 2024
3 checks passed
@snowe2010 snowe2010 deleted the optimize-runOpenApi branch March 9, 2024 04:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants