Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(pluginservers): code refactor & testing #12858

Merged
merged 1 commit into from
Nov 13, 2024

Conversation

gszr
Copy link
Member

@gszr gszr commented Apr 13, 2024

Context

The overall goal of this commit is to refactor the external plugins
implementation, with the following goals in mind:

  • Make plugin server code more approachable to unfamiliar engineers and
    easier to evolve with confidence
  • Harden configuration; ensure configuration defects are caught before
    Kong is started
  • Extend testing coverage

This is related to ongoing work on the Go PDK, with similar goals in
mind.

Summary

This commit implements the following overall changes to plugin server
code:

  • Move configuration related code into conf loader, so that configuration
    loading and validation happens at startup time, rather than lazily, when
    plugin data is loaded or pluginservers are started. Add tests for
    current behavior.

  • Move process-management code - for starting up plugin servers as well
    as querying external plugins info - into the process.lua module.

  • Introduce a kong.runloop.plugin_servers.rpc module that encapsulates
    RPC initialization and protocol-specific implementations. This further
    simplifies the main plugin server main module.

  • Factor exposed API and phase handlers bridging code into a new plugin
    module, which encapsulates an external plugin representation, including
    the expected fields for any Kong plugin, plus external plugin-specific
    bits, such as the RPC instance. Part of this external plugin-specific part
    is the instance life cycle management. With this structure, the kong.runloop.plugin_servers
    main module contains only general external plugin code, including a list
    of loaded external plugins, and associated start/stop functions for
    plugin servers.

Testing

This commit also implements the following improvements to tests:

  • Restructure fixtures to accommodate new external plugin servers --
    namely, targeting for now in the existing Python and Javascript
  • Add new test cases for external plugins:
    • External plugin configuration: add test cases for current behavior;
      in particular:
      • Fail if no query_cmd is provided;
      • Warn if no start_cmd is provided - this is by design, as
        external plugins servers can be managed outside of Kong
    • Plugin server start / stop - for both Go and Python plugins
    • External plugin info querying for both Go and Python plugins
    • External plugin execution - for both Go and Python plugins<!--

KAG-2496

@github-actions github-actions bot added core/language/go core/proxy core/configuration core/language/python core/language/js chore Not part of the core functionality of kong, but still needed cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee labels Apr 13, 2024
@gszr gszr force-pushed the chore/plugin-server-improvements branch 2 times, most recently from 50d7d76 to 38ddd0e Compare April 13, 2024 13:29
@hanshuebner hanshuebner self-assigned this Apr 16, 2024
kong/init.lua Outdated Show resolved Hide resolved
kong/init.lua Outdated Show resolved Hide resolved
kong/runloop/plugin_servers/init.lua Outdated Show resolved Hide resolved
kong/runloop/plugin_servers/plugin.lua Show resolved Hide resolved
kong/runloop/plugin_servers/plugin.lua Outdated Show resolved Hide resolved
kong/runloop/plugin_servers/rpc/init.lua Outdated Show resolved Hide resolved
@gszr gszr force-pushed the chore/plugin-server-improvements branch 3 times, most recently from 16a4581 to dcad94d Compare September 24, 2024 20:13
@gszr gszr force-pushed the chore/plugin-server-improvements branch 2 times, most recently from 5b3e1df to 4807471 Compare October 10, 2024 16:09
@gszr gszr marked this pull request as ready for review October 10, 2024 16:09
@gszr gszr force-pushed the chore/plugin-server-improvements branch 3 times, most recently from 318a540 to 1ca4b8b Compare October 10, 2024 17:55
@gszr gszr force-pushed the chore/plugin-server-improvements branch 3 times, most recently from 3e69461 to 2245cde Compare October 10, 2024 18:01
@gszr gszr force-pushed the chore/plugin-server-improvements branch 6 times, most recently from 65c0931 to 748ebc3 Compare October 15, 2024 17:41
Context
-------

The overall goal of this commit is to refactor the external plugins
implementation, with the following goals in mind:
- Make plugin server code more approachable to unfamiliar engineers and
easier to evolve with confidence
- Harden configuration; ensure configuration defects are caught before
Kong is started
- Extend testing coverage

This is related to ongoing work on the Go PDK, with similar goals in
mind.

Summary
-------

This commit implements the following overall changes to plugin server
code:

- Move configuration related code into conf loader, so that configuration
loading and validation happens at startup time, rather than lazily, when
plugin data is loaded or pluginservers are started. Add tests for
current behavior.

- Move process-management code - for starting up plugin servers as well
as querying external plugins info - into the `process.lua` module.

- Introduce a `kong.runloop.plugin_servers.rpc` module that encapsulates
RPC initialization and protocol-specific implementations. This further
simplifies the main plugin server main module.

- Factor exposed API and phase handlers bridging code into a new `plugin`
module, which encapsulates an external plugin representation, including
the expected fields for any Kong plugin, plus external plugin-specific
bits, such as the RPC instance. Part of this external plugin-specific part
is the instance life cycle management. With this structure, the `kong.runloop.plugin_servers`
main module contains only general external plugin code, including a list
of loaded external plugins, and associated start/stop functions for
plugin servers.

Testing
-------

This commit also implements the following improvements to tests:
- Restructure fixtures to accommodate new external plugin servers --
namely, targeting for now in the existing Python and Javascript
- Add new test cases for external plugins:
  * External plugin configuration: add test cases for current behavior;
    in particular:
    - Fail if no `query_cmd` is provided;
    - Warn if no `start_cmd` is provided - this is by design, as
      external plugins servers can be managed outside of Kong
  * Plugin server start / stop - for both Go and Python plugins
  * External plugin info querying for both Go and Python plugins
  * External plugin execution - for both Go and Python plugins

Internal flow
-------------

`.plugin_servers.init:` loads all external plugins, by calling .plugin_servers.process and `.plugin_servers.plugin`
  `.plugin_servers.process`: queries external plugins info with the command specified in `_query_cmd` proeprties
  `.plugin_servers.plugin`: with info obtained as described above, `.plugin:new` returns a kong-compatible representation
                          of an external plugin, with phase handlers, PRIORITY, and wrappers to the PDK. Calls
                          `.plugin_servers.rpc` to create an RPC through which Kong communicates with the plugin process
    `.plugin_servers.rpc`: based on info contained in the plugin (protocol field), creates the correct RPC for the
                          given external plugin
      `.plugin_servers.rpc.pb_rpc`: protobuf rpc implementation - used by Golang
      `.plugin_servers.rpc.mp.rpc`: messagepack rpc implementation - used by JS and Python
`.plugin_servers.init`: calls `.plugin_servers.process` to start external plugin servers
  `.plugin_servers.process`: optionally starts all external plugin servers (if a `_start_cmd` is found)
     uses the resty pipe API to manage the external plugin process
@gszr gszr force-pushed the chore/plugin-server-improvements branch from 748ebc3 to be40499 Compare October 15, 2024 21:25
@kevwilliams kevwilliams requested a review from flrgh October 25, 2024 17:43
@gszr gszr merged commit f88da7d into master Nov 13, 2024
26 checks passed
@gszr gszr deleted the chore/plugin-server-improvements branch November 13, 2024 20:58
@team-gateway-bot
Copy link
Collaborator

Cherry-pick failed for master, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally.

git remote add upstream https://github.com/kong/kong-ee
git fetch upstream master
git worktree add -d .worktree/cherry-pick-12858-to-master-to-upstream upstream/master
cd .worktree/cherry-pick-12858-to-master-to-upstream
git checkout -b cherry-pick-12858-to-master-to-upstream
ancref=$(git merge-base 9ae36382db4b9485f8889b8cd80d66a0885c86e7 be4049980e16448b609a00b6a1f7453013d57fcf)
git cherry-pick -x $ancref..be4049980e16448b609a00b6a1f7453013d57fcf

@github-actions github-actions bot added the incomplete-cherry-pick A cherry-pick was incomplete and needs manual intervention label Nov 13, 2024
@locao locao removed the incomplete-cherry-pick A cherry-pick was incomplete and needs manual intervention label Nov 18, 2024
lhanjian pushed a commit that referenced this pull request Dec 23, 2024
Context
-------

The overall goal of this commit is to refactor the external plugins
implementation, with the following goals in mind:
- Make plugin server code more approachable to unfamiliar engineers and
easier to evolve with confidence
- Harden configuration; ensure configuration defects are caught before
Kong is started
- Extend testing coverage

This is related to ongoing work on the Go PDK, with similar goals in
mind.

Summary
-------

This commit implements the following overall changes to plugin server
code:

- Move configuration related code into conf loader, so that configuration
loading and validation happens at startup time, rather than lazily, when
plugin data is loaded or pluginservers are started. Add tests for
current behavior.

- Move process-management code - for starting up plugin servers as well
as querying external plugins info - into the `process.lua` module.

- Introduce a `kong.runloop.plugin_servers.rpc` module that encapsulates
RPC initialization and protocol-specific implementations. This further
simplifies the main plugin server main module.

- Factor exposed API and phase handlers bridging code into a new `plugin`
module, which encapsulates an external plugin representation, including
the expected fields for any Kong plugin, plus external plugin-specific
bits, such as the RPC instance. Part of this external plugin-specific part
is the instance life cycle management. With this structure, the `kong.runloop.plugin_servers`
main module contains only general external plugin code, including a list
of loaded external plugins, and associated start/stop functions for
plugin servers.

Testing
-------

This commit also implements the following improvements to tests:
- Restructure fixtures to accommodate new external plugin servers --
namely, targeting for now in the existing Python and Javascript
- Add new test cases for external plugins:
  * External plugin configuration: add test cases for current behavior;
    in particular:
    - Fail if no `query_cmd` is provided;
    - Warn if no `start_cmd` is provided - this is by design, as
      external plugins servers can be managed outside of Kong
  * Plugin server start / stop - for both Go and Python plugins
  * External plugin info querying for both Go and Python plugins
  * External plugin execution - for both Go and Python plugins

Internal flow
-------------

`.plugin_servers.init:` loads all external plugins, by calling .plugin_servers.process and `.plugin_servers.plugin`
  `.plugin_servers.process`: queries external plugins info with the command specified in `_query_cmd` proeprties
  `.plugin_servers.plugin`: with info obtained as described above, `.plugin:new` returns a kong-compatible representation
                          of an external plugin, with phase handlers, PRIORITY, and wrappers to the PDK. Calls
                          `.plugin_servers.rpc` to create an RPC through which Kong communicates with the plugin process
    `.plugin_servers.rpc`: based on info contained in the plugin (protocol field), creates the correct RPC for the
                          given external plugin
      `.plugin_servers.rpc.pb_rpc`: protobuf rpc implementation - used by Golang
      `.plugin_servers.rpc.mp.rpc`: messagepack rpc implementation - used by JS and Python
`.plugin_servers.init`: calls `.plugin_servers.process` to start external plugin servers
  `.plugin_servers.process`: optionally starts all external plugin servers (if a `_start_cmd` is found)
     uses the resty pipe API to manage the external plugin process

(cherry picked from commit f88da7d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-pick kong-ee schedule this PR for cherry-picking to kong/kong-ee chore Not part of the core functionality of kong, but still needed core/configuration core/language/go core/language/js core/language/python core/proxy size/XXL skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants