[Placeholder from PMC] Consider enabling "contrib" modules that don't burden core PBS repo

[scr-oath]

At the moment every adapter, module, analytics, MetricsEngine is compiled into the core repo. This means that any of the existing or newly added options burdens the main repo with its dependencies.

The proposal would be to have a separate repo for contrib where each contribution could have its own go.mod/sum to declare its own dependencies. Then, hosting companies, could pick and choose which pieces they wanted and configure into the system rather than needing everything always even if not used.

[bsardo]

There is some interest from the committee. The point was raised that we should also explore dynamically loading modules. @hhhjort indicates that a few years ago we weren't able to compile the modules independent of the executable so it might be worth exploring whether the libraries have been updated.

I believe it was previously discussed that @scr-oath would provide a proof of concept PR.

[scr-oath]

Yeah I had a good convo with Nils Lind (Assertive Yield) about that plug-ability option, possibly gRPC and an interesting existing proof is terraform "providers".

That said, what I had in mind; the POC if lifted from how we did things on our fork, would be more of treating PBS like a library (think http.Server itself) where all of the main method would be made public and configurable with the functional options pattern.

Both have interesting pros and cons, though I lean towards the latter because ultimately "bringing your own main" and being in the same process feels a whole lot simpler and still opens up all of the unique compilation or dependency choices without needing to modify a fork of the PBS repo. Upgrades can be as simple as upgrading the dependency, rather than resolving conflicts with modifications.

In any case, I still pledge to offer that example. I just haven't found the time. Maybe a meeting with demo and discussion would be fruitful. I'll bring this up at the next PMC, and sorry I couldn't join this mornings backlog review, hopefully will be able to attend in the future.

[scr-oath]

Yeah I had a good convo with Nils Lind (Assertive Yield) about that plug-ability option, possibly gRPC and an interesting existing proof is terraform "providers".

That said, what I had in mind; the POC if lifted from how we did things on our fork, would be more of treating PBS like a library (think http.Server itself) where all of the main method would be made public and configurable with the functional options pattern.

Both have interesting pros and cons, though I lean towards the latter because ultimately "bringing your own main" and being in the same process feels a whole lot simpler and still opens up all of the unique compilation or dependency choices without needing to modify a fork of the PBS repo. Upgrades can be as simple as upgrading the dependency, rather than resolving conflicts with modifications.

In any case, I still pledge to offer that example. I just haven't found the time. Maybe a meeting with demo and discussion would be fruitful. I'll bring this up at the next PMC, and sorry I couldn't join this mornings backlog review, hopefully will be able to attend in the future.

[jwrosewell]

Suggest looking to other projects for inspiration on approach. For example; NGINX.

They then have a certified scheme which incentivises testing and regulator updates to main. 51Degrees have added NGINX certification tests to the nightly pipelines.

[scr-oath]

The upshot of this idea:

1. Split repo into contrib module
2. Add Registry rather than hardcoded/generated
3. Register modules via code - could be init or could be via the functional option pattern if/when the server changes to a library.

[jwrosewell]

Are you thinking of using the plugin feature?