WebAPI 3.0 Change List

[chrisknoll]

Starting this thread to capture WebAPI 3.0 proposed changes New suggestions can be raised in comments and those that make sense can be promoted up to the top thread. This list is not exhaustive but is geared towards the changes that would alter existing functionality in a backwards-breaking change (hence the 3.0 version bump).

Major Version Changes

1. Update JDK to JDK 17+
2. Security is no longer optional
3. Permission Rewrite
4. Update Java Libraries to Latest Versions
5. Caching Implementation
6. Analysis Execution Pipeline via Strategus (including new Analysis Designs deprecating old)
7. Refresh Flyway Migrations: Remove old non-supported Flyway Migrations
8. Maven Dependency Management
9. Remove outdated endpoints that don't have function.
10. Make necessary changes to support multi-homed (ie: load-balanced) configuration
11. Leverage websockets to faciliate server-side changes (ie: polling for cohort generation status)
12. Integrated runtime statistics (time spent in queries, time spent processing results, cache hits/misses) so we can understand runtime dynamics.
13. Move away from storing execution status in job table.

Notes

This section captures notes about items in the above list, as well as reasons why something is not in the list.

1. Module System: There has been discussion about making WebAPI more modular to support customization. While this is something that could be discussed as a 3.x feature, it's not something that currently exists in 2.x. This Issue is focusing on current functionality that changes in a backwards-breaking way. A better approach implementing modules (or miicro services) would be to start a new project with that architecture in mind).

[chrisknoll]

I should also add that the OP list is not the list of changes that must go into next release, just the list of things that would 'break' 2.x compatibility. There was an issue raised that if we didn't do all of these for 3.0, that the ones introduced later would be breaking changes spawning a 4.x release, but we can decide how major we want the 3.x line of changes to be and if a 4.x is required later, then we will live with that decision.

[anthonysena]

Noting that we may want to consider JHipster per @ecruz165 : https://www.jhipster.tech/

[qcaas-nhs-sjt]

I would like to suggest that it might make some sense to split up the WebAPI java into multiple component libraries based on subject areas rather than one big one, each library can then be versioned and tested in isolation before being integrated into the overarching project. This should reduce build times and should have other benefits.

[rajkumardas01]

We should approach microservice based architecture to simplify the large functionality of WebAPI. We can pick small small functionality and create the corresponding microservices. Replace current WebAPI when the services are ready.

Please let me know your thoughts on it.

[chrisknoll]

We should approach microservice based architecture to simplify the large functionality of WebAPI. We can pick small small functionality and create the corresponding microservices. Replace current WebAPI when the services are ready.

Please let me know your thoughts on it.

I'd have to see an example of this approach put into practice. Microservices aren't a free lunch, and in large enterprise applications like netflix, doordash, uber, etc, it might make sense, but the functionality of WebAPI is narrowly scoped (and should be thought of as infinite possibilities which microservices leans towards).

Here's a recent conversation about the implementation of microservices:
https://www.youtube.com/watch?v=LcJKxPXYudE

Before thinking about any major redesign of architecture, I think those involved should have a clear understanding of what the feature set of WebAPI is and then how they should be encapsulated. I think we currently have a pretty good package-encapsulation of the different functions in WebAPI, but I think if we did something like a refactor of those encapsulated functions into 'submodules' would improve our build times (I believe currently when you build WebAPI you are going to rebuild all the classes involved...and if we had a sub-module structure where only the changed module needs to be recompiled, it could greatly reduce our build times (which currently can be minutes, which is very annoying from a developer experience).

But throwing a microservices layer will incur overhead that we don't necessarily benefit from. I'd have to see the cost-benefit.

For example:
Benefit: Microservices can be isolated and deployed independently of the rest of the services that are running.
Cost: all microservices are now Remote Procedure Calls which will incur a RPC cost, and debugging will be handled through some remote process debug protocol.

Just looking at this one example, I'm not sure the benefit outweighs the cost. But there may be a number of other cost-benefit examples.

Perhaps if we took the step of better encapsulating functions into modules it can put us on the path of isolating those 'integrated modules' into external micro services.