SDK Monorepo vs Separate Repo for Plugin SDK

[aaronsteers]

New discussion here for how we want to manage the upcoming Meltano Plugin SDK.

If we add here in an "SDK Monorepo" approach, we may have to solve some challenges around how to manage CI and releases when multiple packages are being managed.

If we create a new repo, we bifurcate dev efforts and possibly slow down iterations.

Assumptions:

1. I think we still want singer-sdk and meltano-plugin-sdk to be distinct packages when published to PyPi. (Prevents bloat.)
2. We've discussed bifurcating tap-sdk and target-sdk previously, and presumably if we solve the monorepo challenges when adding meltano-plugin-sdk, we'd also be in a good place (if we choose to) to have separate importable libraries for taps and targets down the road.
3. Still TBD if we want https://sdk.meltano.com to be the home for Meltano Plugins as well as Singer Taps and Targets, versus having separate docs sites for each.

Timing:

1. Our decision here doesn't have to be immediately implemented. We can decide that we want the repos separate or joined, but for now take a more pragmatic approach to get first deliverables and first round of POCs complete.

[edgarrmondragon]

If we go with a monorepo for the future Plugin SDK, i.m.o. it makes more sense for it to live in meltano/meltano since that library would ostensibly reuse parts of the Meltano core codebase, and probably need to be tested side-by-side.

[aaronsteers]

@edgarrmondragon - Those are really good points and I could see that working well.

If we do go that direction, I think I would advocate that we rename meltano/sdk repo to meltano/singer-sdk to avoid confusion, and anyway this would then matches the PyPi package name singer-sdk. (I don't think we should to combine the singer-sdk into meltano/meltano because we want to keep that repo separate and dedicated to the wider Singer community.)

Then also, we may need to repurpose sdk.meltano.com as a marketing/landing page. To resolve ambiguity, sdk.meltano.com would likely lead users to something like a choice between singer-sdk.meltano.com or plugin-sdk.meltano.com or maybe better yet: meltano.com/singer-sdk and meltano.com/plugin-sdk

We don't have to make final decisions on the web properties naming/pathing right now, but yes, I like this idea to put the new plugin SDK inside meltano/meltano and then resolve new ambiguities and naming collisions as needed once we have two "Meltano SDKs" 😄