Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Think about whether and how to add support for multiple logs within the same deployment in Terraform #267

Open
AlCutter opened this issue Sep 25, 2024 · 0 comments

Comments

@AlCutter
Copy link
Collaborator

(I'm talking about this in the context of GCP deployments here, but it probably applies to at least other Cloudy infras and perhaps MySQL too).

Currently, we have a terraform module for creating the services required to deploy a log built with Tessera on GCP, and then other tf modules which build on that to add instances of logs too (e.g. conformance testing).

Given that e.g. a Spanner instance costs money (since you have to spring for at least 100 PU), it would be nice to be able to bin-pack a bunch of logs together and have them share resources. Examples where this may be useful are:

  1. a number of low-throughput logs, where each uses << 100 PU for sequencing
  2. a time sharded log (e.g. CT) where successive shards might share the spanner instance since load will mostly shift towards subsequent shards over time, but some number of recent-ish shards may remain "open" to writes to collect stragglers.

One way to accomplish this would be to have another tf module, so we'd have:

  1. an environment module which creates Tessera-specific resources which can be shared (so Spanner, perhaps service account, KMS, etc.) under some name e.g. prod-eu1
  2. an instance module which creates the Tessera-specific per-log resources (e.g., Spanner DB, Bucket, Key, etc.) under some log name (e.g. argon2024) and within a specific environment (e.g. `prod-eu1).

Module (1) would be used manually as a "one-off" operation to create an env, and consumers of Tessera might then have their own terraform config which builds on (2) and deploys their binary configured to point at the appropriate log-specific resources within that env.

Arranging like this would also allow operators to safely turn-down and delete specific logs via terraform, leaving the environment and any other logs it contains untouched.

@transparency-dev transparency-dev deleted a comment from ayazjam Oct 14, 2024
@AlCutter AlCutter assigned AlCutter and unassigned AlCutter Nov 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant