Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jk/cumulus 3020 #3800

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open

Jk/cumulus 3020 #3800

wants to merge 7 commits into from

Conversation

Jkovarik
Copy link
Member

Summary: Summary of changes

Addresses CUMULUS-3020

PR Checklist

  • Update CHANGELOG
  • Unit tests
  • Ad-hoc testing - Deploy changes and test manually
  • Integration tests

Copy link
Contributor

@npauzenga npauzenga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @Jkovarik, just one nit about the description to make it painfully obvious what the unit is.

tf-modules/archive/variables.tf Outdated Show resolved Hide resolved
tf-modules/cumulus/variables.tf Outdated Show resolved Hide resolved
tf-modules/archive/sf_event_sqs_to_db_records.tf Outdated Show resolved Hide resolved
tf-modules/cumulus/archive.tf Outdated Show resolved Hide resolved
Copy link
Contributor

@npauzenga npauzenga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Jkovarik! Looks good!

@@ -626,3 +626,9 @@ variable "report_sns_topic_subscriber_arns" {
default = null
description = "Account ARNs to supply to report SNS topics policy with subscribe action"
}

variable "sf_event_sqs_lambda_timeout" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw this ticket in review for the sprint board, and didn't know it was already approved, but looking at it, instead of adding this variable like this, can we not just add the check for SfEventSqsLambda timeout like we do for https://github.com/nasa/cumulus/blob/master/tf-modules/archive/variables.tf#L219 here:

? seems easier, but this timeout looks a bit different than the others

Copy link
Member Author

@Jkovarik Jkovarik Oct 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nnaga1 that's more or less what @jennyhliu asked. I have strong reservations about if we should really lump internal lambdas into a generic user configurable timeout for framework AND workflow lambdas, given it's difficult to document how modifications to this could cause serious negative impacts and/or support obligations, but I'm willing to go with the team's opinion on it.

We definitely don't have documentation on this configuration, option other than a mention in 'task_configuration.md', which seems inadequate long-term if we've expanded the scope of this configuration value.

Apologies that discussion wasn't captured here, this ticket has just been on the back burner since the start of the sprint.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@npauzenga @Nnaga1 @jennyhliu take a look at e002598

I'll add a documentation ticket to cover the doc concern above prior to closing this ticket.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for missing the back half of this conversation. I thought we'd agreed to NOT include this as part of the lambda_timeouts. The sfevent lambda is a central framework component that (ideally) we don't want users configuring in this way if they don't have to. At least that's my impression. I'm not sure what the consequences could be if this value changes.

Having this config in a separate, explicit, mildly buried variable for only users that really really know what they're doing was appealing. Giving the keys over in an object like lambda_timeouts makes it easier to see and adjust, which I'm not sure is what we want?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tl;dr I kind of liked the initial approach and am a little concerned about this one but, like Jonathan, can be persuaded 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity: discussed this in a huddle on 4 October and agreed with the current PR + scoped work to address and a arch meeting topic to discuss the approach with the team.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants