PostgreSQL 11 is EOL and out of standard support for RDS causing additional cost

[hexa00]

The PostgreSQL version used in the datastore module is currently 11 as seen in :

terraform-aws-metaflow/modules/datastore/variables.tf

Line 14 in ee7093c

default = "11"

However this version is now at EOL and costs extra to be supported by AWS (0.10$/h/VCPU) see:

https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-release-calendar.html
https://aws.amazon.com/rds/postgresql/pricing/

Could this be upgraded ?

[savingoyal]

sure! would you like to submit a PR?

[hexa00]

I could I'm just not sure what we'd need to watch out for for CI / testing of this... ?

If it's all automatic , I'm fine with a 1-3 liner for it that will change to the latest 11 (11.22) to 12.17
see: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_UpgradeDBInstance.PostgreSQL.html#USER_UpgradeDBInstance.PostgreSQL.MajorVersion

You can't just do 11 -> 12 .. needs to be 12.17 and we need to assume that people are on the latest minor for the upgrade to work without issue on aws's side (which is a resonable assumption I think given it upgrades minors automatically)

You also need to add allow_major_version_upgrade = true
And potentially apply_immediately = true

Is that ok ?

[hexa00]

Actually I'm going to keep allow_major & apply to false and add that as a var for the user

[hexa00]

See: #97

[mszepieniec]

I'm hitting this issue as well, in fact the terraform apply no longer works:

│ Error: creating RDS DB Instance (metaflowmetaflowkrut0myf): operation error RDS: CreateDBInstance, https response error StatusCode: 400, RequestID: 469e6a44-04c0-49ea-8b0a-d12a8b323ad1, api error InvalidParameterCombination: RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.small, Engine=postgres, EngineVersion=11.22, LicenseModel=postgresql-license. For supported combinations of instance class and database engine version, see the documentation.
│
│   with module.metaflow-datastore.aws_db_instance.this[0],
│   on .terraform/modules/metaflow-datastore/modules/datastore/rds.tf line 98, in resource "aws_db_instance" "this":
│   98: resource "aws_db_instance" "this" {
│
╵
╷

Is this repo still the recommended way to set up metaflow?