TF_CLI_ARGS_plan="-out=tfplan.out" doesn't work

[suzuki-shunsuke]

Environment

$ terraform version
Terraform v1.2.9
on darwin_arm64
+ provider registry.terraform.io/hashicorp/null v3.1.1

$ tfmigrate --version
0.3.7

Overview

I would like to test the result of tfmigrate plan with Conftest. ref

e.g.

terraform plan -out tfplan.out
terraform show -json tfplan.out > tfplan.json
conftest test tfplan.json

To do it, we have to run Conftest against a plan file.
To create a plan file with tfmigrate, I passed the environment variable TF_CLI_ARGS_plan=-out=tfplan.out.

$ env TF_CLI_ARGS_plan="-out=tfplan.out" tfmigrate plan

2022/09/12 22:34:08 [INFO] AWS Auth provider used: "SharedCredentialsProvider"
2022/09/12 22:34:10 [INFO] [runner] unapplied migration files: [20220912214947_mv_fooo.hcl]
2022/09/12 22:34:10 [INFO] [runner] load migration file: tfmigrate/20220912214947_mv_fooo.hcl
2022/09/12 22:34:10 [INFO] [migrator] start state migrator plan
2022/09/12 22:34:10 [INFO] [migrator@.] terraform version: 1.2.9
2022/09/12 22:34:10 [INFO] [migrator@.] initialize work dir
2022/09/12 22:34:11 [INFO] [migrator@.] get the current remote state
2022/09/12 22:34:13 [INFO] [migrator@.] override backend to local
2022/09/12 22:34:13 [INFO] [executor@.] create an override file
2022/09/12 22:34:13 [INFO] [migrator@.] creating local workspace folder in: terraform.tfstate.d/default
2022/09/12 22:34:13 [INFO] [executor@.] switch backend to local
2022/09/12 22:34:13 [INFO] [migrator@.] compute a new state
2022/09/12 22:34:14 [INFO] [migrator@.] check diffs
2022/09/12 22:34:14 [INFO] [executor@.] remove the override file
2022/09/12 22:34:14 [INFO] [executor@.] remove the workspace state folder
2022/09/12 22:34:14 [INFO] [executor@.] switch back to remote
2022/09/12 22:34:16 [INFO] [migrator] state migrator plan success!

But unfortunately, a plan file isn't created.

I think TF_CLI_ARGS_plan should work. #27 (comment)

How to reproduce

resource "null_resource" "foo" {}

terraform {
  backend "s3" {
    bucket = "***"
    key    = "github/foo/v1/terraform.tfstate"
    region = "ap-northeast-1"
  }
}

Create a resource for migration.

$ terraform init
$ terraform apply

Change the resource address and create a migration file.

resource "null_resource" "fooo" {}

.tfmigrate.hcl

tfmigrate {
  migration_dir = "./tfmigrate"
  history {
    storage "s3" {
      bucket = "***"
      key    = "github/foo/history.json"
    }
  }
}

tfmigrate/20220912214947_mv_fooo.hcl

migration "state" "mv_foo" {
  actions = [
    "mv null_resource.foo null_resource.fooo",
  ]
}

Run tfmigrate plan.

$ env TF_CLI_ARGS_plan="-out=tfplan.out" tfmigrate plan

Expected Behaviour

tfplan.out is created.

Actual Behaviour

tfplan.out isn't created.

Of course, when we run terraform plan -out=tfplan.out, plan file is created properly.

$ terraform plan -out=tfplan.out
null_resource.foo: Refreshing state... [id=1351938300054449217]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  + create
  - destroy

Terraform will perform the following actions:

  # null_resource.foo will be destroyed
  # (because null_resource.foo is not in configuration)
  - resource "null_resource" "foo" {
      - id = "1351938300054449217" -> null
    }

  # null_resource.fooo will be created
  + resource "null_resource" "fooo" {
      + id = (known after apply)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: tfplan.out

To perform exactly these actions, run the following command to apply:
    terraform apply "tfplan.out"

Reference

• Run conftest on tfmigrate-plan suzuki-shunsuke/tfaction#511 (comment)

[suzuki-shunsuke]

Oh, I got it.

tfmigrate/tfexec/terraform_plan.go

Lines 26 to 43 in 6bd3c21

	// To return a plan file as a return value, we always use an -out option and load it to memory.
	// if the option exists just use it else create a temporary file.
	planOut := ""
	if hasPrefixOptions(opts, "-out=") {
	planOut = getOptionValue(opts, "-out=")
	} else {
	tmpPlan, err := os.CreateTemp("", "tfplan")
	if err != nil {
	return nil, fmt.Errorf("failed to create temporary plan file: %s", err)
	}
	planOut = tmpPlan.Name()
	defer os.Remove(planOut)
	if err := tmpPlan.Close(); err != nil {
	return nil, fmt.Errorf("failed to close temporary plan file: %s", err)
	}
	args = append(args, "-out="+planOut)
	}

[suzuki-shunsuke]

As a workaround, we can use tfmigrate plan's -out option.
But this option is deprecated. dba98a6

As you described in dba98a6 , the plan file can't be used for terraform apply, but this still can be used for validation using conftest or something.

ref. suzuki-shunsuke/tfaction#511 (comment)

[minamijoyo]

Hi @suzuki-shunsuke, Thank you for reporting this!

I’m investigating the terraform show command behavior across some versions, and it turns out that the old terraform before v1.1.3 rejects a plan file generated by tfmigrate plan —out=tfplan.out as stale.

$ tfmigrate plan tfmigrate_test.hcl --out=tfplan.out
$ terraform show -json tfplan.out

• Terraform v1.0.11: NG
• Terraform v1.1.0: NG
• Terraform v1.1.3: OK
• Terraform v1.2.9: OK
• Terraform v1.3.0-beta1: OK

It works after this change: hashicorp/terraform#30205.

I think this limitation is acceptable for me as long as it works with the latest Terraform version. If no other concerns are found, we could revert the deprecation.

[suzuki-shunsuke]

Thank you for your investigation!

it turns out that the old terraform before v1.1.3 rejects a plan file generated by tfmigrate plan —out=tfplan.out as stale.

Oh, I didn't know that.

I think this limitation is acceptable for me as long as it works with the latest Terraform version. If no other concerns are found, we could revert the deprecation.

I agree.

[minamijoyo]

@suzuki-shunsuke Finally reverted the deprecation in #108. Thank you for reporting this!

[suzuki-shunsuke]

Thank you for your quick update!