astronomer · yanmastin-astro · Oct 18, 2024 · Oct 18, 2024 · Oct 21, 2024 · Oct 21, 2024
@@ -0,0 +1,223 @@
+---
+page_title: "Get started with Astro Terraform Provider"
+---
+
+# Get started with Astro Terraform Provider
+In this guide, you will automate the onboarding of a new team onto Astro by creating and managing a Workspace and Deployment. By the end of this tutorial, you will have a fully automated setup that is reproducible and easily scalable to more teams.
+
+## Prerequisites
+- An Astro Organization
+- [Terraform](https://developer.hashicorp.com/terraform/install)
+
+
+## Step 1: Create Your Terraform Working Directory
+1. Create a folder, `my-data-platform` for your Terraform project.
+2. Save the following code in a file named `terraform.tf`:
+```
+terraform {
+  required_providers {
+    astro = {
+      source  = "astronomer/astro"
+      version = "1.0.0"
+    }
+  }
+}
+
+provider "astro" {
+  organization_id = &lt;your-organization-id&gt;
+}
+```
+3. Edit the `terraform.tf` file with your Organization's ID for `<your-organization-id>`. The working directory contains all your Terraform code, and all Terraform commands run from this directory.
+
+## Step 2: Initialize the Terraform Working Directory
+1. Run `terraform init`. You will see Terraform downloading and installing the Astro Terraform provider locally:
+```
+$ terraform init
+
+Initializing the backend...
+
+Initializing provider plugins...
+- Finding astronomer/astro versions matching "1.0.0"...
+- Installing astronomer/astro v1.0.0...
+- Installed astronomer/astro v1.0.0 (signed by a HashiCorp partner, key ID F5206453FDEA33CF)
+
+...
+
+Terraform has been successfully initialized!
+
+You may now begin working with Terraform. Try running "terraform plan" to see
+any changes that are required for your infrastructure. All Terraform commands
+should now work.
+```
+2. The versions and hashes of providers are stored in a generated file `.terraform.lock.hcl`.
+
+-> Astronomer recommends you store `.terraform.lock.hcl` in your version control system so that you can track potential changes to your external dependencies.
+
+## Step 3: Authenticate with Astro
+1. [Create an API token](https://www.astronomer.io/docs/astro/automation-authentication#step-1-create-an-api-token) in Astro. Since you are creating a Workspace, you need an [Organization API token](https://www.astronomer.io/docs/astro/organization-api-tokens) with [Organization Owner permissions](https://www.astronomer.io/docs/astro/user-permissions#organization-roles).
+2. Configure the API token as an environment variable `ASTRO_API_TOKEN` to run Terraform commands:
+`export ASTRO_API_TOKEN=<your-api-token>`
+
+Alternatively, users can set their API token value in the provider block:
+```
+provider "astro" {
+  organization_id = "&lt;your-organization-id&gt;"
+  token = "&lt;your-api-token&gt;"
+}
+```
+
+## Step 4: Define Resources in Terraform
+In a file `main.tf`, define two resources, an `astro_workspace` and an `astro_deployment`. These resources will represent an Astro workspace and Astro Deployment, defined in Terraform code:
+```
+# Create a new workspace
+resource "astro_workspace" "my_first_tf_workspace" {
+  name                  = "My first TF workspace"
+  description           = "My first Terraform-created workspace"
+  cicd_enforced_default = false
+}
+
+# Create a new Deployment
+resource "astro_deployment" "my_first_tf_deployment" {
+  name                    = "My first TF deployment"
+  description             = "My first Terraform-created deployment"
+  type                    = "STANDARD"
+  cloud_provider          = "AWS"
+  region                  = "us-east-1"
+  contact_emails          = []
+  default_task_pod_cpu    = "0.25"
+  default_task_pod_memory = "0.5Gi"
+  executor                = "CELERY"
+  is_cicd_enforced        = true
+  is_dag_deploy_enabled   = true
+  is_development_mode     = false
+  is_high_availability    = false
+  resource_quota_cpu      = "10"
+  resource_quota_memory   = "20Gi"
+  scheduler_size          = "SMALL"
+  workspace_id            = astro_workspace.my_first_tf_workspace.id
+  environment_variables   = []
+  worker_queues = [{
+    name               = "default"
+    is_default         = true
+    astro_machine      = "A5"
+    max_worker_count   = 10
+    min_worker_count   = 0
+    worker_concurrency = 1
+  }]
+}
+```
+-> One of the key characteristics (and benefits) of using Terraform is that it's *declarative*. For example, `workspace_id = astro_workspace.my_first_tf_workspace.id` tells Terraform to configure the Workspace ID in the Deployment. This means the Workspace must be created first, producing an ID which is a generated value and unknown at the time of writing. You don't have to instruct Terraform to create resources in a certain order, you only have to instruct what to create. The resources above can be defined in any order. Terraform takes the relationships between resources into account when deciding the order of creating resources.
+
+## Step 5: (Optional) Define Outputs
+In a file called `outputs.tf`, define values you want to log after creating the infrastructure. The following code configures Workspace and Deployment IDs as the output:
+```
+output "terraform_workspace" {
+  description = "ID of the TF created workspace"
+  value       = astro_workspace.my_first_tf_workspace.id
+}
+
+output "terraform_deployment" {
+  description = "ID of the TF created deployment"
+  value       = astro_deployment.my_first_tf_deployment.id
+}
+```
+
+## Step 6: Preview the Terraform changes
+You now have up to 3 files: `terraform.tf`, `main.tf` and the optional `outputs.tf`.
+1. Run [`terraform plan`](https://developer.hashicorp.com/terraform/cli/commands/plan) to let Terraform create an execution plan and preview the infrastructure changes that Terraform will make. You should see the following text:
+```
+$ terraform plan
+
+Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+  + create
+
+Terraform will perform the following actions:
+
+  # astro_deployment.my_first_tf_deployment will be created
+  + resource "astro_deployment" "my_first_tf_deployment" {
+      ...
+    }
+
+  # astro_workspace.my_first_tf_workspace will be created
+  + resource "astro_workspace" "my_first_tf_workspace" {
+      ...
+    }
+
+Plan: 2 to add, 0 to change, 0 to destroy.
+
+Changes to Outputs:
+  + terraform_deployment = (known after apply)
+  + terraform_workspace  = (known after apply)
+  ```
+2. Verify the generated plan contains the text `Plan: 2 to add, 0 to change, 0 to destroy.` This validates that the plan is to create two resources, which are the Workspace and Deployment as defined in `main.tf`.
+
+## Step 7: Apply the Terraform Plan
+Run `terraform apply` and select `yes` to execute the plan. This creates the Astro resources and will print their ids, as you defined in `outputs.tf`: 
+```
+$ terraform apply
+
+Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+  + create
+
+Terraform will perform the following actions:
+
+...
+
+Do you want to perform these actions?
+  Terraform will perform the actions described above.
+  Only 'yes' will be accepted to approve.
+
+  Enter a value: yes
+
+astro_workspace.my_first_tf_workspace: Creating...
+astro_workspace.my_first_tf_workspace: Creation complete after 0s [id=&lt;workspace-id&gt]
+astro_deployment.my_first_tf_deployment: Creating...
+astro_deployment.my_first_tf_deployment: Creation complete after 1s [id=&lt;deployment-id&gt]
+
+Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
+
+Outputs:
+
+terraform_deployment = "&lt;deployment-id&gt"
+terraform_workspace = "&lt;workspace-id&gt"
+```
+The resources were created and will now be visible in Astro.
+
+## Step 8: Clean Up Terraform-Created Resources
+Run `terraform destroy` and select `yes`:
+```
+$ terraform destroy
+astro_workspace.my_first_tf_workspace: Refreshing state...
+  [id=&lt;workspace-id&gt]
+astro_deployment.my_first_tf_deployment: Refreshing state...
+  [id=&lt;deployment-id&gt]
+
+Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+  - destroy
+
+Terraform will perform the following actions:
+
+  ...
+
+Plan: 0 to add, 0 to change, 2 to destroy.
+
+Changes to Outputs:
+  - terraform_deployment = "&lt;deployment-id&gt" -> null
+  - terraform_workspace  = "&lt;workspace-id&gt" -> null
+
+Do you really want to destroy all resources?
+  Terraform will destroy all your managed infrastructure, as shown above.
+  There is no undo. Only 'yes' will be accepted to confirm.
+
+  Enter a value: yes
+
+astro_deployment.my_first_tf_deployment: Destroying...
+  [id=&lt;deployment-id&gt]
+astro_deployment.my_first_tf_deployment: Destruction complete after 1s
+astro_workspace.my_first_tf_workspace: Destroying...
+  [id=&lt;workspace-id&gt]
+astro_workspace.my_first_tf_workspace: Destruction complete after 0s
+
+Destroy complete! Resources: 2 destroyed.
+```
+The output shows two destroyed resources which are the Workspace and Deployment that you first created.
@@ -0,0 +1,132 @@
+---
+page_title: "Use Terraform Import Script to migrate existing resources"
+---
+
+# Use Import Script to migrate existing resources
+The Astro Terraform Import Script is a tool designed to help you import existing Astro resources into your Terraform configuration.
+
+In this guide, we will migrate an existing Workspace, API token and Team into Terraform using the Terraform Import Script.
+
+## Import Script options
+- `-resources`: Comma-separated list of resources to import. Accepted values are 
+`workspace`, `deployment`, `cluster`, `api_token`, `team`, `team_roles`,and `user_roles`. If not provided, all resources are imported.
+
+-> Ensure you have the necessary permissions in your Astro organization to access the resources you're attempting to import.
+
+- `-token`: API token to authenticate with the Astro platform. If not provided, the script will attempt to use the `ASTRO_API_TOKEN` environment variable.
+- `-organizationId`: Organization ID to import resources from.
+- `-runTerraformInit`: Run terraform init after generating the import configuration. Used for initializing the Terraform state in our GitHub Actions.
+- `-help`: Display help information.
+
+
+## Prerequisites
+- An [Astro](https://www.astronomer.io/product/) organization with a Workspace, Team, and API token
+- An initialized Terraform working directory
+
+## Step 1: Download the Import Script
+1. Download the `terraform-provider-astro-import-script` executable file from the [Astro Terraform Provider releases](https://github.com/astronomer/terraform-provider-astro/releases) based on your OS and architecture. For this guide, the script will be `terraform-provider-astro-import-script_v0.1.3_darwin_arm64`.
+
+## Step 2: Run the Import Script
+
+-> Make sure you run `terraform init` before using the Import Script, or use the `-runTerraformInit` option when running the Import Script.
+
+1. Authenticate with Astro by creating an [API token](https://www.astronomer.io/docs/astro/organization-api-tokens#create-an-organization-api-token) with the **Organization owner** role and configure it as an `ASTRO_API_TOKEN` environment variable:
+```
+export ASTRO_API_TOKEN=&lt;your-api-token&gt;
+```
+
+2. If you are using a Unix-based systems, add execute permission to the script file: 
+```
+chmod +x terraform-provider-astro-import-script_&lt;version-number&gt;_&lt;os&gt;_&lt;arc&gt;
+```
+3. Run the Import Script. Insert the script's version, your computer's operating system, and your computer's architecture for `<version-number>`, `<os>` and `<arc>`.
+
+- On Unix-based systems:
+```
+./terraform-provider-astro-import-script_&lt;version-number&gt;_&lt;os&gt;_&lt;arc&gt; [options]
+```
+- On Windows:
+
+```
+.\terraform-provider-astro-import-script_&lt;version-number&gt;_&lt;os&gt;_&lt;arc&gt;.exe [options]
+```
+
+To import your existing Workspace, API token and Team, specify those resources with the `-resources` option. The other option you need to specify is `-organizationId`:
+```
+./terraform-provider-astro-import-script_v0.1.3_darwin_arm64 -organizationId &lt;your-organization-id&gt; -resources api_token,team,workspace
+```
+
+You should see the following output:
+```
+Terraform Import Script Starting
+Resources to import:  [api_token team workspace]
+Using organization ID: &lt;your-organization-id&gt
+Terraform version 1.9.7 is installed and meets the minimum required version.
+Importing teams for organization &lt;your-organization-id&gt
+Importing API tokens for organization &lt;your-organization-id&gt
+Importing workspaces for organization &lt;your-organization-id&gt
+Importing Workspaces: [&lt;workspace-id&gt]
+Successfully handled resource workspace
+Importing API Tokens: [&lt;api_token-id&gt]
+Successfully handled resource api_token
+Importing Teams: [&lt;team-id&gt]
+Successfully handled resource team
+Successfully wrote import configuration to import.tf
+Successfully deleted generated.tf
+terraform.tfstate does not exist, no need to delete
+astro_team.team_&lt;team-id&gt: Preparing import... [id=&lt;team-id&gt]
+astro_api_token.api_token_&lt;api_token-id&gt: Preparing import... [id=&lt;api_token-id&gt]
+astro_workspace.workspace_&lt;workspace-id&gt: Preparing import... [id=&lt;workspace-id&gt]
+astro_workspace.workspace_&lt;workspace-id&gt: Refreshing state... [id=&lt;workspace-id&gt]
+astro_api_token.api_token_&lt;api_token-id&gt: Refreshing state... [id=&lt;api_token-id&gt]
+astro_team.team_&lt;team-id&gt: Refreshing state... [id=&lt;team-id&gt]
+
+Terraform will perform the following actions:
+
+...
+
+Plan: 3 to import, 0 to add, 0 to change, 0 to destroy.
+
+Terraform has generated configuration and written it to generated.tf. Please review the configuration and edit it as
+necessary before adding it to version control.
+
+Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you
+run "terraform apply" now.
+Import process completed. Summary:
+Resource workspace processed successfully
+Resource api_token processed successfully
+Resource team processed successfully
+```
+-> If you import Deployments, they will not count torwards the `Plan: 3 to import, 0 to add, 0 to change, 0 to destroy` line of the output even when the Deployments are successfully imported. This is a known issue and is in the process of being fixed.
+
+## Step 3: Review output
+The script will generate two main files:
+- `import.tf`: Contains the Terraform import blocks for the specified resources.
+- `generated.tf`: Contains the Terraform resource configurations for the imported resources.
+The generated Terraform configurations might require some manual adjustment to match your specific requirements or to resolve any conflicts.
+
+## Step 4: Extract and organize resources
+The `generated.tf` file created by the Import Script contains all of the specified resources in one file. Astronomer recommends that you extract and modularize the resources so they are easily maintained and reusable. The following example shows a well structured Terraform project for managing Astro infrastructure:
+```
+terraform-astro-project/
+├── environments/
+│   ├── dev/
+│   │   ├── main.tf              # Root module for development
+│   │   ├── variables.tf         # Dev-specific variables
+│   │   ├── outputs.tf           # Dev-specific outputs
+│   │   └── dev.tfvars           # Variable values for development
+│   ├── prod/
+│   │   ├── main.tf              # Root module for production
+│   │   ├── variables.tf         # Prod-specific variables
+│   │   ├── outputs.tf           # Prod-specific outputs
+│   │   └── prod.tfvars          # Variable values for production
+├── modules/
+│   ├── astro/
+│   │   ├── main.tf              # Entry point for the module
+│   │   ├── workspace.tf         # Defines Astro workspaces
+│   │   ├── deployment.tf        # Defines Astro Deployments
+│   │   ├── users.tf             # Defines user roles and access
+│   │   ├── variables.tf         # Variables used in the astro module
+│   │   └── outputs.tf           # Outputs from the astro module
+└── cloud_provider.tf     
+```
@@ -1,17 +1,21 @@
 ---
-# generated by https://github.com/hashicorp/terraform-plugin-docs
-page_title: "astro Provider"
-subcategory: ""
-description: |-
-
+page_title: "Astro Provider"
+description: 
+Astro Provider for Terraform allows you to manage Astro resources from a Terraform script. The Provider uses the Astro API.
 ---
 
-# astro Provider
+# Astro Provider
+[Astro](https://www.astronomer.io/product/) is a unified data platform built on Apache Airflow® that ensures data is delivered on time, securely, and accurately.
+This is the official [Astro Terraform Provider](https://github.com/astronomer/terraform-provider-astro), built to allow you to automate, scale, and manage your Astro infrastructure.
+Reduce manual effort, minimize errors, and create a more streamlined approach to managing data orchestration.
 
+## Authentication
+Create an [API token](https://www.astronomer.io/docs/astro/automation-authentication#step-1-create-an-api-token) with the minimum required [permissions](https://www.astronomer.io/docs/astro/user-permissions) to securely use Terraform, ensuring it only has access to the resources necessary.
+You can create a [Deployment](https://www.astronomer.io/docs/astro/deployment-api-tokens), [Workspace](https://www.astronomer.io/docs/astro/workspace-api-tokens), or [Organizaton](https://www.astronomer.io/docs/astro/organization-api-tokens) API token. 
+An Organizaton token is the most flexible option to authenticate for high level changes accross multiple different resources.
+Ensure the API token is configured as an environment variable, `ASTRO_API_TOKEN` when running Terraform commands.
 
-
-## Example Usage
-
+## Example usage
 ```terraform
 provider "astro" {
   organization_id = "cljzz64cc001n01mln1p12345"
@@ -28,4 +32,4 @@ provider "astro" {
 ### Optional
 
 - `host` (String) API host to use for the provider. Default is `https://api.astronomer.io`
-- `token` (String, Sensitive) Astro API Token. Can be set with an `ASTRO_API_TOKEN` env var.
+- `token` (String, Sensitive) Astro API Token. Can be set with an `ASTRO_API_TOKEN` env var.