Skip to content

Commit

Permalink
Add an example of Composer 3 configuration with network attachment (#…
Browse files Browse the repository at this point in the history
…11288) (#18966)

[upstream:6a9d8c19ec6af03bbf6704ddce35aed5e3c21b3a]

Signed-off-by: Modular Magician <[email protected]>
  • Loading branch information
modular-magician authored Aug 1, 2024
1 parent 22ca227 commit f53032d
Show file tree
Hide file tree
Showing 2 changed files with 69 additions and 5 deletions.
3 changes: 3 additions & 0 deletions .changelog/11288.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:none

```
71 changes: 66 additions & 5 deletions website/docs/r/composer_environment.html.markdown
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,11 @@ To get more information about Environments, see:
* [Connect an environment to a VPC network](https://cloud.google.com/composer/docs/composer-3/connect-vpc-network)
* [Apache Airflow Documentation](http://airflow.apache.org/)

<Note>
-> **Note**
Cloud Composer 1 is in the post-maintenance mode. Google does
not release any further updates to Cloud Composer 1, including new versions
of Airflow, bugfixes, and security updates. We recommend using
Cloud Composer 2 or Cloud Composer 3 instead.
</Note>

Several special considerations apply to managing Cloud Composer environments
with Terraform:
Expand Down Expand Up @@ -92,18 +91,16 @@ resource "google_composer_environment" "test" {
}
```


### With GKE and Compute Resource Dependencies

<Note>
-> **Note**
To use custom service accounts, you must give at least the
`role/composer.worker` role to the service account of the Cloud Composer
environment. For more information, see the
[Access Control](https://cloud.google.com/composer/docs/how-to/access-control)
page in the Cloud Composer documentation.
You might need to assign additional roles depending on specific workflows
that the Airflow DAGs will be running.
<Note>

#### GKE and Compute Resource Dependencies (Cloud Composer 3)

Expand Down Expand Up @@ -302,6 +299,69 @@ resource "google_project_iam_member" "composer-worker" {
}
```

### Cloud Composer 3 networking configuration

In Cloud Composer 3, networking configuration is simplified compared to
previous versions. You don't need to specify network ranges, and can attach
custom VPC networks to your environment.

-> **Note**
It's not possible to detach a VPC network using Terraform. Instead, you can
attach a different VPC network in its place, or detach the network using
other tools like Google Cloud CLI.

Use Private IP networking:

```hcl
resource "google_composer_environment" "example" {
name = "example-environment"
region = "us-central1"
config {
enable_private_ip_environment = true
# ... other configuration parameters
}
}
```

Attach a custom VPC network (Cloud Composer creates a new network attachment):

```hcl
resource "google_composer_environment" "example" {
name = "example-environment"
region = "us-central1"
config {
node_config {
network = "projects/example-project/global/networks/example-network"
subnetwork = "projects/example-project/regions/us-central1/subnetworks/example-subnetwork"
}
# ... other configuration parameters
}
}
```

Attach a custom VPC network (use existing network attachment):

```hcl
resource "google_composer_environment" "example" {
name = "example-environment"
region = "us-central1"
config {
node_config {
composer_network_attachment = projects/example-project/regions/us-central1/networkAttachments/example-network-attachment
}
# ... other configuration parameters
}
}
```

### With Software (Airflow) Config

Expand All @@ -327,6 +387,7 @@ resource "google_composer_environment" "test" {
}
}
```

## Argument Reference - Cloud Composer 1

The following arguments are supported:
Expand Down

0 comments on commit f53032d

Please sign in to comment.