Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add terraform provider template #41

Closed
wants to merge 34 commits into from
Closed
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
92f0aae
Refactor template processing in generator
Mar 15, 2024
8fd2a2c
Add option for creating terraform provider alongside gosdk
Mar 29, 2024
5071a16
Add static files for terraform provider
Mar 29, 2024
e7e3648
Generate exclusive file feature
Mar 29, 2024
0788bc3
Copy assets to the correct directories
Mar 29, 2024
a6e3205
add test files for further template
pimielowski Apr 2, 2024
89e075a
Merge branch 'refs/heads/main' into feat-add-terraform-provider-template
Apr 4, 2024
23af4d9
Distinguish which type of run CodeGen is running
Apr 4, 2024
5987571
Add Makefile and update go.sum dependencies
Apr 4, 2024
5ff1321
Fix CI
Apr 4, 2024
7f4c5f5
Fix normalized_test.go tests
Apr 4, 2024
be7c72b
Refactor imports and implement Terraform provider template
pimielowski Apr 8, 2024
a4d039f
add go.sum for assets
pimielowski Apr 8, 2024
5605f33
fix ci/cd
pimielowski Apr 8, 2024
ff1c464
add test go file
pimielowski Apr 8, 2024
386bfee
Update configuration and add scripts for the Terraform provider
pimielowski Apr 8, 2024
dcde5ca
rename test file
pimielowski Apr 8, 2024
e8bf556
fix cicd
pimielowski Apr 8, 2024
7511854
Update package paths in config and move files to SDK folder
pimielowski Apr 8, 2024
1c12558
add license to the sdk assets
pimielowski Apr 8, 2024
0548e23
Refactor and update tests for improved code
pimielowski Apr 8, 2024
9c54dd2
Improve template file filter in generator
pimielowski Apr 8, 2024
7d0da05
Update generator and terraform template, fix logging and remove debug…
pimielowski Apr 9, 2024
8f7889e
Remove spec params from provider.yaml
pimielowski Apr 9, 2024
d5fc6d4
Remove unnecessary comments in code
pimielowski Apr 10, 2024
f581520
Merge branch 'refs/heads/main' into feat-add-terraform-provider-template
pimielowski Apr 10, 2024
f4aba9d
Update RenderImports to accept multiple template types
pimielowski Apr 10, 2024
dc9bd3d
Refactor CreateResourceList function in terraform/provider.go
pimielowski Apr 10, 2024
477d1ab
Update Terraform Provider code generation and release workflow
pimielowski Apr 10, 2024
a38d4ff
Remove tf_provider_scripts from codegen config
pimielowski Apr 10, 2024
af2f5d5
Remove exclusive field and add custom templates support
pimielowski Apr 10, 2024
70e3a35
Simplify codegen test and modify error handling
pimielowski Apr 10, 2024
a0ff119
Merge branch 'refs/heads/main' into feat-add-terraform-provider-template
pimielowski Apr 12, 2024
5801ce6
Remove redundant code files in the tf_provider directory
pimielowski Apr 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
all: test_go build test_sdk

build:
go run ./cmd/codegen/main.go

test_sdk:
cd ../generated/pango && \
go run ./example/main.go

clean:
rm -rf ../generated

test_go:
go test -v ./...
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
373 changes: 373 additions & 0 deletions assets/sdk/sdk/LICENSE

Large diffs are not rendered by default.

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
42 changes: 42 additions & 0 deletions assets/tf_provider/.github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Build

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

jobs:

build:
name: Build
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
go-version: [ 1.17, 1.18 ]
pimielowski marked this conversation as resolved.
Show resolved Hide resolved
pimielowski marked this conversation as resolved.
Show resolved Hide resolved

steps:

- name: Set up Go 1.x
uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v2

- name: Get dependencies
run: |
go get -v -t -d ./...
if [ -f Gopkg.toml ]; then
curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
dep ensure
fi

- name: Build
run: go build -v .

- name: Test
run: go test -v .
42 changes: 42 additions & 0 deletions assets/tf_provider/.github/workflows/release.yml
pimielowski marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# This GitHub action can publish assets for release when a tag is created.
# Currently its setup to run on any tag that matches the pattern "v*" (ie. v0.1.0).
#
# This uses an action (paultyng/ghaction-import-gpg) that assumes you set your
# private key in the `GPG_PRIVATE_KEY` secret and passphrase in the `PASSPHRASE`
# secret. If you would rather own your own GPG handling, please fork this action
# or use an alternative one for key handling.
#
# You will need to pass the `--batch` flag to `gpg` in your signing step
# in `goreleaser` to indicate this is being used in a non-interactive mode.
#
name: release
on:
push:
tags:
- 'v*'
jobs:
goreleaser:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Unshallow
run: git fetch --prune --unshallow
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: '1.17'
- name: Import GPG key
id: import_gpg
uses: paultyng/[email protected]
env:
GPG_PRIVATE_KEY: ${{ secrets.GPG_PRIVATE_KEY }}
PASSPHRASE: ${{ secrets.PASSPHRASE }}
- name: Run GoReleaser
uses: goreleaser/[email protected]
with:
version: latest
args: release --rm-dist
env:
GPG_FINGERPRINT: ${{ steps.import_gpg.outputs.fingerprint }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
58 changes: 58 additions & 0 deletions assets/tf_provider/docs/data-sources/tfid.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "panos_tfid Data Source - panos"
subcategory: ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will probably need a subcategory, tho I don't know what it should be at this point...

Just keep it in mind, we'll need to set this subcategory to something that makes sense for users before the provider is released.

Additionally I may need to update this as I'm working on the uuid-style resource, so I'm not sure that the data source or the docs are in a final state right now.

description: |-
Helper data source: create a tfid from the given information. Note that the tfid ouptut from this data source may not exactly match what a resource uses, but it will still be a valid ID to use for resource imports.
---

# panos_tfid (Data Source)

Helper data source: create a tfid from the given information. Note that the tfid ouptut from this data source may not
exactly match what a resource uses, but it will still be a valid ID to use for resource imports.

## Example Usage

```terraform
# Note that IDs created by this data source may not exactly match IDs created
# by a given resource, but it will be compatible and a valid ID for
# doing state imports.

# Example of how to create the ID for panos_nested_address_object named
# "foo" in vsys1.
#
# All variables should be specified for a given location, default value or not.
data "panos_tfid" "example1" {
name = "foo"
location = "vsys"
variables = {
"name" : "vsys1",
"ngfw_device" : "localhost.localdomain",
}
}

# Example of how to create the ID for panos_nested_address_object
# named "foo" in shared.
data "panos_tfid" "example2" {
name = "foo"
location = "shared"
}
```

<!-- schema generated by tfplugindocs -->

## Schema

### Required

- `location` (String) The location path name.

### Optional

- `name` (String) (Singleton resource) The config's name.
- `names` (List of String) (Grouping resources) The names of the configs.
- `variables` (Map of String) The variables and values for the specified location.

### Read-Only

- `tfid` (String) The tfid created from the given parts.
188 changes: 188 additions & 0 deletions assets/tf_provider/docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,188 @@
---
page_title: "Provider: panos"
description: |-
The Terraform provider for Palo Alto Networks PAN-OS.
---

# Provider panos

PAN-OS&reg; is the operating system for Palo Alto Networks&reg; NGFWs and Panorama&trade;. The panos provider allows you
to manage various aspects of a firewall's or a Panorama's config, such as data interfaces and security policies.

Use the navigation to the left to read about the available Panorama and NGFW resources.

Refer to [the changelog](https://github.com/PaloAltoNetworks/terraform-provider-panos/blob/master/CHANGELOG.md) to see
what's new.

## Versioning

The minimum version of PAN-OS is PAN-OS 10.1 and onwards. In PAN-OS 10.1.5 a change was made with regards to the XML API
that removed the provider's easy access to remove multiple config objects in a single API call.

To compensate for this loss of functionality, the provider now does multi-config API calls against PAN-OS when
multiple `type=config` API calls must be run.

With regards to data sources and resources, the minimum version for any given data source or resource will be present in
the documentation if the version is higher than PAN-OS 10.1.

If you need to work with multiple versions of PAN-OS where some versions have a new parameter and some don't, then make
sure to use [conditionals](https://www.terraform.io/docs/configuration/expressions/conditionals.html) along with
the `panos_system_info` data source to only set these variables when the version of PAN-OS is appropriate.

## List Filtering

Data sources that return a list of objects now supports filtering. Please refer to the guide on the side for more
information on how to use filtering.

## Provider Modes

The provider features two distinct modes: API and local inspection.

The API mode is the standard mode where the provider connects to a running PAN-OS instance and executes API commands
against it. This mode can be used across both data sources and resources.

The local inspection mode in contrast allows users to use Terraform to inspect a locally saved XML schema that was
previously exported from PAN-OS. Interaction with the exported XML schema is limited to data sources only, but there may
still be some types of data sources that are incompatible with this (aka - User-ID data sources or exporting the tech
support file). This combined with data source list filtering should enable users to quickly inspect previously saved
configs from the comfort of Terraform. The versions of PAN-OS that [pango](https://github.com/PaloAltoNetworks/pango)
supports determines the versions that the provider can successfully function in local inspection mode.

## Candidate vs Running Config

Most data sources (aka - those that are `type=config`) now support querying both the candidate config as well as the
running config. The default is to query candidate config (`get`), but if the data source supports the `action` param
then this can be set to `show` to query the running config instead.

If the provider is in local inspection mode, then the `action` is ignored as the provider is getting answers directly
from the given XML schema.

## Terraform / PAN-OS Interaction

### `lifecycle.create_before_destroy`

The order of operations that Terraform handles updates / deletes does not by default work the way that PAN-OS does
things. In order to make Terraform behave properly, inside of **each and every resource** you need to specify
a [`lifecycle`](https://www.terraform.io/language/meta-arguments/lifecycle) block like so:

```hcl
resource "panos_address_object" "example" {
name = "web server 1"
# continue with the rest of the definition
...

lifecycle {
create_before_destroy = true
}
}
```

### Parallelism

Terraform uses goroutines to speed up deployment, but the number of parallel operations is launches
exceeds [what is recommended](https://docs.paloaltonetworks.com/pan-os/10-0/pan-os-panorama-api/pan-os-xml-api-request-types/apply-user-id-mapping-and-populate-dynamic-address-groups-api.html):

```
Limit the number of concurrent API calls to five. This limit ensures that there is no performance impact to the firewall web interface as the management plane web server handles requests from both the API and the web interface.
```

In order to accomplish this, make sure you set
the [parallelism](https://www.terraform.io/cli/commands/apply#parallelism-n) value at or below this limit to prevent
performance impacts.

## Commits

As of right now, Terraform does not provide native support for commits, so commits are handled out-of-band. Please refer
to the commit guide to the left for more information.

## AWS / GCP Considerations

If you are launching PAN-OS in AWS or GCP, there are additional considerations that you should be aware of with regards
to initial configuration. Please see the AWS / GCP Considerations guide off to the left.

## Example Provider Usage

```terraform
# Traditional provider example.
provider "panos" {
hostname = "10.1.1.1"
username = "admin"
password = "secret"
}

# Local inspection mode provider example.
provider "panos" {
config_file = file("/tmp/candidate-config.xml")

# This is only used if a "detail-version" attribute is not present in
# the exported XML schema. If it's there, this can be omitted.
panos_version = "10.2.0"
}

terraform {
required_providers {
panos = {
source = "paloaltonetworks/terraform-provider-panos"
version = "2.0.0"
}
}
}
```

## Provider Parameter Priority

There are multiple ways to specify the provider's parameters. If overlapping values are configured for the provider,
then this is the resolution order:

1. Directly in the `provider` block
2. Environment variable (where applicable)
3. From the JSON config file

<!-- schema generated by tfplugindocs -->

## Schema

### Optional

- `additional_headers` (Map of String) Additional HTTP headers to send with API calls Environment
variable: `PANOS_HEADERS`. JSON config file variable: `additional_headers`.
- `api_key` (String) The API key for PAN-OS. Either specify this or give both username and password. Environment
variable: `PANOS_API`. JSON config file variable: `api_key`.
- `api_key_in_request` (Boolean) Send the API key in the request body instead of using the authentication header.
Environment variable: `PANOS_API_KEY_IN_REQUEST`. JSON config file variable: `api_key_in_request`.
- `auth_file` (String) Filesystem path to a JSON config file that specifies the provider's params.
- `config_file` (String) (Local inspection mode) The PAN-OS config file to load read in using `file()`
- `hostname` (String) The hostname or IP address of the PAN-OS instance (NGFW or Panorama). Environment
variable: `PANOS_HOST`. JSON config file variable: `hostname`.
- `panos_version` (String) (Local inspection mode) The version of PAN-OS that exported the config file. This is only
used if the root "config" block does not contain the "detail-version" attribute. Example: `10.2.3`.
- `password` (String, Sensitive) The password. This is required if the api_key is not configured. Environment
variable: `PANOS_PASSWORD`. JSON config file variable: `password`.
- `port` (Number) If the port is non-standard for the protocol, the port number to use. Environment
variable: `PANOS_PORT`. JSON config file variable: `port`.
- `protocol` (String) The protocol (https or http). Default: `https`. Environment variable: `PANOS_PROTOCOL`. JSON
config file variable: `protocol`.
- `skip_verify_certificate` (Boolean) (For https protocol) Skip verifying the HTTPS certificate. Environment
variable: `PANOS_SKIP_VERIFY_CERTIFICATE`. JSON config file variable: `skip_verify_certificate`.
- `target` (String) Target setting (NGFW serial number). Environment variable: `PANOS_TARGET`. JSON config file
variable: `target`.
- `username` (String) The username. This is required if api_key is not configured. Environment
variable: `PANOS_USERNAME`. JSON config file variable: `username`.

The following is an example of the contents of a JSON config file:

```json
{
"hostname": "127.0.0.1",
"api_key": "secret",
"skip_verify_certificate": true
}
```

## Support

This template/script/solution is released “as-is”, with no warranty and no support. These should be seen as community
supported and Palo Alto Networks may contribute its expertise at its discretion. Palo Alto Networks, including through
its Authorized Support Centers (ASC) partners and backline support options, will not provide technical support or help
in using or troubleshooting this template/script/solution. The underlying product used by this template/script/solution
will still be supported in accordance with the product’s applicable support policy and the customer’s entitlements.
23 changes: 23 additions & 0 deletions assets/tf_provider/examples/data-sources/panos_tfid/data-source.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Note that IDs created by this data source may not exactly match IDs created
# by a given resource, but it will be compatible and a valid ID for
# doing state imports.

# Example of how to create the ID for panos_nested_address_object named
# "foo" in vsys1.
#
# All variables should be specified for a given location, default value or not.
data "panos_tfid" "example1" {
name = "foo"
location = "vsys"
variables = {
"name" : "vsys1",
"ngfw_device" : "localhost.localdomain",
}
}

# Example of how to create the ID for panos_nested_address_object
# named "foo" in shared.
data "panos_tfid" "example2" {
name = "foo"
location = "shared"
}
24 changes: 24 additions & 0 deletions assets/tf_provider/examples/provider/provider.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Traditional provider example.
provider "panos" {
hostname = "10.1.1.1"
username = "admin"
password = "secret"
}

# Local inspection mode provider example.
provider "panos" {
config_file = file("/tmp/candidate-config.xml")

# This is only used if a "detail-version" attribute is not present in
# the exported XML schema. If it's there, this can be omitted.
panos_version = "10.2.0"
}

terraform {
required_providers {
panos = {
source = "paloaltonetworks/terraform-provider-panos"
version = "2.0.0"
}
}
}
Loading