Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Copy static files (assets) #29

Merged
merged 28 commits into from
Mar 8, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
2155e70
new function to make indentation equal in list of strings
sebastianczech Mar 4, 2024
d5ffed6
execute new function creating definition of structs
sebastianczech Mar 4, 2024
dcb8f2e
new function creating definition of structs (with tests)
sebastianczech Mar 4, 2024
7c5f19b
fix logic after tests
sebastianczech Mar 4, 2024
8171827
refactor code (remove duplication and extract func)
sebastianczech Mar 4, 2024
7aad155
fix random order in keys for maps
sebastianczech Mar 5, 2024
b2bef13
update testify package
sebastianczech Mar 5, 2024
cd267f2
new functions to generate functions for locations
sebastianczech Mar 5, 2024
e975329
use new functions for locations
sebastianczech Mar 5, 2024
b5095ac
extend template, do not concat strings to generate structs
sebastianczech Mar 5, 2024
a07ae67
extend template, do not concat strings to generate functions
sebastianczech Mar 5, 2024
2b7514c
clean code
sebastianczech Mar 5, 2024
5bc88c5
tests for functions used while generating structs and funcs
sebastianczech Mar 5, 2024
d9fdd83
fix issues after tests
sebastianczech Mar 5, 2024
f1f2525
remove not required spaces from templates
sebastianczech Mar 5, 2024
b2fb714
introduce new struct NameVariant for Location names and variables, ge…
sebastianczech Mar 6, 2024
b535826
Change Go SDK path for security policy rule
sebastianczech Mar 6, 2024
ea21716
extend configuration file with settings for copying static assets
sebastianczech Mar 6, 2024
d2424f3
static files from pango in develop branch
sebastianczech Mar 6, 2024
0824d64
add function to copy static assets
sebastianczech Mar 6, 2024
54f99e3
change go sdk path for service and address group
sebastianczech Mar 6, 2024
f10d323
Merge branch 'render-location' into copy-static-files
sebastianczech Mar 6, 2024
96e300a
add error handling for func AsEntryXpath
sebastianczech Mar 8, 2024
4b35c89
Merge branch 'render-location' into copy-static-files
sebastianczech Mar 8, 2024
82b673b
rename assets.go
sebastianczech Mar 8, 2024
d67d35c
ignore error while closing file in defer
sebastianczech Mar 8, 2024
f1f1e8b
remove files copied from pango
sebastianczech Mar 8, 2024
12df7cc
Merge branch 'main' into copy-static-files
sebastianczech Mar 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion cmd/mktp/config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,22 @@
output:
go_sdk: "../generated/pango"
terraform_provider: "../generated/terraform-provider-panos"
terraform_provider: "../generated/terraform-provider-panos"
assets:
# util_package:
# source: "assets/util"
# target:
# go_sdk: true
# terraform_provider: false
# destination: "util"
# errors_package:
# source: "assets/errors"
# target:
# go_sdk: true
# terraform_provider: false
# destination: "errors"
# version_package:
# source: "assets/version"
# target:
# go_sdk: true
# terraform_provider: false
# destination: "version"
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
module github.com/paloaltonetworks/pan-os-codegen

require (
github.com/stretchr/testify v1.8.4
github.com/stretchr/testify v1.9.0
gopkg.in/yaml.v3 v3.0.1
)

Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
Expand Down
91 changes: 91 additions & 0 deletions pkg/generate/assets.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
package generate

import (
"bytes"
"fmt"
"github.com/paloaltonetworks/pan-os-codegen/pkg/properties"
"io"
"io/fs"
"os"
"path/filepath"
)

func CopyAssets(config *properties.Config) error {
for _, asset := range config.Assets {
files, err := listAssets(asset)
if err != nil {
return err
}

if asset.Target.GoSdk {
if err = copyAsset(config.Output.GoSdk, asset, files); err != nil {
return err
}
}
if asset.Target.TerraformProvider {
if err = copyAsset(config.Output.TerraformProvider, asset, files); err != nil {
return err
}
}
}

return nil
}

func listAssets(asset *properties.Asset) ([]string, error) {
var files []string

// Walk through directory and get list of all files
err := filepath.WalkDir(asset.Source, func(path string, entry fs.DirEntry, err error) error {
if err != nil {
return err
}
if !entry.IsDir() {
files = append(files, path)
}
return nil
})
if err != nil {
return nil, err
}

return files, nil
}

func copyAsset(target string, asset *properties.Asset, files []string) error {
// Prepare destination path
destinationDir := target + "/" + asset.Destination

// Create the destination directory if it doesn't exist
if err := os.MkdirAll(destinationDir, os.ModePerm); err != nil {
return err
}

for _, sourceFilePath := range files {
// Prepare destination path
destinationFilePath := filepath.Join(destinationDir, filepath.Base(sourceFilePath))
fmt.Printf("Copy file from %s to %s\n", sourceFilePath, destinationFilePath)

// Read the contents of the source file
data, err := os.ReadFile(sourceFilePath)
if err != nil {
return err
}

// Create the destination file
destinationFile, err := os.Create(destinationFilePath)
if err != nil {
return err
}
defer func(destinationFile *os.File) {
_ = destinationFile.Close()
}(destinationFile)

// Write the contents into the destination file
_, err = io.Copy(destinationFile, bytes.NewReader(data))
if err != nil {
return err
}
}
return nil
}
8 changes: 7 additions & 1 deletion pkg/generate/generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,13 @@ func (c *Creator) generateOutputFileFromTemplate(tmpl *template.Template, output
func (c *Creator) parseTemplate(templateName string) (*template.Template, error) {
templatePath := fmt.Sprintf("%s/%s", c.TemplatesDir, templateName)
funcMap := template.FuncMap{
"packageName": translate.PackageName,
"packageName": translate.PackageName,
"locationType": translate.LocationType,
"omitEmpty": translate.OmitEmpty,
"contains": func(full, part string) bool {
return strings.Contains(full, part)
},
"asEntryXpath": translate.AsEntryXpath,
}
tmpl, err := template.New(templateName).Funcs(funcMap).ParseFiles(templatePath)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion pkg/generate/generator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ func TestListOfTemplates(t *testing.T) {
assert.Equal(t, 4, len(templates))
}

func TestParseTemplate(t *testing.T) {
func TestParseTemplateForInterfaces(t *testing.T) {
// given
spec := properties.Normalization{
GoSdkPath: []string{"object", "address"},
Expand Down
5 changes: 5 additions & 0 deletions pkg/mktp/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,11 @@ func (c *Cmd) Execute() error {
// Output as Terraform code.
}

// Copy assets (static files)
if err = generate.CopyAssets(config); err != nil {
return fmt.Errorf("error copying assets %s", err)
}

// Finalize pango code:
// * make fmt

Expand Down
14 changes: 13 additions & 1 deletion pkg/properties/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,26 @@ package properties
import "github.com/paloaltonetworks/pan-os-codegen/pkg/content"

type Config struct {
Output OutputPaths `json:"output" yaml:"output"`
Output OutputPaths `json:"output" yaml:"output"`
Assets map[string]*Asset `json:"assets" yaml:"assets"`
}

type OutputPaths struct {
GoSdk string `json:"go_sdk" yaml:"go_sdk"`
TerraformProvider string `json:"terraform_provider" yaml:"terraform_provider"`
}

type Asset struct {
Source string `json:"source" yaml:"source"`
Target *Target `json:"target" yaml:"target"`
Destination string `json:"destination" yaml:"destination"`
}

type Target struct {
GoSdk bool `json:"go_sdk" yaml:"go_sdk"`
TerraformProvider bool `json:"terraform_provider" yaml:"terraform_provider"`
}

func ParseConfig(input []byte) (*Config, error) {
var ans Config
err := content.Unmarshal(input, &ans)
Expand Down
14 changes: 14 additions & 0 deletions pkg/properties/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,13 @@ func TestConfig(t *testing.T) {
const content = `output:
go_sdk: "../generated/pango"
terraform_provider: "../generated/terraform-provider-panos"
assets:
util_package:
source: "assets/util"
target:
go_sdk: true
terraform_provider: false
destination: "util"
`

// when
Expand All @@ -20,4 +27,11 @@ func TestConfig(t *testing.T) {
assert.NotEmptyf(t, config.Output, "Config output cannot be empty")
assert.NotEmptyf(t, config.Output.GoSdk, "Config Go SDK path cannot be empty")
assert.NotEmptyf(t, config.Output.TerraformProvider, "Config Terraform provider path cannot be empty")
assert.NotEmpty(t, config.Assets)
assert.Equal(t, 1, len(config.Assets))
assert.Equal(t, 1, len(config.Assets))
assert.Equal(t, "assets/util", config.Assets["util_package"].Source)
assert.True(t, config.Assets["util_package"].Target.GoSdk)
assert.False(t, config.Assets["util_package"].Target.TerraformProvider)
assert.Equal(t, "util", config.Assets["util_package"].Destination)
}
36 changes: 33 additions & 3 deletions pkg/properties/normalized.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"github.com/paloaltonetworks/pan-os-codegen/pkg/content"
"github.com/paloaltonetworks/pan-os-codegen/pkg/naming"
"io/fs"
"path/filepath"
"runtime"
Expand All @@ -21,7 +22,13 @@ type Normalization struct {
Spec *Spec `json:"spec" yaml:"spec"`
}

type NameVariant struct {
Underscore string
CamelCase string
}

type Location struct {
Name *NameVariant
Description string `json:"description" yaml:"description"`
Device *LocationDevice `json:"device" yaml:"device"`
Xpath []string `json:"xpath" yaml:"xpath"`
Expand All @@ -35,6 +42,7 @@ type LocationDevice struct {
}

type LocationVar struct {
Name *NameVariant
Description string `json:"description" yaml:"description"`
Required bool `json:"required" yaml:"required"`
Validation *LocationVarValidation `json:"validation" yaml:"validation"`
Expand Down Expand Up @@ -128,9 +136,31 @@ func GetNormalizations() ([]string, error) {
}

func ParseSpec(input []byte) (*Normalization, error) {
var ans Normalization
err := content.Unmarshal(input, &ans)
return &ans, err
var spec Normalization

err := content.Unmarshal(input, &spec)

err = spec.AddNameVariants()

return &spec, err
}

func (spec *Normalization) AddNameVariants() error {
for key, location := range spec.Locations {
location.Name = &NameVariant{
Underscore: key,
CamelCase: naming.CamelCase("", key, "", true),
}

for subkey, variable := range location.Vars {
variable.Name = &NameVariant{
Underscore: subkey,
CamelCase: naming.CamelCase("", subkey, "", true),
}
}
}

return nil
}

func (spec *Normalization) Sanity() error {
Expand Down
24 changes: 24 additions & 0 deletions pkg/properties/normalized_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,9 @@ xpath_suffix:
- address
locations:
device_group:
name:
underscore: device_group
camelcase: DeviceGroup
description: Located in a specific device group.
device:
panorama: true
Expand All @@ -167,16 +170,25 @@ locations:
read_only: false
vars:
device_group:
name:
underscore: device_group
camelcase: DeviceGroup
description: The device group.
required: true
validation:
not_values:
shared: The device group cannot be "shared". Use the "shared" path instead.
panorama_device:
name:
underscore: panorama_device
camelcase: PanoramaDevice
description: The panorama device.
required: false
validation: null
from_panorama:
name:
underscore: from_panorama
camelcase: FromPanorama
description: Located in the config pushed from Panorama.
device:
panorama: false
Expand All @@ -187,6 +199,9 @@ locations:
read_only: true
vars: {}
shared:
name:
underscore: shared
camelcase: Shared
description: Located in shared.
device:
panorama: true
Expand All @@ -197,6 +212,9 @@ locations:
read_only: false
vars: {}
vsys:
name:
underscore: vsys
camelcase: Vsys
description: Located in a specific vsys.
device:
panorama: true
Expand All @@ -210,10 +228,16 @@ locations:
read_only: false
vars:
ngfw_device:
name:
underscore: ngfw_device
camelcase: NgfwDevice
description: The NGFW device.
required: false
validation: null
vsys:
name:
underscore: vsys
camelcase: Vsys
description: The vsys.
required: false
validation:
Expand Down
16 changes: 16 additions & 0 deletions pkg/translate/funcs.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package translate

import (
"errors"
"github.com/paloaltonetworks/pan-os-codegen/pkg/naming"
"strings"
)

func AsEntryXpath(location, xpath string) (string, error) {
if !strings.Contains(xpath, "$") || !strings.Contains(xpath, "}") {
return "", errors.New("$ followed by } should exists in xpath'")
}
xpath = strings.TrimSpace(strings.Split(strings.Split(xpath, "$")[1], "}")[0])
xpath = naming.CamelCase("", xpath, "", true)
return "util.AsEntryXpath([]string{o." + location + "." + xpath + "}),", nil
}
16 changes: 16 additions & 0 deletions pkg/translate/funcs_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package translate

import (
"github.com/stretchr/testify/assert"
"testing"
)

func TestAsEntryXpath(t *testing.T) {
// given

// when
asEntryXpath, _ := AsEntryXpath("DeviceGroup", "{{ Entry $panorama_device }}")

// then
assert.Equal(t, "util.AsEntryXpath([]string{o.DeviceGroup.PanoramaDevice}),", asEntryXpath)
}
Loading