Skip to content

Commit

Permalink
Merge pull request #12 from Azure/pr-xml-regions
Browse files Browse the repository at this point in the history
Support promotion to more than two regions
  • Loading branch information
boumenot authored Jun 27, 2017
2 parents e1c20bd + 472fdc5 commit 61113e8
Show file tree
Hide file tree
Showing 55 changed files with 2,236 additions and 162 deletions.
18 changes: 14 additions & 4 deletions Godeps/Godeps.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 36 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,24 +55,22 @@ USAGE:
azure-extensions-cli [global options] command [command options] [arguments...]
COMMANDS:
new-extension-manifest Creates an XML file used to publish or update extension.
new-extension Creates a new type of extension, not for releasing new versions.
new-extension-version Publishes a new type of extension internally.
promote-single-region Promote published internal extension to a PROD Location.
promote-two-regions Promote published extension to two PROD Locations.
promote-to-prod Promote published extension to all PROD Locations.
list-versions Lists all published extension versions for subscription
new-extension-manifest Creates an XML file used to publish or update extension.
new-extension Creates a new type of extension, not for releasing new versions.
new-extension-version Publishes a new type of extension internally.
promote Promote published internal extension to one or more PROD Locations.
promote-all-regions Promote published extension to all PROD Locations.
list-versions Lists all published extension versions for subscription
replication-status Retrieves replication status for an uploaded extension package
unpublish-version Marks the specified version of the extension internal. Does not delete.
delete-version Deletes the extension version. It should be unpublished first.
help, h Shows a list of commands or help for one command
delete-version Deletes the extension version. It should be unpublished first.
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help
--version, -v print the version
```


## Installing (or building from source)

You can head over to the **Releases** section to download a binary built for various platforms.
Expand All @@ -81,10 +79,36 @@ If you need to compile from the source code, make sure you have Go compiler 1.6+
Check out the project, set the GOPATH environment variable correctly (if necessary) and
run `go build`. This should compile a binary.

## Author
## Overview

The CLI makes it easy (easier) to publish an Azure extension. An example workflow is provided below. This workflow
assumes an extension type already exists, which is why the command **new-extension-version** is used. (If the type does
not exist use substitute for the command new-extension instead.)

Not all command line parameters are shown for each command, only the salient options are shown.

Step 1 - create an extension manifest.

1. ./azure-extensions-cli new-extension-manifest

Step 2 - publish an extension internally.

1. ./azure-extensions-cli new-extension-version

Step 3 - rollout the extension to Azure, by slowly including more and more regions. It is recommended that you pause
24 hours between regions.

Ahmet Alp Balkan
> Every time a new region is added, the previous regions must be included with the promote command.
1. ./azure-extensions-cli promote --region "West Central US"
1. ./azure-extensions-cli promote --region "West Central US" --region "North US"
1. ./azure-extensions-cli promote --region "West Central US" --region "North US" --region "West US"
1. ./azure-extensions-cli promote ...

Step 4 - promote the extension to all Azure regions.

1. ./azure-extensions-cli promote-all-regions

## TODO

- [ ] make `replication-status` exit with appropriate code if replication is not completed.
Expand Down
26 changes: 9 additions & 17 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,10 @@ var (
flStorageAccount = cli.StringFlag{
Name: "storage-account",
Usage: "Name of an existing storage account to be used in uploading the extension package temporarily."}
flRegion1 = cli.StringFlag{
Name: "region-1",
Usage: "Primary pilot location to roll out the extension (e.g. 'Japan East')",
EnvVar: "REGION1"}
flRegion2 = cli.StringFlag{
Name: "region-2",
Usage: "Primary pilot location to roll out the extension (e.g. 'Brazil South')",
EnvVar: "REGION2"}
flRegion = cli.StringSliceFlag{
Name: "region",
Usage: "List of one or more regions to rollout an extension (e.g. 'Japan East')",
}
flJSON = cli.BoolFlag{
Name: "json",
Usage: "Print output as JSON"}
Expand Down Expand Up @@ -120,21 +116,17 @@ func main() {
Usage: "Publishes a new type of extension internally.",
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert, flManifest},
Action: updateExtension},
{Name: "promote-single-region",
Usage: "Promote published internal extension to PROD in a Location.",
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert, flManifest, flRegion1},
Action: promoteToFirstSlice},
{Name: "promote-two-regions",
Usage: "Promote published extension to PROD in two Locations.",
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert, flManifest, flRegion1, flRegion2},
Action: promoteToSecondSlice},
{Name: "promote",
Usage: "Promote published internal extension to PROD in one or more locations.",
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert, flManifest, flRegion},
Action: promoteToRegions},
{Name: "promote-all-regions",
Usage: "Promote published extension to all Locations.",
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert, flManifest},
Action: promoteToAllRegions},
{Name: "list-versions",
Usage: "Lists all published extension versions for subscription",
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert},
Flags: []cli.Flag{flMgtURL, flSubsID, flSubsCert, flJSON},
Action: listVersions},
{Name: "replication-status",
Usage: "Retrieves replication status for an uploaded extension package",
Expand Down
205 changes: 152 additions & 53 deletions manifest.go
Original file line number Diff line number Diff line change
@@ -1,76 +1,175 @@
package main

import (
"os"
"text/template"
"encoding/xml"
"fmt"

log "github.com/Sirupsen/logrus"
"github.com/codegangsta/cli"
"io/ioutil"
"strings"
)

type certificate struct {
StoreLocation string `xml:"StoreLocation,omitempty"`
StoreName string `xml:"StoreName,omitempty"`
ThumbprintRequired bool `xml:"ThumbprintRequired,omitempty"`
ThumbprintAlgorithm string `xml:"ThumbprintAlgorithm,omitempty"`
}

// NOTE(@boumenot): there is probably a better way to express this. If
// you know please share...
//
// The only difference between ExtensionImage and ExtensionImageGlobal is the
// Regions element. This element can be in three different states to my
// knowledge.
//
// 1. not defined
// 2. <Regions>Region1;Region2</Regions>
// 3. <Regions></Regions>
//
// Case (1) occurs when an extension is first published. Case(2) occurs when
// an extension is promoted to one or two regions. Case (3) occurs when an
// extension is published to all regions.
//
// I do not know how to express all three cases using Go's XML serializer.
//
type extensionImage struct {
XMLName string `xml:"ExtensionImage"`
NS string `xml:"xmlns,attr"`
ProviderNameSpace string `xml:"ProviderNameSpace"`
Type string `xml:"Type"`
Version string `xml:"Version"`
Label string `xml:"Label"`
HostingResources string `xml:"HostingResources"`
MediaLink string `xml:"MediaLink"`
Certificate *certificate `xml:"Certificate,omitempty"`
PublicConfigurationSchema string `xml:"PublicConfigurationSchema,omitempty"`
PrivateConfigurationSchema string `xml:"PrivateConfigurationSchema,omitempty"`
Description string `xml:"Description"`
BlockRoleUponFailure string `xml:"BlockRoleUponFailure,omitempty"`
IsInternalExtension bool `xml:"IsInternalExtension"`
Eula string `xml:"Eula,omitempty"`
PrivacyURI string `xml:"PrivacyUri,omitempty"`
HomepageURI string `xml:"HomepageUri,omitempty"`
IsJSONExtension bool `xml:"IsJsonExtension,omitempty"`
CompanyName string `xml:"CompanyName,omitempty"`
SupportedOS string `xml:"SupportedOS,omitempty"`
Regions string `xml:"Regions,omitempty"`
}

type extensionImageGlobal struct {
XMLName string `xml:"ExtensionImage"`
NS string `xml:"xmlns,attr"`
ProviderNameSpace string `xml:"ProviderNameSpace"`
Type string `xml:"Type"`
Version string `xml:"Version"`
Label string `xml:"Label"`
HostingResources string `xml:"HostingResources"`
MediaLink string `xml:"MediaLink"`
Certificate *certificate `xml:"Certificate,omitempty"`
PublicConfigurationSchema string `xml:"PublicConfigurationSchema,omitempty"`
PrivateConfigurationSchema string `xml:"PrivateConfigurationSchema,omitempty"`
Description string `xml:"Description"`
BlockRoleUponFailure string `xml:"BlockRoleUponFailure,omitempty"`
IsInternalExtension bool `xml:"IsInternalExtension"`
Eula string `xml:"Eula,omitempty"`
PrivacyURI string `xml:"PrivacyUri,omitempty"`
HomepageURI string `xml:"HomepageUri,omitempty"`
IsJSONExtension bool `xml:"IsJsonExtension,omitempty"`
CompanyName string `xml:"CompanyName,omitempty"`
SupportedOS string `xml:"SupportedOS,omitempty"`
Regions string `xml:"Regions"`
}

type extensionManifest interface {
Marshal() ([]byte, error)
}

func isGuestAgent(providerNameSpace string) bool {
return "Microsoft.OSTCLinuxAgent" == providerNameSpace
}

func newExtensionImageManifest(filename string, regions []string) (extensionManifest, error) {
b, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}

var manifest extensionImage
err = xml.Unmarshal(b, &manifest)
if err != nil {
return nil, err
}

manifest.Regions = strings.Join(regions, ";")

if !isGuestAgent(manifest.ProviderNameSpace) {
manifest.IsInternalExtension = false
} else {
log.Debug("VM agent namespace detected, IsInternalExtension ignored")
}

return &manifest, nil
}

func newExtensionImageGlobalManifest(filename string) (extensionManifest, error) {
b, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}

var manifest extensionImageGlobal
err = xml.Unmarshal(b, &manifest)
if err != nil {
return nil, err
}

manifest.IsInternalExtension = !isGuestAgent(manifest.ProviderNameSpace)
return &manifest, nil
}

func (ext *extensionImage) Marshal() ([]byte, error) {
return xml.Marshal(*ext)
}

func (ext *extensionImageGlobal) Marshal() ([]byte, error) {
return xml.Marshal(*ext)
}

func newExtensionManifest(c *cli.Context) {
cl := mkClient(checkFlag(c, flMgtURL.Name), checkFlag(c, flSubsID.Name), checkFlag(c, flSubsCert.Name))
storageRealm := checkFlag(c, flStorageRealm.Name)
storageAccount := checkFlag(c, flStorageAccount.Name)
extensionPkg := checkFlag(c, flPackage.Name)

var p struct {
Namespace, Name, Version, BlobURL, Label, Description, Eula, Privacy, Homepage, Company, OS string
}
flags := []struct {
ref *string
fl string
}{
{&p.Namespace, flNamespace.Name},
{&p.Name, flName.Name},
{&p.Version, flVersion.Name},
{&p.Label, "label"},
{&p.Description, "description"},
{&p.Eula, "eula-url"},
{&p.Privacy, "privacy-url"},
{&p.Homepage, "homepage-url"},
{&p.Company, "company"},
{&p.OS, "supported-os"},
}
for _, f := range flags {
*f.ref = checkFlag(c, f.fl)
}

// Upload extension blob
blobURL, err := uploadBlob(cl, storageRealm, storageAccount, extensionPkg)
if err != nil {
log.Fatal(err)
}
log.Debugf("Extension package uploaded to: %s", blobURL)
p.BlobURL = blobURL

// doing a text template is easier and let us create comments (xml encoder can't)
// that are used as placeholders later on.
manifestXML := `<?xml version="1.0" encoding="utf-8" ?>
<ExtensionImage xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<!-- WARNING: Ordering of fields matter in this file. -->
<ProviderNameSpace>{{.Namespace}}</ProviderNameSpace>
<Type>{{.Name}}</Type>
<Version>{{.Version}}</Version>
<Label>{{.Label}}</Label>
<HostingResources>VmRole</HostingResources>
<MediaLink>{{.BlobURL}}</MediaLink>
<Description>{{.Description}}</Description>
<IsInternalExtension>true</IsInternalExtension>
<Eula>{{.Eula}}</Eula>
<PrivacyUri>{{.Privacy}}</PrivacyUri>
<HomepageUri>{{.Homepage}}</HomepageUri>
<IsJsonExtension>true</IsJsonExtension>
<CompanyName>{{.Company}}</CompanyName>
<SupportedOS>{{.OS}}</SupportedOS>
<!--%REGIONS%-->
</ExtensionImage>
`
tpl, err := template.New("manifest").Parse(manifestXML)
if err != nil {
log.Fatalf("template parse error: %v", err)

manifest := extensionImage{
ProviderNameSpace: checkFlag(c, flNamespace.Name),
Type: checkFlag(c, flName.Name),
Version: checkFlag(c, flVersion.Name),
Label: "label",
Description: "description",
IsInternalExtension: true,
MediaLink: blobURL,
Eula: "eula-url",
PrivacyURI: "privacy-url",
HomepageURI: "homepage-url",
IsJSONExtension: true,
CompanyName: "company",
SupportedOS: "supported-os",
}
if err = tpl.Execute(os.Stdout, p); err != nil {
log.Fatalf("template execute error: %v", err)

bs, err := xml.MarshalIndent(manifest, "", " ")
if err != nil {
log.Fatalf("xml marshall error: %v", err)
}

fmt.Println(string(bs))
}
17 changes: 17 additions & 0 deletions manifest_test.TestRoundTripExtensionImage.approved.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
<ExtensionImage xmlns="http://schemas.microsoft.com/windowsazure">
<ProviderNameSpace>Microsoft.OSCTExtensions</ProviderNameSpace>
<Type>CustomScriptForLinux</Type>
<Version>4.3.2.1</Version>
<Label>Microsoft Azure Custom Script Extension for Linux Virtual Machines</Label>
<HostingResources>VmRole</HostingResources>
<MediaLink>http://localhost/extension.zip</MediaLink>
<Description>Please consider using Microsoft.Azure.Extensions.CustomScript instead.</Description>
<IsInternalExtension>true</IsInternalExtension>
<Eula>https://github.com/Azure/azure-linux-extensions/blob/master/LICENSE-2_0.txt</Eula>
<PrivacyUri>http://www.microsoft.com/privacystatement/en-us/OnlineServices/Default.aspx</PrivacyUri>
<HomepageUri>https://github.com/Azure/azure-linux-extensions</HomepageUri>
<IsJsonExtension>true</IsJsonExtension>
<CompanyName>Microsoft</CompanyName>
<SupportedOS>Linux</SupportedOS>
<Regions>South Central US</Regions>
</ExtensionImage>
Loading

0 comments on commit 61113e8

Please sign in to comment.