Skip to content

Latest commit

 

History

History
322 lines (248 loc) · 19.8 KB

cli-overview.md

File metadata and controls

322 lines (248 loc) · 19.8 KB
Raw

AKS Engine CLI Overview

AKS Engine is designed to be used as a CLI tool (aks-engine). This document outlines the functionality that aks-engine provides to create and maintain a Kubernetes cluster on Azure.

aks-engine commands

To get a quick overview of the commands available via the aks-engine CLI tool, just run aks-engine with no arguments (or include the --help argument):

$ aks-engine
Usage:
  aks-engine [flags]
  aks-engine [command]

Available Commands:
  addpool       Add a node pool to an existing AKS Engine-created Kubernetes cluster
  completion    Generates bash completion scripts
  deploy        Deploy an Azure Resource Manager template
  generate      Generate an Azure Resource Manager template
  get-logs      Collect logs and current cluster nodes configuration.
  get-versions  Display info about supported Kubernetes versions
  help          Help about any command
  rotate-certs  (experimental) Rotate certificates on an existing AKS Engine-created Kubernetes cluster
  scale         Scale an existing AKS Engine-created Kubernetes cluster
  update        Update an existing AKS Engine-created VMSS node pool
  upgrade       Upgrade an existing AKS Engine-created Kubernetes cluster
  version       Print the version of aks-engine

Flags:
      --debug                enable verbose debug logs
  -h, --help                 help for aks-engine
      --show-default-model   Dump the default API model to stdout

Use "aks-engine [command] --help" for more information about a command.

Operational Cluster Commands

These commands are provided by AKS Engine in order to create and maintain Kubernetes clusters. Note: there is no aks-engine command to delete a cluster; to delete a Kubernetes cluster created by AKS Engine, you must delete the resource group that contains cluster resources. If the resource group can't be deleted because it contains other, non-Kubernetes-relate Azure resources, then you must manually delete the Virtual Machine and/or Virtual Machine Scale Set (VMSS), Disk, Network Interface, Network Security Group, Public IP Address, Virtual Network, Load Balancer, and all other resources specified in the aks-engine-generated ARM template. Because manually deleting resources is tedious and requires following serial dependencies in the correct order, it is recommended that you dedicate a resource group for the Azure resources that AKS Engine will create to run your Kubernetes cluster. If you're running more than one cluster, we recommend a dedicated resource group per cluster.

aks-engine deploy

The aks-engine deploy command will create a new cluster from scratch, using an API model (cluster definition) file as input to define the desired cluster configuration and shape, in the subscription, region, and resource group you provide, using credentials that you provide. Use this command to create a new cluster.

$ aks-engine deploy --help
Deploy an Azure Resource Manager template, parameters file and other assets for a cluster

Usage:
  aks-engine deploy [flags]

Flags:
  -m, --api-model string             path to your cluster definition file
      --auth-method client_secret    auth method (default:client_secret, `cli`, `client_certificate`, `device`) (default "cli")
      --auto-suffix                  automatically append a compressed timestamp to the dnsPrefix to ensure unique cluster name automatically
      --azure-env string             the target Azure cloud (default "AzurePublicCloud")
      --ca-certificate-path string   path to the CA certificate to use for Kubernetes PKI assets
      --ca-private-key-path string   path to the CA private key to use for Kubernetes PKI assets
      --certificate-path string      path to client certificate (used with --auth-method=client_certificate)
      --client-id string             client id (used with --auth-method=[client_secret|client_certificate])
      --client-secret string         client secret (used with --auth-method=client_secret)
  -p, --dns-prefix string            dns prefix (unique name for the cluster)
  -f, --force-overwrite              automatically overwrite existing files in the output directory
  -h, --help                         help for deploy
      --identity-system azure_ad     identity system (default:azure_ad, `adfs`) (default "azure_ad")
      --language string              language to return error messages in (default "en-us")
  -l, --location string              location to deploy to (required)
  -o, --output-directory string      output directory (derived from FQDN if absent)
      --private-key-path string      path to private key (used with --auth-method=client_certificate)
  -g, --resource-group string        resource group to deploy to (will use the DNS prefix from the apimodel if not specified)
      --set stringArray              set values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)
  -s, --subscription-id string       azure subscription id (required)

Global Flags:
      --debug   enable verbose debug logs

Detailed documentation on aks-engine deploy can be found here.

aks-engine scale

The aks-engine scale command will scale (in or out) a specific node pool participating in a Kubernetes cluster created by AKS Engine. Use this command to manually scale a node pool to a specific number of nodes.

$ aks-engine scale --help
Scale an existing AKS Engine-created Kubernetes cluster by specifying a new desired number of nodes in a node pool

Usage:
  aks-engine scale [flags]

Flags:
  -m, --api-model string            path to the generated apimodel.json file
      --apiserver string            apiserver endpoint (required to cordon and drain nodes)
      --auth-method client_secret   auth method (default:client_secret, `cli`, `client_certificate`, `device`) (default "cli")
      --azure-env string            the target Azure cloud (default "AzurePublicCloud")
      --certificate-path string     path to client certificate (used with --auth-method=client_certificate)
      --client-id string            client id (used with --auth-method=[client_secret|client_certificate])
      --client-secret string        client secret (used with --auth-method=client_secret)
  -h, --help                        help for scale
      --identity-system azure_ad    identity system (default:azure_ad, `adfs`) (default "azure_ad")
      --language string             language to return error messages in (default "en-us")
  -l, --location string             location the cluster is deployed in
  -c, --new-node-count int          desired number of nodes
      --node-pool string            node pool to scale
      --private-key-path string     path to private key (used with --auth-method=client_certificate)
  -g, --resource-group string       the resource group where the cluster is deployed
  -s, --subscription-id string      azure subscription id (required)

Global Flags:
      --debug   enable verbose debug logs

The scale command has limitations for scaling in (reducing the number of nodes in a node pool):

  • It accepts a new, desired node count; it does not accept a list of specific nodes to remove from the pool.
  • For VMSS-backed node pools, the removed nodes will not be cordoned and drained prior to being removed, which means any running workloads on nodes-to-be-removed will be disrupted without warning, and temporary operational impact is to be expected.

We generally recommend that you manage node pool scaling dynamically using the cluster-autoscaler project. More documentation about cluster-autoscaler is here, including how to automatically install and configure it at cluster creation time as an AKS Engine addon.

Detailed documentation on aks-engine scale can be found here.

aks-engine update

The aks-engine update command will update the VMSS model of a node pool according to a modified configuration of the aks-engine-generated apimodel.json. The updated node configuration will not take affect on any existing nodes, but will be applied to all future, new nodes created by VMSS scale out operations. Use this command to update the node configuration (such as the OS configuration, VM SKU, or Kubernetes kubelet configuration) of an existing VMSS node pool.

Note: aks-engine update can not be used to update the control plane! To update control plane VM configuration, see aks-engine upgrade --control-plane-only documentation here.

$ aks-engine update --help
Update an existing AKS Engine-created VMSS node pool in a Kubernetes cluster by updating its VMSS model

Usage:
  aks-engine update [flags]

Flags:
  -m, --api-model string            path to the generated apimodel.json file
      --auth-method client_secret   auth method (default:client_secret, `cli`, `client_certificate`, `device`) (default "cli")
      --azure-env string            the target Azure cloud (default "AzurePublicCloud")
      --certificate-path string     path to client certificate (used with --auth-method=client_certificate)
      --client-id string            client id (used with --auth-method=[client_secret|client_certificate])
      --client-secret string        client secret (used with --auth-method=client_secret)
  -h, --help                        help for update
      --identity-system azure_ad    identity system (default:azure_ad, `adfs`) (default "azure_ad")
      --language string             language to return error messages in (default "en-us")
  -l, --location string             location the cluster is deployed in
      --node-pool string            node pool to scale
      --private-key-path string     path to private key (used with --auth-method=client_certificate)
  -g, --resource-group string       the resource group where the cluster is deployed
  -s, --subscription-id string      azure subscription id (required)

Global Flags:
      --debug   enable verbose debug logs

Detailed documentation on aks-engine update can be found here.

aks-engine addpool

The aks-engine addpool command will add a new node pool to an existing AKS Engine-created cluster. Using a JSON file to define a the new node pool's configuration, and referencing the aks-engine-generated apimodel.json, you can add new nodes to your cluster. Use this command to add a specific number of new nodes using a discrete configuration compared to existing nodes participating in your cluster.

$ aks-engine addpool --help
Add a node pool to an existing AKS Engine-created Kubernetes cluster by referencing a new agentpoolProfile spec

Usage:
  aks-engine addpool [flags]

Flags:
  -m, --api-model string            path to the generated apimodel.json file
      --auth-method client_secret   auth method (default:client_secret, `cli`, `client_certificate`, `device`) (default "cli")
      --azure-env string            the target Azure cloud (default "AzurePublicCloud")
      --certificate-path string     path to client certificate (used with --auth-method=client_certificate)
      --client-id string            client id (used with --auth-method=[client_secret|client_certificate])
      --client-secret string        client secret (used with --auth-method=client_secret)
  -h, --help                        help for addpool
      --identity-system azure_ad    identity system (default:azure_ad, `adfs`) (default "azure_ad")
      --language string             language to return error messages in (default "en-us")
  -l, --location string             location the cluster is deployed in
  -p, --node-pool string            path to a JSON file that defines the new node pool spec
      --private-key-path string     path to private key (used with --auth-method=client_certificate)
  -g, --resource-group string       the resource group where the cluster is deployed
  -s, --subscription-id string      azure subscription id (required)

Global Flags:
      --debug   enable verbose debug logs

Detailed documentation on aks-engine addpool can be found here.

aks-engine upgrade

The aks-engine upgrade command orchestrates a Kubernetes version upgrade across your existing cluster nodes. Use this command to upgrade the Kubernetes version running your control plane, and optionally on all your nodes as well.

$ aks-engine upgrade --help
Upgrade an existing AKS Engine-created Kubernetes cluster, one node at a time

Usage:
  aks-engine upgrade [flags]

Flags:
  -m, --api-model string            path to the generated apimodel.json file
      --auth-method client_secret   auth method (default:client_secret, `cli`, `client_certificate`, `device`) (default "cli")
      --azure-env string            the target Azure cloud (default "AzurePublicCloud")
      --certificate-path string     path to client certificate (used with --auth-method=client_certificate)
      --client-id string            client id (used with --auth-method=[client_secret|client_certificate])
      --client-secret string        client secret (used with --auth-method=client_secret)
      --control-plane-only          upgrade control plane VMs only, do not upgrade node pools
      --cordon-drain-timeout int    how long to wait for each vm to be cordoned in minutes (default -1)
  -f, --force                       force upgrading the cluster to desired version. Allows same version upgrades and downgrades.
  -h, --help                        help for upgrade
      --identity-system azure_ad    identity system (default:azure_ad, `adfs`) (default "azure_ad")
  -b, --kubeconfig string           the path of the kubeconfig file
      --language string             language to return error messages in (default "en-us")
  -l, --location string             location the cluster is deployed in (required)
      --private-key-path string     path to private key (used with --auth-method=client_certificate)
  -g, --resource-group string       the resource group where the cluster is deployed (required)
  -s, --subscription-id string      azure subscription id (required)
  -k, --upgrade-version string      desired kubernetes version (required)
      --upgrade-windows-vhd         upgrade image reference of the Windows nodes (default true)
      --vm-timeout int              how long to wait for each vm to be upgraded in minutes (default -1)

Global Flags:
      --debug   enable verbose debug logs

Detailed documentation on aks-engine upgrade can be found here.

Generate an ARM Template

AKS Engine also provides a command to generate a reusable ARM template only, without creating any actual Azure resources.

aks-engine generate

The aks-engine generate command is similar to aks-engine deploy: it uses an API model (cluster definition) file as input to define the desired cluster configuration and shape of a new Kubernetes cluster. Unlike deploy, aks-engine generate does not actually submit any operational requests to Azure, but is instead used to generate a reusable ARM template which may be deployed at a later time. Use this command as a part of a workflow that creates one or more Kubernetes clusters via an ARM group deployment that takes an ARM template as input (e.g., az deployment group create using the standard az Azure CLI).

$ aks-engine generate --help
Generates an Azure Resource Manager template, parameters file and other assets for a cluster

Usage:
  aks-engine generate [flags]

Flags:
  -m, --api-model string             path to your cluster definition file
      --ca-certificate-path string   path to the CA certificate to use for Kubernetes PKI assets
      --ca-private-key-path string   path to the CA private key to use for Kubernetes PKI assets
      --client-id string             client id
      --client-secret string         client secret
  -h, --help                         help for generate
      --no-pretty-print              skip pretty printing the output
  -o, --output-directory string      output directory (derived from FQDN if absent)
      --parameters-only              only output parameters files
      --set stringArray              set values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2)

Global Flags:
      --debug   enable verbose debug logs

Detailed documentation on aks-engine generate can be found here.

aks-engine rotate-certs

The aks-engine rotate-certs command is currently experimental and not recommended for use on production clusters.

aks-engine get-logs

The aks-engine get-logs can conveniently collect host VM logs from your Linux node VMs for local troubleshooting. This command does not support Windows nodes. The command assumes that your node VMs have an SSH daemon listening on port 22, that all nodes share a common SSH keypair for interactive login, and that a public endpoint exists on one of the control plane VMs for accommodating SSH agent key forwarding.

$ aks-engine get-logs --help
Usage:
  aks-engine get-logs [flags]

Flags:
  -m, --api-model string               path to the generated apimodel.json file (required)
      --control-plane-only             get logs from control plane VMs only
  -h, --help                           help for get-logs
      --linux-script string            path to the log collection script to execute on the cluster's Linux nodes (required)
      --linux-ssh-private-key string   path to a valid private SSH key to access the cluster's Linux nodes (required)
  -l, --location string                Azure location where the cluster is deployed (required)
  -o, --output-directory string        collected logs destination directory, derived from --api-model if missing
      --ssh-host string                FQDN, or IP address, of an SSH listener that can reach all nodes in the cluster (required)

Global Flags:
      --debug   enable verbose debug logs

The aks-engine codebase contains a working log retrieval script in scripts/collect-logs.sh, so you can use it to quickly gather logs from your node VMs:

$ git clone https://github.com/Azure/aks-engine.git && cd aks-engine
Cloning into 'aks-engine'...
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 92107 (delta 13), reused 15 (delta 1), pack-reused 92063
Receiving objects: 100% (92107/92107), 92.86 MiB | 7.27 MiB/s, done.
Resolving deltas: 100% (64711/64711), done.
$ export LATEST_AKS_ENGINE_RELEASE=v0.56.0

$ git checkout $LATEST_AKS_ENGINE_RELEASE
Note: checking out 'v0.56.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 666073d49 chore: updating Windows VHD with new cached artifacts (#3843)
$ bin/aks-engine get-logs --api-model _output/$CLUSTER_NAME/apimodel.json --location $CLUSTER_NAME --linux-ssh-private-key _output/$CLUSTER_NAME-ssh --linux-script ./scripts/collect-logs.sh --ssh-host $CLUSTER_NAME.$LOCATION.cloudapp.azure.com
...
INFO[0062] Logs downloaded to _output/<name of cluster>/_logs

The following example assumes that the $CLUSTER_NAME environment variable is assigned to the value of the cluster name (properties.masterProfile.dnsPrefix in the cluster API model), and that $LOCATION is assigned to the location string of the resource group that your cluster was created into.