AKS data-gatherer missing some information

[wwwil]

Currently the AKS data gatherer collects information about the configuration of an AKS cluster from the Azure API.

https://docs.microsoft.com/en-us/rest/api/aks/managedclusters/get

The information returned includes a list of node pools, referred to as agent pools. However it does not give details of each of these pools. This needs to be fetched separately. We should get the configuration of each node pool so we can make the checks performed in the AKS package more comprehensive.

https://docs.microsoft.com/en-us/rest/api/aks/agentpools/get

These will both return separate JSON documents, in fact there will be a JSON document for each node pool. We need to work out how this will be handled in Preflight. We could put them all in a list in master JSON document to evaluate with Rego. Alternatively we could make an AKS node pool data gatherer separately, but this would require support for multiple instances of the same data gatherer type to fetch multiple node pools and seems like more work for users.

I had misunderstood the problem here. Using the az aks show --resource-group preflight --name preflight-test-wil command I can see all the required information, as described in the API spec: https://docs.microsoft.com/en-us/rest/api/aks/managedclusters/get

However when using HTTP GET requests, as the data gatherer does, some information is missing. This also occurs when doing the same thing manually with the curl command.

Related to #30

[wwwil]

Reading though the docs a bit closer it looks like the agentPoolProfiles part of what we get from the cluster JSON should have what we need, so we don't need to get the agent pool JSON separately. However, in examples it seems to be missing some values... Specifically I want properties.agentPoolProfiles[_].type.

[wwwil]

I'm going to look into how the az command gets it's information on Monday and try to work out what we need to change in Preflight. Going to start looking at this function that @j-fuentes found: https://github.com/Azure/azure-cli/blob/addbaf43b4618a500e5690a84096ddbef00df642/src/azure-cli/azure/cli/command_modules/acs/custom.py#L1902

[wwwil]

/assign @wwwil

[wwwil]

I'm now finding curl gets the info I want, but it is still not shown in the Preflight intermediate data.

$ curl -i -H "Authorization: Bearer $(jq -r .accessToken /tmp/credentials.json)" https://management.azure.com/subscriptions/4b4cdf50-7109-4d7a-9866-bbbb3d34acf9/resourceGroups/preflight/providers/Microsoft.ContainerService/managedClusters/preflight-test-wil\?api-version\=2019-08-01

Gives:

...
   "agentPoolProfiles": [
    {
     "name": "agentpool",
     "count": 3,
     "vmSize": "Standard_DS2_v2",
     "osDiskSizeGB": 100,
     "maxPods": 110,
     "type": "VirtualMachineScaleSets",
     "provisioningState": "Succeeded",
     "orchestratorVersion": "1.14.8",
     "osType": "Linux"
    }
  ]
...

However looking at the aks_basic.intermediate.json file it looks like this:

...
        "agentPoolProfiles": [
          {
            "name": "agentpool",
            "count": 3,
            "vmSize": "Standard_DS2_v2",
            "osDiskSizeGB": 100,
            "storageProfile": "ManagedDisks",
            "maxPods": 110,
            "osType": "Linux"
          }
        ],
...

[wwwil]

Confirmed the problem! I'm not sure why I was not getting full info previously with curl, but when that started to work I looked into our code again and noticed the JSON from the API is unmarshalled into a struct that is missing the type field.