Merge pull request kubernetes#53182 from itowlson/azure-blobdiskcontr…

…oller-retry-logic

Automatic merge from submit-queue (batch tested with PRs 53444, 52067, 53571, 53182). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Retry when checking  Azure storage account readiness

**What this PR does / why we need it**: When the Azure cloud provider ensures that a default storage container exists, if the storage account exists but is still provisioning, it exits without retrying.  This is a bug as the code is wrapped in a backoff policy but never signals the policy to retry.  This PR fixes this behaviour by returning values which allow the backoff policy to operate.

**Which issue this PR fixes**: fixes kubernetes#53052

**Special notes for your reviewer**: Not sure how to test this - I have done a deployment using acs-engine and it seems to work but I am not sure of the best way to exercise the failure path.

**Release note**:

```release-note
NONE
```

pkg/cloudprovider/providers/azure/azure_blobDiskController.go

@@ -471,20 +471,22 @@ func (c *BlobDiskController) ensureDefaultContainer(storageAccountName string) e
 
 			if err != nil {
 				glog.V(4).Infof("azureDisk - GetStorageAccount:%s err %s", storageAccountName, err.Error())
-				return false, err
+				return false, nil // error performing the query - retryable
 			}
 
 			if provisionState == storage.Succeeded {
 				return true, nil
 			}
 
-			glog.V(4).Infof("azureDisk - GetStorageAccount:%s not ready yet", storageAccountName)
-			// leave it for next loop/sync loop
-			return false, fmt.Errorf("azureDisk - Account %s has not been flagged Succeeded by ARM", storageAccountName)
+			glog.V(4).Infof("azureDisk - GetStorageAccount:%s not ready yet (not flagged Succeeded by ARM)", storageAccountName)
+			return false, nil // back off and see if the account becomes ready on next retry
 		})
 		// we have failed to ensure that account is ready for us to create
 		// the default vhd container
 		if err != nil {
+			if err == kwait.ErrWaitTimeout {
+				return fmt.Errorf("azureDisk - timed out waiting for storage account %s to become ready", storageAccountName)
+			}
 			return err
 		}
 	}