chore: backport latest VMs pool, HasInstance(), cloud-provider-azure defork, misc inconsistencies, for 1.27

[comtalyst]

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

These differences have effectively disabled cherry-pick automation in these areas. Upcoming changes (e.g., more VMs pool implementation, cloud-provider-azure integration rework, more fixes that need to be done in all versions) will be agonizing experiences, not to mention the past ones.
This PR will amend some of those differences, which is mostly VMs pool and HasInstance() implementations. They cannot be cherry-picked in normal means easily due to conflict with Azure defork operations, which have to resolve these conflicts at some point anyway. In addition, VMs pool and cloud-provider-azure are required to be cherry-picked.

Some of the inconsistencies are that older versions (1.27, 1.28) have went through linter. This PR might undo that and result in a temporary decrement in quality. However, the trade-off we get is consistency, and has been decided to be worth it. Then all branches will go through linter at the same time later on.

Which issue(s) this PR fixes:

• [BUG] Cluster autoscaler bug requires Azure specific implementation to resolve Azure/AKS#4286
• CA fails to scale-up or cancel in-progress scale down when there are un-schedulable pods  #4456

Special notes for your reviewer:

Original HasInstance() implementation PR: #6956
Original cloud-provider-azure defork PR: #6947

The first half of the commits are basically copy-pasting code from 1.29 while trying to keep differences in features (e.g., vmTypeAKS support that is deprecated from 1.29 and beyond, differences in provisioning states handling). Given that 1.29 is already matured, it is reliable enough.
The second half of the commits are cherry-pick of cloud-provider-azure defork, with some modifications to support this version (e.g., use cloud-provider-azure v1.27, replacement for nonexistent MaxDeploymentsCount and KubernetesRateLimit for VMAS and AKS pools respectively, alternative function to support workload identity when cloud-provider-azure still don't support it #5390). Verified that this cloud-provider-azure version has support for AADClientCert.

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[comtalyst]

/label tide/merge-method-squash
/hold

[k8s-ci-robot]

@comtalyst: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-autoscaler-e2e-azure	2b8881b	link	false	/test pull-cluster-autoscaler-e2e-azure

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[Bryce-Soghigian]

/approve
/lgtm

[feiskyer]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Bryce-Soghigian, comtalyst, feiskyer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/OWNERS [feiskyer]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[comtalyst]

/unhold