Replace go-azure-helpers with azidentity

[thomas11]

Overview

This PR implements #3493 which is part of Epic #3576, Replace deprecated REST and auth packages in Azure Native.

The goal is to replace outdated and deprecated libraries that receive no bug fixes, block us from fixing some issues, and might pose security risks.

Parts

The legacy authentication setup is in auth.go. This PR adds auth_azidentity.go in parallel, with the same purpose: read all auth-related configuration, decide on the correct authentication method, and initialize it.

The core library used is Azure's official azidentity. It has various FooCredential types that all return an azcore.TokenCredential, abstracting the authentication method being used.

Thanks to the existing AzureClient abstraction, which has an azcore implementation, we can simply pass the new TokenCredential from auth_azidentity.go there without further changes.

One place where we have to plug in the new auth backend manually is the getClientToken RPC method in provider.go.

Rollout

For the sake of caution, the new authentication backend is off by default, behind feature flag PULUMI_USE_LEGACY_AUTH (defaulting to true). It's not 100% clear yet how and when we'll decide to turn it on.

Testing

New unit tests should be self-explanatory.

Existing integration tests inherently cover authentication, as long as they're run with the feature flag. For that purpose, there's a new GH workflow azcore-scheduled.yml. It runs every night and can be run on demand via workflow dispatch as well. Note that this means that the tests that are part of this PR's checks do not run on the new backend, only azcore_scheduled does.

I expanded the existing go-azure-in-azure test to use the provider binary under test even on the remote VM, and to create two user-managed identities. That forces the test program to configure which one should be used, increasing test coverage of user-managed identities.

A missing chunk of test coverage I'm aware of is that there's no integration test using the az CLI for authentication. It's quite tricky as the only interesting authentication method here is interactive user authentication via the Azure portal. The CLI can also do service principal authentication but we already test that.

Missing parts

1. The CLI integration test mentioned above.
2. Our two custom Azure storage resources, StorageAccountStaticWebsite and Blob, are built using an obsolete Azure SDK that only works with the legacy authentication stack. I've started on updating and converting it but kept it out of this PR since it can be committed separately. Unless that's completed, the legacy auth stack cannot be removed, though.

[github-actions]

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

[thomas11]

This works only if the test machine and the remote VM have the same arch and OS. That's the case in CI.

[codecov]

Codecov Report

Attention: Patch coverage is 58.02469% with 68 lines in your changes missing coverage. Please review.

Project coverage is 59.74%. Comparing base (350d0cb) to head (3bea772).

Files with missing lines	Patch %	Lines
provider/pkg/provider/auth_azidentity.go	60.80%	44 Missing and 5 partials ⚠️
provider/pkg/provider/provider.go	45.71%	18 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3630      +/-   ##
==========================================
- Coverage   59.86%   59.74%   -0.12%     
==========================================
  Files          69       70       +1     
  Lines        8780     8929     +149     
==========================================
+ Hits         5256     5335      +79     
- Misses       3033     3096      +63     
- Partials      491      498       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.