Add Personal Access Token Resource/Data Source to create/update/revok…

[cparkins]

Add the ability to create/update/revoke/get Personal Access Tokens using the 7.1-preview API. Currently this requires a custom build of azure-devops-go-api that adds these features.

In testing there is currently an issue Authenticating against the Azure DevOps API which is difficult to understand. I believe this issue is related to the fact that these endpoints require an OAuth 2.0 token:

https://learn.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/service-principal-managed-identity?toc=%2Fazure%2Fdevops%2Forganizations%2Fsecurity%2Ftoc.json&view=azure-devops#get-a-microsoft-entra-id-token

Here is an example of the error seen when debug logging is enabled:

azuredevops_personal_access_token.pat: Creating...
2024-08-21T17:03:00.896-0600 [INFO]  Starting apply for azuredevops_personal_access_token.pat
2024-08-21T17:03:00.897-0600 [DEBUG] azuredevops_personal_access_token.pat: applying the planned Create change
2024-08-21T17:03:00.897-0600 [INFO]  provider.terraform-provider-azuredevops_v0.0.1: 2024/08/21 17:03:00 [DEBUG] setting computed for "scope" from ComputedKeys: timestamp=2024-08-21T17:03:00.897-0600
2024-08-21T17:03:00.897-0600 [INFO]  provider.terraform-provider-azuredevops_v0.0.1: 2024/08/21 17:03:00 [DEBUG] setting computed for "target_accounts" from ComputedKeys: timestamp=2024-08-21T17:03:00.897-0600
2024-08-21T17:03:01.968-0600 [ERROR] provider.terraform-provider-azuredevops_v0.0.1: Response contains error diagnostic: tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/microsoft/azuredevops diagnostic_severity=ERROR diagnostic_summary=" creating pat token in Azure DevOps: API resource location 55967393-20ef-45c6-a96c-b5d5d5986a9a is not registered on https://vssps.dev.azure.com/***REDACTED***" tf_resource_type=azuredevops_personal_access_token @caller=/Users/cparkins/code/terraform-provider-azuredevops/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_detail= tf_rpc=ApplyResourceChange @module=sdk.proto tf_req_id=4e9562d5-dd4d-c121-f24f-141b2f69d312 timestamp=2024-08-21T17:03:01.968-0600
2024-08-21T17:03:01.974-0600 [DEBUG] State storage *statemgr.Filesystem declined to persist a state snapshot
2024-08-21T17:03:01.974-0600 [ERROR] vertex "azuredevops_personal_access_token.pat" error:  creating pat token in Azure DevOps: API resource location 55967393-20ef-45c6-a96c-b5d5d5986a9a is not registered on https://vssps.dev.azure.com/***REDACTED***.
╷
│ Error:  creating pat token in Azure DevOps: API resource location 55967393-20ef-45c6-a96c-b5d5d5986a9a is not registered on https://vssps.dev.azure.com/***REDACTED***.
│
│   with azuredevops_personal_access_token.pat,
│   on main.tf line 20, in resource "azuredevops_personal_access_token" "pat":
│   20: resource "azuredevops_personal_access_token" "pat" {
│
╵
2024-08-21T17:03:01.980-0600 [DEBUG] provider.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = error reading from server: EOF"
2024-08-21T17:03:01.981-0600 [DEBUG] provider: plugin process exited: path=.terraform/providers/terraform.local/local/azuredevops/0.0.1/darwin_arm64/terraform-provider-azuredevops_v0.0.1 pid=21160
2024-08-21T17:03:01.981-0600 [DEBUG] provider: plugin exited

Until these issues are resolved this code is unusable. I submitted the PR to see if I could recruit some help in getting this fully tested (I have been able to get these endpoints working without Terraform).

All Submissions:

• Have you added an explanation of what your changes do and why you'd like us to include them?
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.
• My code follows the code style of this project.
• I ran lint checks locally prior to submission.
• Have you checked to ensure there aren't other open PRs for the same update/change?

What about the current behavior has changed?

A new Resource and Data Source are added to represent Personal Access Tokens.

Issue Number:

Does this introduce a change to go.mod, go.sum or vendor/?

• Yes
• No

This change requires the addition of the v7/tokens module.

Does this introduce a breaking change?

• Yes
• No

Any relevant logs, error output, etc?

See above.

Other information

There is currently an issue with the Authentication methods that the Provider uses that requires some additional work to get this fully functional.

[cparkins]

@xuzhang3 Since this change depends on a change to the Azure DevOps Go API and that project seems to be no longer solvent could someone take over that repository and update this provider to use it?

[xuzhang3]

@cparkins how do you generate the token SDK files? The location location 55967393-20ef-45c6-a96c-b5d5d5986a9a is mapped to a configuration template for generating the request URLs, which is managed by the service.

[cparkins]

@xuzhang3 - I did a combination of things to generate the SDK file. First I used some automation tools to use the OpenAPI definitions to generate the endpoints and models. I tried 2 different tools, neither of which seemed to match the other files. And then I took the one that was closest and tried to match it up manually. The result can be found here: https://github.com/microsoft/azure-devops-go-api/pull/164/files#diff-b7bfa06027df021e82f51c88b0c7d4bd044c3c618faddc04f6ea1a2569cc1e82R23

After making those modifications I linked my local build to my local terraform providers and then I was able to get the result seen above. I believe this issue is related to the fact that an "On Behalf Of" OAuth Token has to be used to make this type of request. I think there may be another change for another Resource that has added OIDC support and that might be capable of solving that issue. But even if that is resolved it does not resolve the problem that the underlying library may never get updated as it seems to be defunct.

I do have a question about what you said though. What do you mean by "The location is mapped to a configuration template for generating the request URLs, which is managed by the service"? Can you show me an example of what you mean?

[xuzhang3]

@cparkins The vendor files come from the azuredevops-go-sdk. If you generated the tokens SDK yourself, you should not mix it with the . We will add the newly generated SDK directly to this project (https://github.com/microsoft/terraform-provider-azuredevops/tree/main/azuredevops/utils) This is one of the workarounds to solve the azuredevops-go-sdk issue.

Regarding the locations, you can get all the API locations by OPTIONS https://dev.azure.com/<organizatiopn name>/_apis.

[cparkins]

@xuzhang3 The repository you linked is redirecting to the repository that I have created the PR to add the Token endpoints for. Is there an example of how to do the following:

• Generate the SDK from the API Definitions (I generated my changes from the Open API JSON files)
• Add the newly generated project to this project

Any information or examples you can provide would be helpful. Or if you could take the work that I have done and branch it
to show me in code I would appreciate that.

The API Specifications that I used were from here: https://github.com/MicrosoftDocs/vsts-rest-api-specs

[xuzhang3]

@cparkins As I know the azuredevops-go-sdk was generate based on the API specs but not based on https://github.com/MicrosoftDocs/vsts-rest-api-specs, @nechvatalp for more details.
Additionally, we added internal SDK support in the ADO project as a workaround when we found that the azuredevops-go-sdk did not cover our requirements (https://github.com/microsoft/terraform-provider-azuredevops/tree/main/azuredevops/utils).