feat(azuredevops): implement work items as issues #7809
Conversation
dosubot
bot
added
size:XXL
There was a problem hiding this comment.
Could you please check the GitHub Actions workflow? Some of the checks are failing e.g the golangci-lint check
| @@ -0,0 +1,2 @@ | |||
| id,params,data,url,input | |||
| 1234,"{""OrganizationId"":""johndoe"",""RepositoryId"":""0d50ba13-f9ad-49b0-9b21-d29eda50ca33"",""ProjectId"":""test-project""}",com.intellij.database.extractors.TextInfo@d75361af,https://dev.azure.com/johndoe/test-project/_apis/wit/workitemsbatch?%24skip=0&%24top=0&api-version=7.1 | |||
There was a problem hiding this comment.
Could you please check the value of the data column? com.intellij.database.extractors.TextInfo@d75361af appears to be an export error to me.
| thisRequestBody := func(reqData *api.RequestData) map[string]interface{} { | ||
| return map[string]interface{}{ | ||
| "ids": currentWorkItems, | ||
| "fields": []string{"System.Id", "System.Title", "System.TeamProject", "System.Description", "System.Reason", "System.AreaPath", "System.WorkItemType", "System.State", "System.CreatedDate", "System.ChangedDate", "System.CreatedBy", "System.AssignedTo", "Microsoft.VSTS.Scheduling.Effort", "Microsoft.VSTS.Common.Priority", "Microsoft.VSTS.Common.Severity"}, |
There was a problem hiding this comment.
I think it makes sense to make this configurable for the end user. Due to the dynamic nature of Azure DevOps, it is very hard to make assumptions about which property is used and which is not. If I got it right, Microsoft.VSTS.Scheduling.StoryPoints is used in the Agile template instead of Microsoft.VSTS.Scheduling.Effort.
|
|
||
| const RawWorkitemsTable = "azuredevops_go_api_workitems" | ||
|
|
||
| var CollectWorkitemsMeta = plugin.SubTaskMeta{ |
There was a problem hiding this comment.
If I'm not mistaken, this task runs for each data scope linked to a DevLake project, meaning we read all work items in an Azure DevOps project n times if n data scopes are attached. This can quickly deplete the quota for large projects with many work items.
| RequestBody: func(reqData *api.RequestData) map[string]interface{} { | ||
|
|
||
| return map[string]interface{}{ | ||
| "query": `SELECT [System.Id] FROM workitems WHERE [System.TeamProject] = "` + data.Options.ProjectId + `" ORDER BY [System.ChangedDate] DESC`, |
There was a problem hiding this comment.
Integrating an additional time filter condition in the WHERE clause can help us to integrate the project-level sync policy settings and provide an incremental mode. There is a nice example in the jira implementation:
| thisRequestBody := func(reqData *api.RequestData) map[string]interface{} { | ||
| return map[string]interface{}{ | ||
| "ids": currentWorkItems, | ||
| "fields": []string{"System.Id", "System.Title", "System.TeamProject", "System.Description", "System.Reason", "System.AreaPath", "System.WorkItemType", "System.State", "System.CreatedDate", "System.ChangedDate", "System.CreatedBy", "System.AssignedTo", "Microsoft.VSTS.Scheduling.Effort", "Microsoft.VSTS.Common.Priority", "Microsoft.VSTS.Common.Severity"}, |
There was a problem hiding this comment.
Can you please check if all of the required fields are actually being used? System.Reason is a very special one that is also not mapped to the Devlake Issue domain type.
| rawWorkItems.ForEach(func(key, value gjson.Result) bool { | ||
| currentWorkItems = append(currentWorkItems, value.String()) | ||
|
|
||
| if len(currentWorkItems) == 9 || key.Int() == int64(len(rawWorkItems.Array())-1) { |
There was a problem hiding this comment.
Can you please explain how you came up with this condition? Is this a restriction of the /workitemsbatch API?
Summary
This PR adds support for Azure DevOps work items as issues. The main thing to consider is that in Azure DevOps, work items are connected to projects and orgs, not repositories. Furthermore, current API definitions in Azure DevOps have a few limitations on retrieving work items data. Finally, work item queries are heavy and consume a lot of TSTUs, meaning that a simple query can impact overall organization usage of Azure DevOps and block access to services.
Thus, the decision was to:
Does this close any open issues?
Closes #5996
Screenshots
Other Information
With all work items collected, and their boards stored as component, having incidents mapped is as easy as querying work items with
type = bug.