Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Azure devops] Collect Boards from Azure DevOps to supply incidents data for DORA #5996

Open
2 of 3 tasks
klemen-df opened this issue Aug 31, 2023 · 24 comments · May be fixed by #7809
Open
2 of 3 tasks

[Feature][Azure devops] Collect Boards from Azure DevOps to supply incidents data for DORA #5996

klemen-df opened this issue Aug 31, 2023 · 24 comments · May be fixed by #7809
Labels
type/feature-request This issue is a proposal for something new

Comments

@klemen-df
Copy link

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Use case

We use Azure repos/boards/pipelines.

The documentation says:

DORA - Median Time to Restore Service
What is this metric?
The time to restore service after service incidents, rollbacks, or any type of production failure happened.

and

DORA - Change Failure Rate
What is this metric?
The percentage of changes that were made to a code that then resulted in incidents, rollbacks, or any type of production failure.

Our 'outage' of services can be seen in our board by viewing tickets with type 'Bug'. There's no other way to report an 'INCIDENT' in Azure devops board. When something is wrong you open a ticket with type='Bug'. You can also set priority (like Critical) and add some labels (like production).

I wish to see DORA metrics in Grafana ; 'Median time to restore service' and 'Change failure rate'.

Description

This should be possible by creating a different 'scrapper' for Azure Devops.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@klemen-df klemen-df added the type/feature-request This issue is a proposal for something new label Aug 31, 2023
@klesh
Copy link
Contributor

klesh commented Sep 1, 2023

@keon94 I think we need to collect AzDo tickes/issues data for those metrics to work. Would you like to work on it? Thanks.

@klesh
Copy link
Contributor

klesh commented Sep 1, 2023

@KlemenDanfoss Hi, the Azure Devops plugin is written in Python, I would like to ask if you are interested in contributing such a feature? @keon94 could help if you have any questions.

@klemen-df
Copy link
Author

@KlemenDanfoss Hi, the Azure Devops plugin is written in Python, I would like to ask if you are interested in contributing such a feature? @keon94 could help if you have any questions.

Sorry, probably can't help with a PR

@klesh klesh unassigned keon94 Sep 1, 2023
@klesh
Copy link
Contributor

klesh commented Sep 1, 2023

Alright, let's collect votes first. cc. @keon94

Copy link

github-actions bot commented Nov 1, 2023

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Nov 1, 2023
@klemen-df
Copy link
Author

Anything new here?

@github-actions github-actions bot removed the Stale label Nov 2, 2023
@Startrekzky
Copy link
Contributor

@KlemenDanfoss No progress yet. You can try to share it with the Slack community and see if anyone is interested in it. Azure DevOps is implemented with Python btw.

Copy link

github-actions bot commented Jan 7, 2024

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Jan 7, 2024
@klemen-df
Copy link
Author

Any thing new here? Is this really a feature that doesn't make sense?

@Startrekzky
Copy link
Contributor

Hi @KlemenDanfoss , as a feature it makes sense. However, we don't have the bandwidth to do this recently.

@github-actions github-actions bot removed the Stale label Jan 10, 2024
@edolic
Copy link

edolic commented Feb 13, 2024

We would need this feature as well. Especially the Change Failure Rate can't be only calculated based on Health Checks or Automated Post Deployment Tests. Sometimes there are bugs which are not covered by this automated tests and a Bug is then created.

  • Is it possible to manually mark a deployment as a failure?
  • Probably the same code which exists for gitlab or other providers can be used, just the fetching of the items needs to be adapted based on Azure Devops REST Apis?

@hezyin hezyin changed the title [Feature][Azure devops] Gather information from Azure devops so Grafana boards (2) are populated [Feature][Azure devops] Collect Boards from Azure DevOps to supply incidents data for DORA Feb 28, 2024
Copy link

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Apr 29, 2024
Copy link

github-actions bot commented May 6, 2024

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.

@github-actions github-actions bot closed this as completed May 6, 2024
@slaterx
Copy link

slaterx commented May 30, 2024

Any chance we can reopen this? If given directions, I can submit a PR for this.

@d4x1 d4x1 removed the Stale label May 31, 2024
@d4x1 d4x1 reopened this May 31, 2024
@d4x1
Copy link
Contributor

d4x1 commented May 31, 2024

@slaterx This issue is reopened.
There are two implementations about AzureDevOps in DevLake so far, but they both don't collect incidents.
I think it is similar to jira.
step1: issues in Azure DevOps' board should be collected
step2: make a rule about how to regard issues as incidents. (I think @Startrekzky can help.)
step3: Update dora and AzureDevOps dashboard if neccessary.

cc @klesh

@slaterx
Copy link

slaterx commented Jun 6, 2024

Hi @d4x1, I've reviewed the repo and found out that we have two plugins - one in python and another one in go.

Based on the comment here, the Go plugin skips disabled git repos, but in Azure DevOps, it is a valid use case to have disabled repos hosting work items. I would like some guidance to understand if I can remove this skip in the Go repo, otherwise, I'll attempt the implementation in the python plugin.

The reason for my question is because we have migrated from Azure DevOps to GitHub, however the issues are still collected in the disabled Azure DevOps repos. Keen to hear your thoughts.

cc @klesh

@klesh
Copy link
Contributor

klesh commented Jun 7, 2024

@slaterx Your use case sounds reasonable to me. @mr-ks I think we should allow adding disabled repos in case like this.

@slaterx slaterx linked a pull request Aug 1, 2024 that will close this issue
@slaterx
Copy link

slaterx commented Aug 1, 2024

Once #7809 is merged, just tweak the query in grafana to obtain all bug type work items from Azure DevOps.

@klesh
Copy link
Contributor

klesh commented Aug 1, 2024

@mr-ks Would you like to take a look at the implementation? Thanks in advance.

Copy link

github-actions bot commented Oct 1, 2024

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Oct 1, 2024
Copy link

github-actions bot commented Oct 9, 2024

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.

@github-actions github-actions bot closed this as completed Oct 9, 2024
@klemen-df
Copy link
Author

PR still open...

@d4x1 d4x1 removed the Stale label Oct 15, 2024
@d4x1 d4x1 reopened this Oct 15, 2024
@Startrekzky
Copy link
Contributor

There was a discussion on the data scope definition of Azure Boards in our Slack community at: https://devlake-io.slack.com/archives/C03APJ20VM4/p1726117905373619?thread_ts=1725410497.368769&cid=C03APJ20VM4

image

@baranpost
Copy link

Hello there, are there any updates on this? The PR seems to be open since 3~ months now.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request This issue is a proposal for something new
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants