Skip to content
This repository has been archived by the owner on Jul 2, 2024. It is now read-only.

DEVPROD-4166 Add breaking commits option in previous commits dropdown #2254

Merged
merged 24 commits into from
Feb 29, 2024

Conversation

ZackarySantana
Copy link
Contributor

@ZackarySantana ZackarySantana commented Feb 9, 2024

DEVPROD-4166

Description

  • Adds a new option to the previous commits selector called "breaking commit".
    • The breaking commit should be enabled when the current task's status is failed and there is a previous passing task.
    • It should point to the task that made it go from passing -> failing.
    • This is targeted for mainline commits, not cli patches. For cli patches it is grayed out.
  • Refactors "previous commits" -> "relevant commits" (as per design; gives more context to what the user should expect when opening it)

Designs

Note that the designs include a guide cue and task metadata addition- but those have been separated to different tickets.

Screenshots

Simple Mainline Commit Example:
https://github.com/evergreen-ci/spruce/assets/64446617/89f7d4c4-004f-4f0a-a0b0-48d93beb4f32

Complex Mainline Commit Example (Stepback Bisection):
https://github.com/evergreen-ci/spruce/assets/64446617/2353e1ba-00f8-4ee6-bbed-73596b4505f9

CLI Example:
https://github.com/evergreen-ci/spruce/assets/64446617/9b6bb2da-a153-4456-a942-0e54457f2a81

Testing

Manual testing with different commits/tasks and appended info to existing tests

@ZackarySantana ZackarySantana self-assigned this Feb 9, 2024
Copy link

cypress bot commented Feb 9, 2024

5 flaky tests on run #15969 ↗︎

0 547 10 0 Flakiness 5
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.

Details:

Merge branch 'main' into DEVPROD-4166
Project: Spruce Commit: c71b07f5f4
Status: Passed Duration: 19:57 💡
Started: Feb 29, 2024 2:59 PM Ended: Feb 29, 2024 3:19 PM

Review all test suite changes for PR #2254 ↗︎

@ZackarySantana ZackarySantana requested a review from a team February 13, 2024 17:17
@khelif96
Copy link
Contributor

@ZackarySantana is this pr open for review? You tagged the team but the title still says [NOT FOR REVIEW]

@ZackarySantana ZackarySantana changed the title DEVPROD-4166 Add breaking commits option in previous commits dropdown [NOT FOR REVIEW] DEVPROD-4166 Add breaking commits option in previous commits dropdown Feb 15, 2024
@ZackarySantana
Copy link
Contributor Author

ZackarySantana commented Feb 15, 2024

Yes! It is, sorry about that

Copy link
Contributor

@minnakt minnakt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good so far!


describe("previous commits", () => {
describe("relevant commits", () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to add a test in here for the new "breaking commit" button? :0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added to the existing tests instead of a new one and added some data! I think it makes sense to test it like the others but I am new to testing and if you feel otherwise I can switch it up

@@ -42,6 +46,7 @@ export const PreviousCommits: React.FC<PreviousCommitsProps> = ({ taskId }) => {
buildVariant,
displayName,
projectIdentifier,
status: baseStatus,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you may still want to call this status because it's the status of the current task being viewed (rather than its base task status)!

Suggested change
status: baseStatus,
status,

baseStatus === TaskStatus.Succeeded,
variables: {
projectIdentifier,
skipOrderNumber: passingOrderNumber + 2,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the +2 being used to work around the $lt operator that's in use here? I think a comment might be helpful since +2 is a bit unusual

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is! I can add a comment

skip:
!parentTask ||
!lastPassingTask ||
baseStatus === undefined ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious if this is needed! I feel like the task status should not be undefined but let me know if you encountered a scenario where it was 👀

Suggested change
baseStatus === undefined ||

Copy link
Contributor Author

@ZackarySantana ZackarySantana Feb 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2024-02-20 at 4 08 54 PM

It starts as undefined and then updates - but that's because the baseTask starts as undefined and an empty object is used.

I did some more testing and I think it is needed even though the other conditions usually cause this query to skip too but I'm only worried about a race condition- i.e. parentTask and lastPassingTask load before the current task.

EDIT: Screenshot is a useEffect with only status, I did some more testing with the other skip variables that I didn't show since it's more confusing

@@ -22,6 +22,7 @@ query LastMainlineCommit(
}
}
id
order
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can get the order from a task as well (in case that makes passingOrderNumber calculation easier i.e. lastPassingTask.order). Just something I wanted to call out!

Copy link
Contributor

@minnakt minnakt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry it took me some time to get back to you!

Comment on lines 101 to 102
status === undefined ||
status === TaskStatus.Succeeded,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it — In that case, I wonder if you instead want to check that the status isn't a failing task status! I think that you'll want to use the function isFailedTaskStatus, which you can import at the top of the file like:

const { isFailedTaskStatus, isFinishedTaskStatus } = statuses;

I think this should eliminate the need to specifically check for undefined! 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I don't think this would eliminate it since we should check for !isFailedTaskStatus which undefined would return true, then the ! would make it false and not skip for undefined status.
I think I could keep the undefined check or switch it to do status ?? TaskSucceeded (not sure how to reference task success status but however it is)- but that may be confusing. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like isFailedTaskStatus(status) should return false for an undefined status, and then the !isFailedTaskStatus(status) should make it true to skip. Is that not the case for the function? 👀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, not sure what I was thinking when I wrote the above... Thank you for being thorough!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, not sure what I was thinking when I wrote the above... Thank you for being thorough!

Comment on lines 192 to 195
sendEvent({
name: "Submit Previous Commit Selector",
type: CommitType.Breaking,
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we want to rename the analytics event as well. It would cause us to "reset" analytics on this, but it's not something we query often anyways. (Personally I think it's okay to rename 👀)

Copy link
Contributor Author

@ZackarySantana ZackarySantana Feb 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be down to switch it! If it helps properly track analytics for it, but I'm not sure of the process or what they should look like. I was just trying to emulate the other event sendings. I think if it lets us tally and filter by the type field I think we could keep it the same

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline; will change the analytics name to Relevant rather than Previous


// The breaking commit is the first failing commit after the last passing commit.
// The skip order number should be the last passing commit's order number + 1.
// We use + 2 because internally the query does a less than comparison.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest to leave a permalink here to the GitHub line:

We use + 2 to work around the underlying query which uses the $lt operator (link here).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a link but please let me know if I did it right! I used the HEAD commit currently of the current branch so it's unchanging in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline; will change link to link to the query in the Evergreen repo

@@ -156,6 +185,19 @@ export const PreviousCommits: React.FC<PreviousCommitsProps> = ({ taskId }) => {
>
Go to {versionMetadata?.isPatch ? "base" : "previous"} commit
</MenuItem>
<MenuItem
as={Link}
disabled={breakingLoading || breakingTask === undefined}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more of a UX comment, but when testing this on larger projects (like mongodb), it can take a while for the query to finish. I think it could be good to have some indication that the query is still loading (maybe a loading spinner)?

You could make a separate ticket to explore this, since it wasn't covered by the original designs! Or you can experiment (but only if you want to)!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm that might be a good idea. I think adding a loading indicator to the relevant task button might be useful but from my testing, usually it loads as fast as the main page data. I think it would be nice if we did think of some standard for links/buttons/sections that are in the process of loading but we have sent a query for them.

I thought it could be a less grayed out button, italicized, some small icon next to it, or a loading spinner next to it. I just tried some with like italicized but I'm not confident in how they look so I think I'll leave it up for future discussion

Copy link
Contributor

@minnakt minnakt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm mod applying the changes that we discussed offline! great work!


// The breaking commit is the first failing commit after the last passing commit.
// The skip order number should be the last passing commit's order number + 1.
// We use + 2 because internally the query does a less than comparison.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline; will change link to link to the query in the Evergreen repo

Comment on lines 192 to 195
sendEvent({
name: "Submit Previous Commit Selector",
type: CommitType.Breaking,
})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed offline; will change the analytics name to Relevant rather than Previous

@ZackarySantana
Copy link
Contributor Author

I am not used to these mocked queries, I just finally understood them and got a working/sensible test case for the breaking commit selector!

@ZackarySantana ZackarySantana merged commit d43ca82 into evergreen-ci:main Feb 29, 2024
5 of 6 checks passed
@ZackarySantana ZackarySantana deleted the DEVPROD-4166 branch February 29, 2024 15:23
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants