Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: issue-5850 - Settle override conflicts between edges and propagate changes #7025

Open
wants to merge 10 commits into
base: latest
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions workspaces/arborist/lib/edge.js
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ class Edge {
const newTo = this.#from.resolve(this.#name)
if (newTo !== this.#to) {
if (this.#to) {
this.#to.edgesIn.delete(this)
this.#to.deleteEdgeIn(this)
}
this.#to = newTo
this.#error = null
Expand All @@ -273,7 +273,7 @@ class Edge {
detach () {
this.#explanation = null
if (this.#to) {
this.#to.edgesIn.delete(this)
this.#to.deleteEdgeIn(this)
}
this.#from.edgesOut.delete(this.#name)
this.#to = null
Expand Down
95 changes: 93 additions & 2 deletions workspaces/arborist/lib/node.js
Original file line number Diff line number Diff line change
Expand Up @@ -1344,9 +1344,100 @@ class Node {
this.edgesOut.set(edge.name, edge)
}

addEdgeIn (edge) {
recalculateOutEdgesOverrides () {
// For each edge out propogate the new overrides through.
for (const [, edge] of this.edgesOut) {
edge.reload(true)
if (edge.to) {
edge.to.updateNodeOverrideSet(edge.overrides)
}
}
}

findSpecificOverrideSet (first, second) {
for (let overrideSet = second; overrideSet; overrideSet = overrideSet.parent) {
if (overrideSet.isEqual(first)) {
return second
}
}
for (let overrideSet = first; overrideSet; overrideSet = overrideSet.parent) {
if (overrideSet.isEqual(second)) {
return first
}
}
console.log('Conflicting override sets')
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wraithgar This case means that the same node has two edges with completely different override sets pointing to it. This log is useful for debugging, not sure if just to remove the line or there's some other convention used in the arborist.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

proc-log is what we use to log in npm.

log.silly is probably what we want here, defined as

Extremely noisy excessive logging messages that are typically only useful for debugging.

}

updateNodeOverrideSetDueToEdgeRemoval (otherOverrideSet) {
// If this edge's overrides isn't equal to this node's overrides, then removing it won't change newOverrideSet later.
if (!this.overrides || !this.overrides.isEqual(otherOverrideSet)) {
return false
}
let newOverrideSet
for (const edge of this.edgesIn) {
if (newOverrideSet) {
newOverrideSet = this.findSpecificOverrideSet(edge.overrides, newOverrideSet)
} else {
newOverrideSet = edge.overrides
}
}
if (this.overrides.isEqual(newOverrideSet)) {
return false
}
this.overrides = newOverrideSet
if (this.overrides) {
// Optimization: if there's any override set at all, then no non-extraneous node has an empty override set. So if we temporarily have no
// override set (for example, we removed all the edges in), there's no use updating all the edges out right now. Let's just wait until
// we have an actual override set later.
this.recalculateOutEdgesOverrides()
}
return true
}

// This logic isn't perfect either. When we have two edges in that have different override sets, then we have to decide which set is correct.
// This function assumes the more specific override set is applicable, so if we have dependencies A->B->C and A->C
// and an override set that specifies what happens for C under A->B, this will work even if the new A->C edge comes along and tries to change
// the override set.
// The strictly correct logic is not to allow two edges with different overrides to point to the same node, because even if this node can satisfy
// both, one of its dependencies might need to be different depending on the edge leading to it.
// However, this might cause a lot of duplication, because the conflict in the dependencies might never actually happen.
updateNodeOverrideSet (otherOverrideSet) {
if (!this.overrides) {
// Assuming there are any overrides at all, the overrides field is never undefined for any node at the end state of the tree.
// So if the new edge's overrides is undefined it will be updated later. So we can wait with updating the node's overrides field.
if (!otherOverrideSet) {
return false
}
this.overrides = otherOverrideSet
this.recalculateOutEdgesOverrides()
return true
}
if (this.overrides.isEqual(otherOverrideSet)) {
return false
}
const newOverrideSet = this.findSpecificOverrideSet(this.overrides, otherOverrideSet)
if (newOverrideSet) {
if (!this.overrides.isEqual(newOverrideSet)) {
this.overrides = newOverrideSet
this.recalculateOutEdgesOverrides()
return true
}
return false
}
// This is an error condition. We can only get here if the new override set is in conflict with the existing.
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wraithgar This case means that the same node has two edges with completely different override sets pointing to it, which means that this node can't really decide which one is correct.
Our options are:

  1. Go with the flow - in the current logic the out edges continue with the first overrides set they got, so this isn't breaking comparing to the existing logic. But it's obviously the wrong behavior.
  2. Raise an error - just fail the command. Not sure how you do that in the arborist.
  3. Duplicate the node - this is the same package with different override requirements. The proper logic is to duplicate the node, and then dedup it later if there's no conflict.

Currently I chose (1), but maybe it should at least show some kind of warning.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A hard-won lesson we learned with peer dependencies is that 1 is not the right choice here. npm 6 can install completely invalid trees, and this was fixed in npm 7. --legacy-peer-deps can turn the behavior back on, but again it will install trees that have incorrect peer dependencies if there was a conflict. npm should never guess the user's intent.

2 is the easiest solution, especially given that 3 may eventually fail if we end up with an already duplicated node with conflicting override requirements.

So, unless there's a really elegant way to do 3 that does't require an extra mountain of effort, 2 is best here. We'll throw with as much context as possible to help the user build a working override set, but we won't build an invalid tree.

This also leaves the door open later to implementing 3, since we can gradually add smaller, specific cases where 3 will work, and the fallback to 2 will already be in place.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how you do that in the arborist.

Just throw an Error object. If we want npm to format the message in a special way based on any other attributes you assign to the object we can do that. A good example of that is here, with the cli's custom error output switch here.

}

deleteEdgeIn (edge) {
this.edgesIn.delete(edge)
if (edge.overrides) {
this.overrides = edge.overrides
this.updateNodeOverrideSetDueToEdgeRemoval(edge.overrides)
}
}

addEdgeIn (edge) {
// We need to handle the case where the new edge in has an overrides field which is different from the current value.
if (!this.overrides || !this.overrides.isEqual(edge.overrides)) {
this.updateNodeOverrideSet(edge.overrides)
}

this.edgesIn.add(edge)
Expand Down
37 changes: 37 additions & 0 deletions workspaces/arborist/lib/override-set.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,43 @@ class OverrideSet {
}
}

childrenAreEqual (other) {
if (this.children.size !== other.children.size) {
return false
}
for (const [key] of this.children) {
if (!other.children.has(key)) {
return false
}
if (this.children.get(key).value !== other.children.get(key).value) {
return false
}
if (!this.children.get(key).childrenAreEqual(other.children.get(key))) {
return false
}
}
return true
}

isEqual (other) {
if (this === other) {
return true
}
if (!other) {
return false
}
if (this.key !== other.key || this.value !== other.value) {
return false
}
if (!this.childrenAreEqual(other)) {
return false
}
if (!this.parent) {
return !other.parent
}
return this.parent.isEqual(other.parent)
}

getEdgeRule (edge) {
for (const rule of this.ruleset.values()) {
if (rule.name !== edge.name) {
Expand Down
Loading
Loading