S14: Issues in the roles spec

[Ovid]

Many people have read Traits: Composable Units of Behaviour (Composable), but the authors were targeting a Smalltalk audience and made assumptions in this regard which means that some concepts do not translate as well as they could (source: personal email with researchers).

Unfortunately, most people have not read Traits: The Formal Model (Formal), which is an even more important paper.

To truly understand these issues I'm raising here, I would suggest being familiar with both, but I won't assume familiarity.

Background

Traits were partially a response to the failure modes of AOP, an interesting concept which took the reviled concept of "Action at a Distance" and made it a first class concept in programming. However, to make it work, "aspect aware" editors were generally required. These editors would often assume a static language where much information could be inferred at compile-time, such as whether a pointcut applied to a join point and would highlight that in the editor. Thus, if you didn't have an aspect aware editor, you could easily make changes that would fundamentally alter the behavior of the software.

For example, a friend of mine was trying to fix some Java code that would sometimes spit out the wrong answer. What he finally discovered was that someone had used aspects to lock methods in threaded code, but because they were using pointcuts, a programmer had renamed a method and the pointcut no long matched that method, meaning that in threaded code, it was no longer locked. It was literally impossible to read the class and determine that there was a bug, aside from noticing that this threaded system didn't appear to be locking any methods.

With traits, in the "Composable" paper, conflicts are specified as resolving using two precedence rules:

1. Class methods take precedence over trait methods.
2. Trait methods take precedence over superclass methods.

It's the first rule which assumed a Smalltalk environment and, like AOP, required a "trait aware" Smalltalk browser. However, this was omitted from the paper. In Perl 5, when roles were introduced, that was believed to mean that a class silently overrode conflicting trait methods. However, that is not what was intended. From my email with the traits researchers (shared with permission):

Stéphane Ducasse: in the smalltalk implementation the browser is showing us that we are overriding a trait method.

This makes perfect sense because in Smalltalk, the browser and the language are integrated directly together. Further, from Andrew P. Black:

Yes, it is really important that a programmer can see clearly when a trait method is being overridden — just as it is important that it is clear when an inherited method is being overridden.

In Smalltalk, where a program is viewed as a graph of objects, the obvious solution to this problem is to provide an adequate tool to show the programmer interesting properties of the program. The original traits browser did this for Smalltalk; the reason that we implemented it is that traits were really NOT a good idea (that is,they were not very usable or maintainable) without it. Since then, the same sort of "virtual protocols" have been built into the browser for other properties, like "overridden methods".

Similarly, I agree that if the normal Perl environment does not provide good programming tools, then the implementation of traits for Perl ought to have done something to make it clear that a class method was overriding a trait method.

Thus, due to some unspoken assumptions in the Composable paper, many others have gone down the path of silently discarding behavior that may be critical. This has bit me in my time at the BBC working on a large-scale metadata system and when we're talking about large systems, it makes maintainability harder.

Today

Here's a trivial example of the issue in Perl 6 (note that the following example does not use multis, but we'll cover their problems later):

subset NonEmptyString where *.chars > 0;
role Alpha {}
role Beta does Alpha {
    method foo(NonEmptyString $arg) {
        return "Beta::foo:$arg"
     }
}
class MyClass {
    also does Beta;
}

my $object = MyClass.new;
say $object.foo("something");

That works as expected. However, a programmer might decide to add critical behavior to Alpha:

role Alpha {
    method foo(NonEmptyString $arg) {
        # critically important security code for consumers of Alpha
    }
}

In the above code, not only will that critical behavior never be called, but those consuming the role (even indirectly) won't have the opportunity to realize that the critical behavior has been added. It's the same problem as with multiple inheritance when someone adds security constraints to a class which are silently bypassed by a child multiply inheriting. It's important for the developer to know this is happening and have a chance to make an intelligent decision.

Associativity and Commutativity

In both MI and mixins, we have ordering problems. With the former, "first wins" and with the latter, "last wins". This has long been the bane of developers and is one of the many reasons why many programming languages disallow MI outright.

In the Formal paper, section 3.4 says this:

Proposition 1. Symmetric composition is associative and commutative.

Associative:

( a + b ) + c = a + ( b + c )

Commutative:

a + b = b + a

In other words, no matter how you mix and match roles, for a given set (formal term) of roles, no matter how they're ordered and mixed, the programmer is guaranteed the same behavior. However, that is not true for Perl 6, even though the S14 spec says "There is no ordering dependency among the roles."

(As an aside, some might point out the counter-example of a role with an excluded method, but technically, a role is defined by both its name and the methods it provides, so a role with an excluded method is a different role. That makes $object.does($foo) a bit more complicated)

subset NonEmptyString where *.chars > 0;
role Alpha {
    multi method foo(NonEmptyString $arg) { return "Alpha::foo::$arg" }                                                                                                                                                               
}

role Beta {
    multi method foo(NonEmptyString $arg) { return "Beta::foo:$arg" }
}

class MyClass {
    also does Beta;
    also does Alpha;
}

my $object = MyClass.new;
say $object.foo("something");

In the above example, Alpha::foo::something is printed. If we reverse the order of role composition, Beta::foo::something is printed. Thus, roles which implement conflicting multimethods are order dependent. Some programmer thinking that role ordering isn't important (as the spec says), can sort the roles by name to make it easier to find them and silently change the behavior of the program.

The associativity rules are also broken. Consider the following example:

subset NonEmptyString where *.chars > 0;
role Alpha {
    multi method foo(NonEmptyString $arg) { return "Alpha::foo::$arg" }
}

role Beta {
    multi method foo(NonEmptyString $arg) { return "Beta::foo:$arg" }
}

role Gamma does Beta {}

class MyClass does Beta does Alpha {}

my $object = MyClass.new;
say $object.foo("something");

(Note that Gamma is declared, but not consumed).

That prints Alpha::foo::something. However, if you change the class to this:

class MyClass does Beta does Alpha does Gamma {}

That prints Beta::foo::something because Gamma implements Beta and it's now consumed after Alpha. This is not a simple case of "the developer implementing the class should know what Gamma implements" because this behavior may well not have existed at the time MyClass added Gamma.

I've also created similar examples by putting my roles in separate files and having a consuming Gamma, consume Alpha which consumes Beta (all without multis) and then consuming Beta which has been switched to consuming Alpha, such that you get different answers, even though the final class hasn't changed. When working on larger systems, having such role ordering issues is problematic.

Recommendation

This is very simple: rather than silently discard behavior, issue a warning when one class/role is going to override a method provided by a role and allow the programmer to manually resolve the issue.

[Ovid]

Per jnthn's request, an example without multis or subsets:

role Alpha            { method foo() { return "Alpha::foo" } }
role Beta  does Alpha { method foo() { return "Beta::foo" } }
role Gamma does Beta  {}

class MyClass does Gamma {}

my $object = MyClass.new;
say $object.foo;

That prints Beta::foo because &Beta::foo has silently overridden &Alpha::foo. Now let's have Gamma consume Alpha instead of Beta.

role Beta             { method foo() { return "Beta::foo" } }
role Alpha does Beta  { method foo() { return "Alpha::foo" } }
role Gamma does Alpha {}

class MyClass does Gamma {}

my $object = MyClass.new;
say $object.foo;

That prints Alpha::foo. Note that the consuming class is unchanged, and the roles it consumes are the same, but because of the silent override, behavior is different. This makes refactoring more dangerous.

[colomon]

Let me give a practical example. (doing this on my iPad, forgive me if
there are formatting issues.)

role Real { method sign() { self < 0 ?? -1 !! self > 0 ?? 1 !! 0 } }
role Rational does Real { method sign() { numerator.sign } }

ie Real has a very generic version of sign, and Rational a more specific
and presumably efficient version.

A) I'm hoping this is a valid, even encouraged, use of roles. (If it not,
we've got some serious redesigning to do in p6's core.)

B) In this example Real does Rational would have a completely different
meaning.

[Ovid]

@colomon But that's OK. In typical multi dispatch the more specific type wins. Let's say I have this:

class Mammal { ... }
class Cat is Mammal { ... }

class SomethingElse {
    multi method eat(Cat $kitty) { ... }
    multi method eat(Mammal $food) { ... }
}

# later
my $something = SomethingElse.new;
$something.eat($some_cat);

That should dispatch to the method which has a Cat as an argument because it's more specific. That's how multiple dispatch works, so your example is fine and doesn't contradict my example.