The Ghost of Swift Bugs Future

Update: I wrote this with Xcode 7 β1, and playgrounds crashed a lot at the time. As a result, I gave up on testing all the cases, and a lot of errors creeped into the snippets. They are now corrected, thanks to (among others) @CalQL8ed_K_OS and @IanKay, who both corrected me and shamed me into fixing things. Thanks guys!

So Swift 2 is out, and they fixed enums with variable payloads, so the party is on.

I haven’t had a chance to play with it too much, but watching the Protocol-Oriented Programming in Swift session, a particular construct struck me as the most likely source of arcane, incomprehensible bugs in the future. I expect it to be the novice’s crucible, similar to the way deallocated delegates would lead to crashes in the days before the weak attribute was introduced. I’m not yet sure what the searches will look like, but the fundamental question will be a variation of:

“Why does the method that I wrote overriding protocol extension X never get called?”

Stack Overflow will no doubt provide short answers. Here is my longer, more in-depth answer, hoping to explain the details to some lost soul.

• • • • •

Protocol extensions are a new feature in Swift that allows default implementations of methods to be shared across types that conform to the protocol. The exact semantics place them in the vague mixin/typeclass area that other languages implement in different ways—I don’t claim to know enough about the semantics of the construct in any language to get more specific than that. It allows for shorter code and better modularity, and I think it is a fantastic feature. (And if you haven’t watched the session, go do that. Now. I’ll wait.)

However.

However, the dispatch rules can get confusing. To say the least.

Let’s begin with a quick intro; suppose we declare a protocol:

protocol Formattable {
    /// A string.
    var content:String { get }

    /// An formatting function. 
    func formattedContent() -> String
}

As stated, a protocol extension allows us to provide a default implementation for formattedContent():

extension Formattable {
    func formattedContent() -> String {
        return self.content
    }
}

But another thing we can do is add entirely new methods to our protocol:

extension Formattable {
    func debugFormattedContent() -> String {
        return "Content: \(self.content)"
    }
}

And now, any type that implements Formattable has access to both methods. The Swift standard library uses this to have one centrally defined sorting algorithm, dependent on a type conforming to the Comparable protocol. This is very different from how things were in Swift 1.x, where defining a sort([Self]) method on Comparable would require every type to have its own distinct implementation—hence the existence of large numbers of top-leve functions like map, reduce, and, indeed, sort.

Once you understand how protocol extensions work, their power is impressive, and their allure irresistible. But as with all siren songs, dangers lurk. Consider this implementation of Formattable:

struct Day : Formattable {

    var content:String

    func formattedContent() -> String {
        return "Today is \(self.content)"
    }

    func debugFormattedContent() -> String {
        return "Day: \(self.content)"
    }
}

Seems simple enough; it overrides both methods. Now see if you can predict what happens with each of these calls:

let a = Day(content:"Monday")
let b:Formattable = Day(content:"Monday")

a.formattedContent()
b.formattedContent()
a.debugFormattedContent()
b.debugFormattedContent()

Given it a go? Did you come up with:

a.formattedContent() // "Today is Monday"
b.formattedContent() // "Today is Monday"
a.debugFormattedContent() // "Day: Monday"
b.debugFormattedContent() // "Content: Monday"

If yes, and you understand why, congratulations; you don’t need to read any further. If you expected b.debugFormattedContent() to be the same as a.debugFormattedContent(), read on.

• • • • •

In Objective-C and Swift 1.x a protocol, which could not be extended with methods, was an interface that defined behavior that a type conformed to. That is, the conforming type guaranteed that it implemented every function in the protocol. So in Swift 1.x, suppose we had:

protocol A {
    func m1() -> String
}

struct B : A {
    func m1() -> String {
        return "hello"
    }
}

Then let a = B() and let a:A = B() were the same thing as far as calling m1 is concerned: they will both call the only implementation available, which is the one in B. This is still true in Swift 2.

Protocol extensions, however, allow a form of polymorphism, which brings up the question of which method to dispatch. Suppose in Swift 2 we extend A:

extension A {
     func m1() -> String {
        return "hello"
    }

    func m2() -> String {
        return "planet"
    }
}

And suppose we want B to override both methods:

struct B : A {
    func m1() -> String{
        return "greetings"
    }

    func m2() -> String {
        return "earthling"
    }
}

Now, for any instance of B : A, there are two possible implementations of m1 and m2: the one defined in B, and the one defined in the A extension. Which one should we choose?

Suppose we decide we should always choose the one in the type that implements A, if it exists. And suppose we have another type C defined as follows:

struct C : A {
    func m2() -> String {
        return "darkness, my old friend"
    }
}

Now suppose we have a heterogeneous array and call the methods:

let a:[A] = [B(), C()]
let b = a.map {$0.m1()} // ["greetings", "hello"]
let c = a.map {$0.m2()} // ["earthling", "darkness, my old friend"]

This is what we would expect. So far so good. Suppose, however, that we call a function:

func callM2(arr:[A]) -> [String]{
    return arr.map { $0.m2() }
}

let a:[A] = [B(), C()]
let b = a.map {$0.m1()} // ["greetings", "hello"]
let c = callM2(a) // ["earthling", "darkness, my old friend"]

This time, the result, which is the same, might actually surprise the author of callM2 very much. After all m2 is not defined in the original protocol A. So if they chose to call it, it must be because they expected the specific implementation of the extension.

By always calling the value in the type’s implementation, then, we forever hide the default implementation of the extension, even in cases where it would be expected. The solution Swift 2 adopted is to call the default implementation when the protocol is explicitly specified. So let’s look back at our example:

let a = Day(content:"Monday")
let b:Formattable = Day(content:"Monday")

a.debugFormattedContent() // "Day: Monday"
b.debugFormattedContent() // "Content: Monday"

Since b is explicitly specified as being a Formattable, the method that gets called is the default implementation in the extension. Since a is instead inferred to be a Day, the method that gets called on it is the implementation in Day.

In the case of the A and B protocols, this translates into Swift always calling the default implementation for the elements of the array:

func callM2(arr:[A]) -> [String]{
    return arr.map { $0.m2() }
}

let a:[A] = [B(), C()]
let b = a.map {$0.m1()} // ["greetings", "hello"]
let c = callM2(a) // ["planet", "planet"]

This brings up an interesting question. Is it possible to get the type’s implementation every time, instead of the extension’s? Or have we lost that ability forever? If you look back at the call to m1(), you’ll see that it is always the type’s implementation that gets called. The difference is that m1 is declared in A, whereas m2 is declared in the extension. So if we wanted m2 to be called on the actual type of the array’s elements, we would declare it in A.

• • • • •

The rules for dispatch for protocol extensions, then, are:

Note the use of “runtime type” in the first THEN clause. This refers to the type of the variable when the program is actually running, as opposed to the type the compiler infers. This is relevant if we write a function callM1() as follows:

func callM1(arr:[A]) -> [String] {
    return arr.map($0.m1)
}

Here, the compiler doesn’t know the type of the elements of arr; it only knows that they all have an m1 method. At runtime, however, each element will have a specific type, perhaps B, or perhaps C. That’s when the method that gets called will be determined. This is referred to as dynamic dispatch.

By contrast, in the callM2 implementation, per the rules we just established, the compiler will know exactly what method to call: the one in the extension to A. This is referred to as static dispatch and, incidentally, allows for certain optimizations (general rule: if you know something at compile time, odds are you can optimize something with that knowledge).

• • • • •

So, that’s it. As I said, I expect this to be a very common error among beginners; in fact, I expect to run into this issue multiple times myself. It’s just too easy to forget when things are dispatched dynamically and when they’re dispatched statically.

Incidentally, if anyone knows of other reasons to dispatch statically on extensions, please let me know; I’ll amend the post to reflect them.

 
625
Kudos
 
625
Kudos

Now read this

Addendum: Deriving the Third Monad Law From Nested Comprehensions

The third monad law declares that the following identity must hold for a monad M, where a:M<A>, f: A -> M<B>, and g: B -> M<C>: (a >>=- f) >>=- g == a >>=- { b in f(x) >>=- g } To motivate... Continue →