Peopls say that the state of Python packaging/dependency management/package managers are awful. Those people have obviously never done package management with iOS.

Tools like CocoaPods exist to “manage” this problem. The reality is however that instead of shooting yourself in the foot, CocoaPods gives you a machine gun sentry that can obliterate all feet in a 1-mile radius.

It’s not really CocoaPods’ fault. The problem is that the problem is hard, and you can’t just hide it behind an easy-to-use interface and pretend that it’s an easy problem because we have a one-button interface.

The woes of a compiled language

Probably the biggest single problem is this: we work in a compiled language. Whereas the languages that have more reasonable package managers, like Python and Ruby, are all interpreted.

Fundamentally, a Python package is installed simply by copying some files to the right location. That’s really, at the end of the day, all there is to it. There is a folder on your computer where all the packages live, and if you copy files to it you can pretty much install your own packages. When your package manager is basically a glorified xcopy, there’s only so much trouble you can get into ¹.

Meanwhile, in iOS, at the end of the day, software is distributed in the form of a compiled binary. And so by hook or by crook, you have to figure out how to get your app and its 14 libraries compiled and linked into 1 binary executable. This is harder than it sounds.

For example, suppose libraries are distributed as binaries themselves. Now, every time Apple releases a chip with a new instruction set (like 64-bit support for example) you’re beholden to 14 people who must compile and build a new binary in order for you to be able to ship a 64-bit version of your app. You don’t have any commercial relationship with these people, so some of them probably won’t recompile, or at least won’t in any reasonable time frame.

So instead of doing that, you decide “well what if I compiled this myself, and built my own 64-bit version?” Well, that’s not a bad plan. Except that replicating 14 people’s build environments is pretty brutal. Just take the number of build breaks that you have on your own project, and multiply by 14. One person inexplicably has Crashlytics or some other random build script embedded in their xcode project. Another has set up code signing certificates that you don’t have. A third person wants to disable ARC (KISSMetrics, I’m looking at you) for their project. A fourth person wants -ffast-math turned on.

Notice how e.g. Python doesn’t really even allow this at all. If you want your Python module’s math to be faster than other people’s math, tough shit. Python doesn’t have an option that lets you change how division works, or change how memory works, or other things about the language itself, for your little module. You get the same Python that everybody else has, period. ² Meanwhile in iOS you can mix and match 7 different C dialects, 6 different C++ dialects, and Swift. And those are just the coarse choices; the granular ones like whether or not statics are threadsafe push the combinatorial space into the thousands.

Targets, targets, targets

The next problem is, what does the interface between the app code and the library code look like, on a low level? Well in Python world, you just say “I want package foo“, and the Python interpreter looks in the packages folder and finds some sourcecode (or bytecode) that purports to be foo.

In a world where all code lives in the same code section of the same executable, there’s a lot more to it. Here’s just a taste of all the ways you can package a library for iOS:

• You could just drag the source code into your app’s project. Now it ships as part of your binary, but probably not with the build settings that the author intended.
• You could have some kind of xcodeproject, that builds a .a file (static library), and some .h (header) files, deposit all this in some location, and then configure your app’s project to find them in that location. The words “header search paths” and “library search paths” may ring a bell. That’s probably been a special circle of hell for you at some point. Oh and funny story, this option doesn’t work with Swift.
• New in iOS 8, you could have some kind of xcodeproject, that builds a .framework, deposits that in some location, and then configure your app’s project to find them in that location. The error message dyld: Library not loaded: Reason: image not found may ring a bell. BTW, Dropbox doesn’t know how to sync frameworks, so if you’ve ever had your source code in Dropbox for some reason, LOL.
• You could have an open source library like openssl that builds with make, and via some insane and Rube Goldbergian process you cross-compile it 5 ways till Sunday and then archive it into a universal static library and then pray Xcode can find it and its headers.
• You can patch your Xcode install to produce a weird kind of ‘framework’ that is sort of, but not quite, the iOS 8 kind, and requires subtly different build settings to use in your app, LOL. And the words “Framework Search Paths” may start giving you nightmares.

Or, pro tip: you could do most of these, and just keep fiddling with things until it works. That’s what real professionals do.

Think I’m joking? Facebook Pop’s official installation instructions cover at least 3 of these methods vaguely, and their remarks include such gems as this:

For some unknown reason, Xcode simply rejects adding pop.framework as an embedded binary when pop.xcodeproj is placed in the workspace. This only works when pop.xcodeproj is added as a subproject to the current target’s project.

The guy who invented one of these methods reports that:

I haven’t been able to solve the problem of deeply nested projects within projects… Unfortunately, I don’t have the time to solve the last 10% of use cases.

So some quick math for you: If you depend on 14 libraries, and each of them is 90% likely to work, then all of them are only 22% likely to work. And that’s as good as an expert can do, with the time he has available.

Dependency hell

So let’s say you’re working in Python, and you need a package. And that package needs another package. Basically what happens in that case is you have to install both packages. And if you have two packages, that disagree about what version they want of some other package, tough shit. This is usually what people complain about when they complain about Python packaging.

Well in iOS, not only is it completely normal to have two libraries that each ship their own version of a third library. But you pretty much have to, because generally speaking a library is self-contained and the build product includes all of its dependencies.

What makes this more fun is that even though on a logical level what you have is two libraries that each contain more libraries, like folders deeply nested in a filesystem, in reality the way this is implemented is that you have a flat top-level namespace that contains all the libraries in one flat list. So if Library A, Library B, and your app all use some particular JSON library, conceptually there is probably one copy of that JSON library and which version it is, is a fascinating question.

So fascinating that the Objective-C runtime doesn’t even know the answer. Sometimes you get logs like this:

Class MYCLASSNAME is implemented in both EpicPath1 EpicPath2. One of the two will be used. Which one is undefined.

Other times the compiler will notice this situation and give you errors about “duplicate symbols for architecture armv7s”. Good luck figuring out this error is because Vendor1 and Vendor2 are internally using the same JSON library.

Also, I lied a bit. I told you that the namespace of libraries is flat. That’s true… for certain combinations of compiler flags. Remember how everybody compiles their project a little bit differently? Well… they can control the flatness of the library namespace during compile too.

Oh, you wanted to know which build setting to flip? As if there were just one. That’s cute. Let me help you out. On the library itself, “Hidden symbols by default” is involved. But we can’t just set this in one place, no. That wouldn’t be confusing enough. In your app, the “Other linker flags” values -ObjC and -all_load and -force_load are involved. Naturally, the -ObjC value produces a flat namespace for the ObjC language, -all_load produces a flat namespace for the C language, and -force_load produces sort of a flat namespace.

And as if that wasn’t confusing enough, for a very long time there was a bug in -ObjC that caused it not to work. So people got in the habit of not really understanding how any of it is supposed to work, because it didn’t, like at all. Now that it does work, the habit of ignorance has stuck.

As a result of this world of pain what ends up happening is people just start fiddling with -ObjC and -all_load combinations until it builds. Then, when they add another library and it breaks, they fiddle again. Great fun is had by all.

And as if that wasn’t bad enough, the kinds of collisions you can have are not even on a per-library basis. They’re on a per-class basis, and sometimes even a per-method basis. That means if you have two libraries, that happen to both have a class called Vector (cause that would make sense), you’re basically hosed. This is, by the way, why most things in iOS have strange prefixes like NS or UI or CF. Because 2 letter prefixes should totes be enough for everybody. ³

Enter CocoaPods

What CocoaPods does is it looks at this complete clusterfuck and decides, what we need to do is automate this, do some of the heavy lifting and try and standardize this mess a little better. Well, those are okay goals.

The problem is that they are unachievable goals. No amount of applying CocoaPods is going to turn a compiled language into Python. The reason libraries are hard isn’t a “the UI is bad, let’s fix it” type problem. It’s a structural problem, that strikes at the heart of what compiled languages actually are. If you want to produce an application out of a set of libraries then the thing you need to write is called a compiler/linker, not a package manager. What you need is some way to control narrowly which symbols are exported, control the low-level interface between each library, control how headers and code are delivered together, control dependency hell with nested libraries, control what build options people are allowed to choose for their libraries, and so on. These are things that compilers do.

CocoaPods is not that. CocoaPods is a tool that downloads some libraries and makes some best-effort determination of how the compiler and build chain should actually, you know, put them together. But in order to do that it basically papers over a lot of the settings that you need to actually, um, get complex things to compile at scale.

Pop quiz. Earlier in this post we covered at least 5 ways to link libraries and projects together. Which one does CocoaPods use? Most people who use it don’t know. That is fine up until you hit some build problem and then you’re hosed. Not only is it a mystery to you how any of this even works under the hood. But you’ve also got to wrestle with CocoaPods’ clobbering your fixes with its own vision of how your project should be linked.

And even if you think you know, you may not know. For example, the answer is changing in the next release, where “initially” two methods will be supported, and some of the verbiage there suggests they’re eventually going to migrate the whole ecosystem to Frameworks, in a way that probably breaks lots of code out in the wild.

The result is that you’ve got a very complicated system that requires deep knowledge of both CocoaPods and your compiler. What CocoaPods has bought you is a simple GUI around a process that most people don’t really understand, and that lack of understanding will eventually bite you when things are more complicated than they are now. None of that is CP’s fault but it does create a net negative externality by allowing developers who don’t really know what they are doing to build up technical debt by not understanding how to compile their own software.

Now that alone is sort of tolerable, under ideal conditions. If you know for a fact that what you want is a workspace-static-library build process, and CocoaPods provides 1-button setup for that, then automating that workflow isn’t so bad. Practically that describes a small minority of the circumstances in which it is used, but I mean on paper, there’s nothing wrong with it.

However CocoaPods goes far beyond being opinionated about your build process. They have some kind of religion about using the RubyGems versioning system (Yes, for iOS libraries. wat.) Then they require you to update your list of source files in 2 places, because parsing Xcode’s format is too hard I guess?

So what we have at the end of that line of reasoning is a tool that just doesn’t feel right to me. Maybe it’s right for the intersection of people who agree with using static libraries (and agree with CocoaPods’ timeline for migrating to frameworks, whatever that is), who are satisfied without Swift support for long periods, who think Ruby Versioning is the One True Versioning, and whose idea of a good time is editing Podfiles by hand. But I am, quite frankly, not on board with any of that, let alone all of it. It’s one thing to automate a workflow; it’s a completely different thing to have a religion that extends from the workflow to your project organization to your version numbering to your compiler flags.⁴

I want to emphasize here that I think CocoaPods gets a bit of an unfair rap because it doesn’t defy the laws of physics. Orta, one of its maintainers and a conference colleague, wrote:

Every time a flash of “I hate CocoaPods” happens on the internet another person has probably been discouraged at getting involved in the larger community. You’re welcome to not like the tool, you’re also welcome to your opinion. Hating on something as mundane and necessary as a dependency manager for a language though, is not constructive.

We’re stuck right now using Apple’s tools which are not production worthy and may not be for a few months. Or maybe we have to wait a year. Who knows? I’m experiencing crashes multiple times a day trying to write a Swift app. These tools are not open, and whilst the developer community has been filling gaps we’re stuck with what we have from Apple side. We’re on the other side, we have to work together and help each other out.

I have tried to be very fair and thorough in describing that most of the problems related to library use in iOS are far beyond the scope of any package manager. Some of them are problems of Apple’s invention, many of them date back many decades to earlier eras.

However I think under these circumstances, the right answer isn’t to create an opinionated tool with One True Way to create and install packages. The right answer is to give the user a choice about how to create and install packages, and give them the information they need to make that choice. The right tool should be equally happy to use frameworks, static libraries, direct project inclusion, etc., and should provide the user some basis to decide which one works best under the situation. e.g., if you’ve got some Swift happening, maybe we steer you away from the options that don’t work with Swift. If you have duplicate symbol issues, maybe we scan for that and suggest some kind of “plain English” resolution choices, that translate into the right -ObjC -force_load incantation.

I think this isn’t done for a few reasons. One, it’s hard. It’s harder to write a tool that can do five things well then it is to do one. It’s harder to reverse-engineer .pbxproj when it breaks in each release. It’s hard to give the user actionable, relevant information when they need it. This is all fair. The thing is, either it’s worth doing it to share code with each other or it’s not. If it is, then let’s do something hard. If not, let’s go home. But I don’t see a lot of middle ground here. The CocoaPods core team are investing effort in ancillary things like building search engines for packages, doing rebrandings, etc. that could go toward some of these hard problems like multiple linker methods and reverse-engineering pbxproj.

Another reason this doesn’t happen is because conventional wisdom says that asking the user to go through a wizard to do something simple like install a package is Considered Bad UX Design (TM). Well, that is true. But I think under the circumstances, there is just no way realistically to keep the user out of the loop. The user has to make some decisions, period, in the real world. I mean if you want to adopt a zero-decision approach go write a compiler, but as far as what can be accomplished in a package manager leveraging today’s compilers, you, the user, are going to have to make decisions. Sorry. The conversation should be about how best to help you to make those decisions, rather than “have you heard the Gospel of Ruby Versioning”.

So that is my beef with CocoaPods. It’s simultaneously unambitious and overambitious. Unambitious because it doesn’t want to do the hard things, like reverse-engineer and manipulate Xcode projects. And overambitious because it seems to believe that “one-click” install is possible without going through the herculean effort of actually doing that.

Now in fairness to them, it’s getting better. The 0.36 release is going to support both frameworks and static targets. That’s a concrete improvement over the old regime (and, you know, it was a lot of work). But I think they view this as an unfortunate, accidental, and temporary increase in complexity, instead of what it should be: a deliberate design decision to support people who need more than one trick in the bag to compile a complex application.

I will also admit that I get irrationally angry whenever somebody breaks the build by committing a podfile and/or clobbering a working build process and/or playing strange games with the /Pods directory in source control. But I think that problem has more to do with the users of the project, than the project itself.

Ultimately, the problem of compiling software with dependencies is hard, and it keeps getting harder. Time was, you could conceivably get patches into LLVM on that topic, which would then trickle down to Xcode (although nobody was exactly jumping up and down to do that). These days, with no Swift source release, that is even more difficult than it was just a year ago.

To me it’s an interesting question, what would have happened in some alternate universe with an open-source Swift. Faced with the same Swift/Frameworks crisis we’re just now pulling ourselves out of, I wonder if instead of doing a bunch of expensive (and the cost is still barely beginning to be paid from the userbase) surface-level hackery with xcodebuild flags we would have dusted off the compiler textbooks and built a proper buildsystem into LLVM itself. A buildsystem that could, as Joel Spolsky taught us o’er a decade ago, make a build in one step. That would have been hard, but it would start the divorce between us and the yearly race to work around Xcode’s bug-du-jour.

I fear now that it isn’t possible. Apple is making no moves on the opening Swift front, and we have been doubling down enough on the current regime in the meantime that our pile of hacks may hold for another six months, and so again in June. As has been the tradition for all the years to the present day.

---

Want me to build your app / consult for your company / speak at your event? Good news! I'm an iOS developer for hire.

Like this post? Contribute to the coffee fund so I can write more like it.