If you are a knowledge worker, the most important thing is to know things.  Although the idea of knowing is the basis for everything I do–from sales to marketing to programming to creativity–I had never sat down to think about it for as much as an hour.  It turns out that knowing is a lot more complicated than I at first thought.  It’s more than just “insert fact; recall fact.”  I’ve identified six rungs of knowledge, plus three freebies that aren’t really “knowledge” at all.

Level -2:  Don’t know don’t know

This is like the Dunning-Kruger effect:  I am so ignorant about a topic that I don’t even know how ignorant I am.  Most programmers have no idea how bad they are at sales, marketing, writing specs, etc.  Until the last couple of years, neither did I.

Level -1:  Don’t know; can’t find out

I experienced this recently trying to solve a problem that seemed to be to be tangentially related to the halting problem.  Specifically, I wanted to know how to group all the possible variable configurations for a program into equivalence classes so that I could write an automated theorem prover which could reason complicated things about C programs.  The alternative is “Consider the case where a = 1.  Now consider the case where a = 2…”  up to INT_MAX.  I don’t even know what to Google to find the relevant research into this topic: automatic theorem-provers C gives me no useful results.  By sheer chance, somebody directed me to control-flow and data-flow analysis,  a subfield of compiler research which has some results that might be useful to my problem, although it’s a little orthogonal.  But for all I know, there could be an entire field of CS out there with 12 different n*lg[n] algorithms already built for this.  I just have no idea how I would find them.

Level 0:  Don’t know; can find out

As an engineer or a mathematician, this is the most dangerous level of knowledge.  It is enough that you feel safe.  Once we’ve proven that an answer exists, we can stop–actually knowing what it is is usually pretty trivial.  The knowledge is just a quick Google search away, which is just a few milliseconds slower than your brain’s memory anyway, right?  The new hip thing to say is that we’re all cyborgs with our iPhones now and we don’t ever have to know anything ever again.

This approach is fine if the “knowledge” is a useless fact like the number of trombone players in Bavaria.  Unless, of course, Bavarian trombone players are the market for your software, in which case you’d better know that number!  If it turns out that you were off by an order of magnitude or two–not at all uncommon–you’re in big trouble!  I can’t count the number of times I’ve worked on a project that hinges on “Suppose there are 10,000 trombone players in Bavaria” and nobody even bothered to Google it, because the existence of an answer seemed sufficient at the time.

Level 1: Know, technically

I should really exercise more.  I should really write more unit tests.  I should get better at sales.  I know these things.  Technically.  I’m just not going to do anything differently because of it.

This level of knowledge is very bad because it’s useless knowledge.  I’ve done the hard work of learning it, but I haven’t put it to any use.

Level 2: Know, mechanically

I know, mechanically, how to calculate the wavelength of light.  Because I did it perhaps a hundred times in preparation for a physics test.  Why I would want to calculate light’s wavelength is a complete mystery to me.  I can’t imagine any situation in which this mechanical knowledge would be useful.  I’m sure it is useful, of course–I don’t mean to say “waah waah physics has no applications!”  I am just saying that mechanical knowledge is so rudimentary that I cannot generate any applications.  Even if somebody presented me a real-world problem in which the solution involved calculating light’s wavelength, I would almost certainly not recognize it as such.  Even though I can, mechanically, solve such a problem.

I’ve played in exactly one chess tournament.  In the first game, I beat my opponent using a famous four-move trick.  He was shocked–game over in 30 seconds!  In the very next game, one hour later, a different opponent beat me with the exact same trick–the one I had used just earlier!  I knew, mechanically, how the trick worked:  move this piece here, move this piece here.  But I did not understand the strategy of the trick in the slightest, and so it bested me easily.

Mechanical knowledge can also be a “fall from grace” from a deeper level of knowledge.  I memorize complicated piano pieces, but once I get to the point that I can play them effortlessly, I find that I have trouble thinking about them consciously again.  You change lanes in your car every day, but try thinking about it, and you may suddenly forget.  Mechanical knowledge failure is why people die in automobile accidents.

Level 3: Know, planning

I know I should be networking for more clients, so maybe I should go to some conference some time and hand out some business cards.  But I can’t do it today, actually, I can’t do it until we ship this product.  And it makes more sense to wait until Spring anyway.  Actually…

I know I should be getting feedback from customers, so my plan is to integrate feedback features right into our products.  Eventually.  Right after I put out this fire.  I promise.

Level 4: Know, planned

OK, OK, this time I’ve actually filed a bug to add a feedback feature to the product.  Happy now?!?

Level 5: Know, recognize cognitive bias

It seems like every time I ship a product, I realize how much better it would have been if I had spent less time implementing and doing and more time talking to customers and getting feedback.  This has happened often enough that I realize it’s a cognitive defect.  The things I worry about are the things I am trained to worry about:  bugs, code quality, is the technical problem solvable, what’s the right technology/platform.  I worry about these things, partly because that’s the training of a programmer, but also partly because I read anecdotes about business guys who make boneheaded technical decisions.

It turns out that I’m that boneheaded guy, except I make dumb marketing and sales decisions, and MBAs write anecdotes about me.  Unless I suddenly go into medical supplies, nothing bad is going to happen if the product has a few bugs.  We will never hit a technical problem we can’t solve.  You don’t have to worry much about code quality when you’re a good developer to begin with.  In short, all the things I worry about are wrong.

What kills projects are:  Are there enough customers?  Can they find out about the product?  Did I talk to enough customers?  Did I talk to the right customers?  I will spend eight hours debugging a weird problem that reproduces only on iPod Touch 1G running OS 2.1 in Germany, but I won’t spend eight minutes talking to a customer to validate the product or network for a new gig.  Incredibly dumb.

And each time this happens I say “I’m serious this time, we’re going to spend more time worrying about customers.”  And I do, a little, and it gets incrementally better, and we build better products, but in spite of knowing that I underestimate the value of customer feedback I continue to marginally-more-realistically underestimate the value of customer feedback even taking my underestimation into account.

Level 6:

So I had a meeting today about my cognitive bias and how important it was that we spend a lot of time thinking about networking, about marketing, about sales, etc.  We had a big realization about how I was solving problem X by thinking like an engineer, and really I should be thinking about customer acquisition instead of worrying about bugs.  I walk right from that meeting into writing a spec for a new product.

That product is basically:  inspired by Joel, I wanted to get alerts when our iOS applications crash.  iTunesConnect does most of the work for you, by collecting crash reports and displaying them–but you have to log into your iTunesConnect account every day and manually click through a bunch of things, hit a “Refresh…” button, wait awhile.  And I “Level 1: Know, technically” that I should be looking at the crash reports, but I’m not, so this is useless knowledge.  And, presumably, there are other iOS developers out there who know, technically, they should be looking at crash reports and aren’t, so… instant market!  Even if there aren’t, it would be useful to us, I don’t want to log in to twelve accounts to check crash reports on dozens of apps each day, so…built-in customer! Let’s spec it!

I am ashamed to say that I got over two hours and 6 mockups into the spec–including a data model and deciding on a platform for the app–before I even looked at a crash report, because I needed one to resolve some design decision about the storage model.  Turns out–get this–there was only one crash report for any production version of any of the 12+ apps that I looked at, several of which haven’t been updated since 2009.  Not just one crashy app.  Not just one type of crash.  One crash.  Ever.  Now this is interesting in itself:  why is there only one crash?  Is it because we write really good software?[1]  Is it because there’s a bug in Apple’s crash collector?  Do we have less active users than I thought?  Are there large groups of users who never sync with iTunes?

But that’s kinda tangential to this post.  Knowing that I have a cognitive defect for failing to spend enough time collecting feedback, knowing that I was in fact the customer, I didn’t take enough time even to collect feedback from myself! I didn’t even ask myself how I was going to use the product, or what problem it was going to solve, even though I knew that was the most important thing and that I would overlook it, not ten minutes after coming out of a meeting about how This Time, Seriously, We’re Not Going to Overlook It (TM).

Now obviously I caught the error in the design phase, two hours in.  (Six months ago I would have caught it four months after launch).  So that’s an improvement.  And I also don’t mean to imply that the product is a bad idea per se.  It turns out that being alerted about even one bug report might be a big deal, or could be a bigger deal to other developers who write crappy code, or whatever.   So I don’t mean to imply that the product is scrapped–it’s certainly worth continuing (*cough*, starting) to collect feedback about (in fact, if you want crash reports e-mailed to you daily, you should send me an e-mail).

So I guess the sixth level of knowledge is instinctive knowledge.  It’s removing the cognitive defect.  It’s being able to take customer development and making it a core and natural part of my decisionmaking the same way that reasoning about technical problems is automatically and naturally a core part of my decisionmaking.  I’m just not sure how to get from here to there.  Suggestions?

[1] Although I think it’s a combination of several things, it’s probably most likely this if I can only pick one.  I focus too much on code quality (unit tests, automation tests, static checks), so crashing bugs rarely leave the building.  Plus, Apple’s QA team.  Also, by definition, the latest production version is the one that it most likely to be bug free, so looking at the production version can be deceptive in terms of the historical quality of the software.

---

Want me to build your app / consult for your company / speak at your event? Good news! I'm an iOS developer for hire.

Like this post? Contribute to the coffee fund so I can write more like it.