Wednesday, 26 December 2012

Stupid is better than Smart (A call for humility)

Software development is a funny thing. It is full of nonlinearities and counterintuitive results.

Here is one of them: It is better to think of oneself as stupid (and be right) than it is to think of oneself as smart (and be wrong).

This sounds nonsensical, doesn't it? Surely it is better to be smart than it is to be stupid. Particularly since we spend so much of our time trying to demonstrate to other people just how smart we really are?

Well, if we were to start thinking of ourselves as smart people, relative to the rest of the population, then it is all too easy to start thinking of ourselves as being smart relative to the problems that we are trying to solve.

This is a problem, because *everybody* is pretty stupid in the grand scheme of things, and hubris is dangerous, particularly in the presence of complexity.

Complexity makes systems difficult to understand and manage, and difficult to fix when they go wrong. It is also difficult to gauge complexity, and humans have a consistent tendency to underestimate the complexity of unexplored functionality.

Let us consider a (mis) quote that illustrates the point:

"Debugging a system is harder than designing it in the first place, so if you are as clever as you possibly can be when you are designing the system, you are (by definition) too stupid to debug it."

The well known Dunning-Kruger effect applies here too: Because we tend to think that we are smarter than we really are, we tend to design systems that are too complex for us to debug and maintain.

In cases such as this, it is helpful to take a broader view. We may call somebody "Smart", but this term is defined relative to other humans, not relative to the problems that we need to solve.

We are frequently faced with problems that would be considered difficult by the very best of us. It is not a sign of weakness to acknowledge that; to treat the problems that we are trying to solve with the deference and respect that they deserve.