Wednesday, 26 December 2012

BAM! The Combinatorial Explosion and Planning

Solving problems requires understanding.

Understanding is built, in part, by measuring features.

One or two features might be sufficient to describe something simple, but describing something complex often takes many more.

With one or two features, a handful of measurements may suffice to capture the behavior of whatever-it-is that we are trying to understand. As the number of features that we need to measure grows, the number of measurements that we need to properly and comprehensively capture that behavior grows faster than fast: It grows stupendously, ridiculously, unreasonably, explosively fast. It grows so fast that it is akin to ... BAM! ... hitting a brick wall.

This is the Combinatorial Explosion:- Know it, and Fear it, because it is Important.

In other words, it stops being possible in any reasonable, practicable sense of the word, to understand any system or problem, the description of which involves more than a small handful of features.

This has some terribly important consequences. Consequences that, as our world becomes more complex, and contains more and more complex, interconnected systems, we particularly need to understand and internalize, for we fail to do so at our peril.

The past becomes less useful as a guide for predicting the future; Our intuition becomes less effective; Unintended consequences and unusual, exceptional events become more prevalent; our ability to predict what will happen next is weakened to the point where it disappears, and the utility of planning (in the way that we commonly understand it) diminishes dramatically.

At no point in history did plans ever survive first contact with the enemy, but as (consciously or unconsciously) we become more liberal with complexity, these traits and characteristics will become more and more prevalent.

We need to adapt, and learn to deal with them.

PS:

This line of thinking is particularly interesting when applied to organizations:
http://daltoncaldwell.com/thoughts-on-organizational-complexity

Which, of course, then go on to produce more complex products:
http://en.wikipedia.org/wiki/Conway's_law