Wednesday, 27 June 2012

Behavioral aspects of Software Engineering

I recently started reading "Thinking, Fast and Slow" by Daniel Kahneman. This book, aside from being awesome, describes a lucid mechanism for analyzing human intellectual mechanisms.

Although I have only yet digested the first couple of chapters,  I feel compelled to write about the parts of Daniel's thesis that resonate with me the most.

His basic premise is that we can better understand human thinking if we decompose our abilities into two categories. "System 1", the "fast" system, corresponds loosely to "gut instinct", whereas "System 2", the "slow" system, corresponds loosely to "concentrated reasoning".

Immediately, I have a mental image of the fast "system 1" as being a highly parallel, wide-but-deep, horizontally-scaled collection of special-purpose systems; classic winner-takes-all type architecture.

In contrast, the slow system seems rather less easily understood; I hesitate to de-emphasize parallelism, because virtually everything in biological neural systems is highly parallel; but it seems (intuitively) to have a fundamentally iterative aspect; involving some kind of interplay between working memory and decisioning/pattern recognition systems.

The fast System 1 operates more often than we think it does, consumes much less energy, but has a tendency towards bias and an over-simplification of the world. The slow System 2 operates much less frequently than we think it does, consumes much more energy, requires mental effort and concentration, but is able to perform somewhat more complex, sequential/algorithmic tasks than the fast System 1.

Anyway, my excitement does not derive from this indulgence of my mechanomorphistic instincts, but rather from the application of this analysis to my favorite topic: Software development and developer productivity. 

My basic thesis is that highly skilled, highly productive developers make more extensive use of the fast System 1, less productive developers make more extensive use of the slow System 2.

The degree to which System 1 is preferred is a function both of experience and of the tools that are available to the developer. When faced with a new task or when one finds oneself in a new environment, frequently one has to concentrate very hard to get anything done; performing novel tasks requires a high level of slow System 2 involvement. As one acclimatizes oneself to the new problem or environment, one builds up "muscle memory", or the cognitive equivalent, and System 1 becomes gradually trained to take over more and more of the task.

To reiterate a point from one of my previous posts; software engineering is fundamentally a learning process. One comes across new problem domains, languages, tools, datasets, etc... all the time. A really good set of developer tools will revolve around the job of supercharging our ability to learn; to absorb new systems; new libraries, new concepts, new mathematics, to understand legacy code, to see patterns in data etc...

Personally, I have a very visual brain; I have always enjoyed drawing and painting, found pleasure in the intricacies of models and 3D structures.  I always studied for exams by drawing diagrams, because I could remember pictures far more easily than words. For me, the ideal developer tools will the semantic content of the subject under study and translate it into rich, intricate visual images.

Not everybody has such a focus on visual information, (although many do), so visualisation is only one possible approach out of many.

The overarching motivation behind tools like these is, in my mind, to translate concepts into a form that enable the developer to engage his instincts, his fast System 1 thinking, to learn the necessary domain-specific skills quickly and frictionlessly.

What representations do you like? What conditions enable you to quickly learn to perform at a high level? How do you make yourself better at what you do?