Stop Feeling Intimidated by Complexity

Let me tell you the conclusion up-front: Complexity is not inherently bad. The complications are the real enemy.

Call it wishful thinking, but I have never been intimidated by complexity. Indeed, I was trained to embrace it during my studies and academic work.

This applies not only to computer science or programming, but also to social sciences. For example, the idea that economic systems are complex, non-linear dynamical systems is more realistic and exciting to me than the equilibrium models of economics.

The first requires a whole new body of knowledge and methods, even from seemingly unrelated disciplines, and how they play together, while the second one is just a set of equations. I have seen how an entire economy can be modeled and simulated using basic premises of economics, and how the results can be used to explain phenomena that are not easily explained by the equilibrium models. It was a fun journey, and I am glad I took part in it.

I welcome complexity. It is not complexity that drains us -- it is the unnecessary complications we create.

Complexity and Complications on a Terrain

Complexity is about the essence of the problem. It is real. Some problems are inherently complex. Life is complex, weather systems are complex, concurrency is complex, versioning operational data is complex, and so on. They are facts, and any solution that genuinely solves such problems will have to deal with complexity deservedly.

Complications, on the other hand, are artificial. We create them by choice, often unknowingly. They are the results of our poor decisions and actions: Ignorance, under-analysis, over-analysis, lack of communication, incoherent documentation, wrong resource allocation, tools mismatch, and so on. Blaming the complexity of the problem is often an excuse, a way to cover for our poor choices or shallow thinking.

Complexity is the first settlement in the village. Complications are the neighborhoods we built around it. One day, we wake up in a tangled city, where we barely find the traces of the first settlement.

The Complexity of the Simplicity

The way we deal with complexity is to analyze it into simpler parts, until we reach the trivial. Solving the trivial is what ultimately solves the problem. Therefore, the essence of simplicity in our context is hard and deep.

An analysis that yields mostly edge cases is simplistic. We do not address the actual complexity of the problem, but usually the symptoms of it. Guess what happens next? Each edge case is then a new problem to solve -- by us or by others.

On the other hand, an analysis that produces trivials disconnected from the original problem is ultimately redundant. This results in too powerful solutions which are to be maintained -- again, by us or by others.

Biggest Source of Complications: Beliefs

How do we know that our analysis of the problem is correct, sufficient and complete?

I have no grand answer to offer. But, I will make two grand suggestions which I keep repeating to myself.

Firstly, we need the right tools to analyze the problem.

You might think this is just common sense. Mind you, I did not say, "the right tools to solve the problem". Unless we see the bottom of the analysis, we should not even begin to think about solutions.

There is a great danger here. I have rarely met a programmer who is not opinionated about tools. Opinions are the result of beliefs. Beliefs can be too strong to be challenged, even more so by the believer himself. I am not immune to this.

Secondly, pick the tools which will survive you.

We do not have the luxury to pick and learn new tools every time we encounter a new problem. Worse, we may have to come up with solutions which demand even newer tools. This is a vicious, energy-draining cycle.

Keep testing your beliefs about the tools you use. Your tools may not survive the next problem -- that is, your beliefs may eventually fail you.

Conclusion

However dry or boring this topic might seem, I keep finding new challenges which excite me to explore new ideas and tools about working on complex problems and avoiding complications.

For example, I can tell you from experience that aligning our tools for analyzing problems with those for solving them is difficult, but worth the effort.

Where do we start?

Let us stop feeling intimidated by complexity itself, by those who appreciate it, and by the tools they use.