Getting the abstraction layers right

Ever since I started my software engineering career, I have constantly been learning to prepare myself for the next growth milestone. The question is learning what?

Transforming yourself into a learning machine is necessary to.. get started. It is a prerequisite, not a promise for results. Perhaps the promise lies in knowing what to learn. As Chris Do from The Futur, puts it:

Ones can wander under the sunlight unharmed. But if the light was put through a magnifying glass, the concentration can be so intense it can burn through a leaf.

In a world of scarce resources, especially time, focus matters. Focus puts all the necessary skills into an irresistible offer, which will make an impact in your domain of choice.

In practice, it is easier said than done. At the start of my career, what to learn is obvious. You can look around and immediately see what others can do that you are not yet capable of. Those low-hanging fruits can run out quickly, if the success formula is so simple, we would all be where we want to be right now.

When you lose sight of the next target, it is easy to fall into panic mode and try to arm yourself with everything and anything you can get your grip on. It is one type of FOMO, I don't know what will prepare me for the next promotion, but I don't want to miss the chance, I will prepare for everything.

During my career, I have dived deep into a wide range of topics. From classic programming concepts such as clean code, database design, object oriented principles, to deeper stack such as network handshake, OAuth flows, PDF specifications, to new shiny technologies and frameworks such as Angular, MVVM, NoSQL.

I don't think learning all those things was a waste of time, but I can't see any one of them contributing directly to the next jump (well, maybe Angular does). Those topics genuinely gripped my attention, and I picked up useful nuggets here and there, but it was like finding needles in a haystack. I felt like I was doing the opposite of focusing.

Until one day I heard of the concept of abstraction layers from Jim Keller, the legendary microprocessor engineer. The way he puts it, I just got it, and everything clicks for me. As a brilliant computer architect, Jim says:

When things are beautifully done, usually, there's a well thought out set of abstractions layers. Two different components can work together without knowing what's the other one is doing.

He went on to mentioned the example of the 7 layer network stack. He mentioned the innovation was really when they broke up that right, so that you can do something new in layer 7 without knowing how layer 4 works.

Also from Jim, beautiful design in human system works the same way. You need to design the layer, e.g. expertise domain, to match the skills and expertise of the people and teams. But each layer is not big enough to make an impact in the world, so you need a framework to coordinate all layers harmoniously.

That is modular design, it makes perfect sense. Once I put all the wandering, seemingly random learning through the lense of modular design, it all clicks. It takes deep understanding to do modular design right, and I was learning deeply in a various programming domains.

Now I can't say I have mastered the art of setting up proper abstraction layers, but at least I have gotten better at designing interfaces, breaking down outward facing APIs, compartmentalizing different feature modules, etc. Time will prove if all these time and effort investment will pay off, but for now, this will be my North Star for developing my career.

If something is important, it's worth repeating. When design is done beautifully, a well thought out stack of abstraction layers empowers different teams to work independently while contributing to the bigger goal. That is distributed focuses, the next level of institutional focus.