Complex Systems: Generalists and Specialists
The following <a href=”https://twitter.com/#!/saschabates/statuses/119793462261452800”
tweet</a> from @saschabates that appeared in my stream this morning caught my attention:
#surgecon emergent theme: complex systems cannot be effectively diagnosed without smart generalists who understand them end to end
This statement is correct (otherwise it wouldn’t have been a theme emerging out of one of the best tech conferences). However it caught my attention not because it’s right - but because it can be hugely misinterpreted.
It's not saying that smart generalists are the only key to diagnosing problems in complex systems - it's saying that without smart generalists operating a complex system is nearly impossible. The key to running a system successfully is having a balanced mix of generalists and specialists. Generalists are usually necessary but not sufficient.
There are generally two types of roles in tech ops - generalists and specialists. A DBA, for example, is initially a specialist. And so is a network engineer. Specialists focus on some part of a bigger system; they have detailed knowledge about all interactions between components within their area of expertise and usually will have significant understanding of how their part integrates into the whole system. Their view is from inside out.
Generalists, on the other hand, approach the system from reverse angle - from outside in. Their focus is on interaction of components within the system; while they study behavior of the system as a whole, they will inevitably develop deeper understanding of individual components but the depth will vary by component.
Why can’t we all be generalists - know everything, interchangeable, able to effectively resolve all issues by ourselves without any help? The answer is simple. You can know tech. You can master troubleshooting techniques based on logic. But in a sufficiently complex system (which usually is changing at quite a fast pace), you won’t have enough time to accumulate enough experience with each component to become effective at running it sole-handedly.
While the idea “let’s all be generalists” could sound appealing, it’s not achievable in general case - the more complex a system gets, the less likely one will be able to become both a generalist and a specialist in it.
But that’s not all. At times an attempt is made to run a system with specialists but without generalists. This approach also doesn’t work universally. While it could work in smaller teams, sooner or later it breaks down as the number of people increases. Generalists are called in to help direct the specialists.
The bottom line: both generalists and specialists are key to successful operation of a complex system. These are distinct skills even if they are assigned to the same individuals. Size of the overall tech ops team and complexity of the system at hand play a decisive role in deteremining when it’s a good time to branch out into separate generalists and specialists.