Behaviorally Speaking – Dysfunctional Ops
By Bob Aiello
IT Operations plays an important role in any organization by ensuring that systems are always functioning as they should and all services meet the needs of customers and other end users. Some developers get frustrated when trying to work with their operations colleagues and may even try to bypass IT Ops whenever possible. My experience has been that developers may loose faith in the competency of the operations team – leading to doubts about their abilities. Are developers just a bunch of undisciplined wild cowboys or are there good reasons why many developers feel that they have to try to avoid working with Ops whenever possible? The truth is that operations, in many large organizations, is incapable of handling their role and developers are well-justified in trying to bypass Ops. In this article, we are going to have a tough conversation about the reality that many operations organizations lack qualified personnel, are more focused on blocking progress than delivering value to customers and exhibit many other dysfunctional behaviors. How did we get to this point where operations is often so dysfunctional?
Computer operators were once solely focused on handling decks of punch cards, data tapes and handling very large print jobs. The job focused on operating the equipment and dealing with any environment issues such as keeping the computer room nice and cold for the sake of the machines themselves. Back then developers were much more skilled than computer operators and required many more years of training in technical areas, from machine code programming to intensive work in languages including Fortran, COBOL and PL/1. By contast, many of today’s operations engineers are just as technical and skilled as any developer, often focusing on provisioning complex cloud-based runtime systems, environment monitoring while designing and implementing fully automated application deployment pipelines. Unfortunately, some operations engineers are not up to the demands of the job and developers may feel justified, and even motivated, to bypass operations whenever possible. In my own work, I have come across operations engineers who lacked even the most basic Unix command line skills, including use of the VI editor. This might seem trivial, but these same engineers had the root (e.g. super user) passwords, which means that they did have the potential to make serious mistakes, which could take down large-scale production systems. Why would IT operations management hire people who lacked the basic skills necessary to successfully administrate systems? Some organizations allocate management compensation based upon the number of people reporting into a function. When this happens, managers are incentivized to have large teams with lots of head count, without the more important focus of whether or not these engineers have the right level of knowledge, skills and abilities.
It has been my experience that when operations engineers lack sufficient technical skills, developers loose faith and resort to working around the operations group instead of partnering with them to get the work done successfully. Operations engineers themselves may exhibit counterproductive coping skills to deal with their limited expertise. Some teams focus narrowly on one specific skill which they understand and can execute successfully. They may end up adopting a siloed mentality where they maintain a very narrow focus, do not share their knowledge and procedures and consequently lack the broader systems knowledge required for success. I have seen many large operations organizations, which consisted of small, specialized siloes which could not effectively meet their objectives even within the operations organization itself.
The best operations groups consist of highly skilled engineers with strong cross-functional capabilities. Having experience with development, particularly scripting and automation, is an absolute must-have. When these colleagues work on deployment automation, we often call these technology professionals DevOps engineers. Going forward, operations must employ engineers with sufficient knowledge, skills and abilities to get the job done. Do you agree or disagree? Feel free to drop me a line to share your opinion!