How to Assess and Improve DevOps
By Bob Aiello and Dovid Aiello
Many of my esteemed colleagues are sharing their approaches to implementing continuous delivery and other DevOps best practices. Their approach often assumes that the organization is willing to completely discard their existing processes and procedures and start from what we often refer to as a “green field”. Internet startups can afford to take this approach. But, if you are implementing DevOps in a large financial services firm or a growing ecommerce retailer, you may not be able to stop the train while we lay down new tracks and switches. In my work, I have always had to help teams improve their CM, ALM & DevOps best practices while simultaneously supporting the existing production operations – introducing new processes without disrupting the continuous implementing of new releases and patches to support the business. If you would like to succeed in the real world, then you need to be able to assess your existing practices and come up with a strategy for improving while you still support a vibrant ongoing business. This article explains how to conduct the DevOps assessment.
Too often, DevOps practitioners begin the journey by administering a questionnaire that seeks to assess an organization’s DevOps process maturity. The development managers who typically provide the answers usually become quite adept at gaming the questionnaire. For all appearances, they are doing just great at managing their source code, automating builds and delivering releases to production on a fairly rapid basis. However, if you speak to the folks doing the actual work, you may get a very different impression.
My assessments start with a pretty robust dance card that includes everyone from the CTO to the engineers who do the actual work. If you are going to implement DevOps, make sure that your dance card includes both developers and the operations engineers. You will also want to speak with the data security engineers, QA testers and, most importantly, the end users or their representatives. I usually start by asking what is going well and what could be improved. If I can get the assessment participants to trust me then I can usually identify which practices really need to be improved and which ones should not be changed – at least not in the beginning.
It is important to start by respecting the fact that the organization has evolved and adapted organically, establishing many practices and procedures along their way. You are there to identify what should be improved, but make sure that you start by understanding that the existing approaches may very well be there for good reasons. I believe that you need to respect the journey that the organization has taken to get to this point and then you can focus on what should be improved.
I usually interview participants individually or in small groups and I ask what is going well and what could be improved. I probe with an ear to identifying their understanding and application of the six core configuration management best practices of source code management, build engineering, environment management, change control, release management and deployment engineering. I then try to help the team determine specific initiatives that should be addressed on their journey to implementing DevOps.
My report synthesizes common themes and helps to highlight which items are more urgent than others. It is essential to create an environment where everyone is comfortable being open and truthful so I never report on what one particular person has stated – instead my results group items along the six core CM best practices. One practice consideration is to pick one or two items that can be achieved easily and quickly. Don’t tackle the tough ones first. After you have achieved some small successes, addressing the bigger challenges will become a lot more practical.
Use the information from your assessment to help plan the roadmap for your own process improvement initiative.
Some common initiatives that I often see are:
- Need for training when adopting new technologies from Bitbucket to Ansible
- Automation of common tasks that everyone has become accustomed to doing manually
- Monitoring and addressing failed builds in the continuous integration server
- Improving coverage for automated testing from unit through functional/regression and don’t forget non-functional testing (including performance)
- Improving communication and collaboration by breaking down those silos (hint: get everyone to share a screen and show what they are working on)
- Bringing in a vendor every now and then and making sure their demo is focused on teaching best practices (instead of just a sales pitch)
- Making it cool to show off what your team is doing
Most of all, remember that change takes time, so take an agile iterative approach to process improvement. As always, view DevOps initiatives as a “team sport” and feel free to drop me a line and share your experiences!