Training – Configuration Management, DevOps and Agile ALM

+
-
0
Training – Configuration Management, DevOps and Agile ALM

 
We provide courses courses on DevOps, Configuration Management, and Agile Application Lifecycle Management.
Configuration Management: Robust Practices for Fast Delivery
(See outline below)
Configuration management (CM) best practices are essential for creating automated application build, package and deployment to support agile (and non-agile) application integration and testing demands, including rapidly packaging, releasing, and deploying applications into production. Classic CM—identifying system components, controlling changes, reporting the system’s configuration, and auditing—won’t do the trick anymore. Bob Aiello presents an in-depth tour of a more robust and powerful approach to CM consisting of six key functions: source code management, build engineering, environment management, change management and control, release management, and deployment, which are the prerequisites for continuous delivery and DevOps. Bob describes current and emerging CM trends—support for agile development, container-based deployments including Docker, cloud computing, and mobile apps development—and reviews the industry standards and frameworks available in practice today. Take back an integrated approach to establish proper IT governance and compliance using the latest CM practices while offering development teams the most effective CM practices available today.
Continuous Delivery: Rapid and Reliable Releases with DevOps
DevOps is an emerging set of principles, methods, and practices that enables the rapid deployment of software systems. DevOps focuses on lowering barriers between development, testing, security, and operations in support of rapid iterative development and deployment. Many organizations struggle when implementing DevOps because of its inherent technical, process, and cultural challenges. Bob Aiello shares DevOps best practices, starting with its role early in the application lifecycle and bridging the gap with testing, security, and operations. Bob explains how to implement DevOps using industry standards and frameworks such as ITIL v3 (IT Service Management) in both agile and non-agile environments, focusing on automated deployment frameworks that quickly deliver value to the business. DevOps includes server provisioning essential for cloud computing in what is becoming known as Infrastructure as Code. Bob equips you with practical and effective DevOps practices—automated application build, packaging, and deployment—essential for meeting today’s business and technology demands.
Agile ALM: Using DevOps to Drive Process Improvement
Many organisations struggle to improve their existing IT processes to drive their software and systems development work. This leaves technology managers and teams to use whatever worked for them on the last project, often resulting in a lack of integration and poor communication and collaboration across the organisation. Agile application lifecycle management (ALM) is a comprehensive approach to defining development and operations processes that are aligned with agile methodology. Bob Aiello explores how to use DevOps principles and practices to drive the entire process improvement effort—establishing agile practices such as continuous integration and delivery that integrates with the IT operations controls. Learn how to use DevOps approaches to iteratively define a pragmatic and real-world ALM framework that will evolve, scale, and help your organisation achieve its software and business goals. Take back a template to define and automate your application lifecycle, accounting for all stakeholders and integrating their processes into a comprehensive agile ALM framework.
Outline of CM Best Practices Class:
Configuration Management Best Practices Training Program
Introduction:
Based upon industry standards and frameworks this course introduces the technology professional to all of the principles, concepts and hands-on best practices necessary to establish a comprehensive configuration and release management functions. Discussing both CM as it is practiced in product companies and IT organizations, Bob Aiello and Leslie Sachs provide expert guidance on establishing configuration management best practices.

  1. Configuration Management Concepts
    * Configuration Identification
    * Status Accounting
    * Change Control
    * Configuration Auditing (physical and functional)
  2. Source Code Management
    * Baselines
    * Sandboxes and Workspaces
    * Variant Management
    * Handling bugfixes
    * Streams
    * Merging
    * Changesets
  3. Build Engineering
    * Versions IDs and branding executables
    * Automating the build
    * Build tools to choose from including Ant, Maven, Make and MS  Build
    * Role of the Build Engineer
    * Continuous Integration
  4. Environment Configuration
    * Supporting Code Promotion
    * Implementing Tokens
    * Practical use of CMDBs
    * Managing Environments
  5. Change Control
    * Types of Change Control
    * Rightsizing the Change Control Process
    * The 29 minute Change Control Meeting
    * Driving the Process Through Change Control
  6. Release Management
    * Packaging the release
    * Ergonomics of Release Management
    * RM as coordination
    * Driving the RM Process
  7. Deployment
    * Staging the release
    * Deployment frameworks
    * Traceability
  8. Architecting Your Application for CM
    * CM Driven Development
    * How CM Facilitates Good Development
    * Build Engineering as a Service
  9. Hardware CM
    * Managing Hardware Configuration Items
    * Hybrid hardware/software CM
  10. Process Engineering (Rightsizing)
    * Agile/Waterfall
    * Hybrid Approaches
    * Agile Process Maturity
  11. Establishing IT Governance
    * Establishing IT Governance
    * Transparency
    * Improving the Process
  12. What you need to know about Compliance
    * Common Compliance Requirements
    * Establishing IT Controls
  13. Standards and Frameworks
    * What you need to know about standards and frameworks
  14. CM Assessments
    * Evaluating the Existing CM Practices
    * Documenting “As-Is” and “To-Be”
    * Writing your CM Best Practices Implementation Plan

About the Instructor:
Bob Aiello, CM Best Practices Consulting
Bob Aiello is a consultant and author with more than thirty years of experience as a technical manager at leading financial services firms, with company-wide responsibility for CM, ALM and DevOps. He often provides hands-on technical support for enterprise source code management tools, SOX/Cobit compliance, build engineering, continuous integration, and automated application deployment. He serves on the IEEE Software and Systems Engineering Standards Committee (S2ESC) management board, is the current chair of the IEEE P2675 DevOps Working Group and served as the technical editor for CM Crossroads for more than 15 years. Bob is also editor of the Agile ALM DevOps journal, coauthor of Configuration Management Best Practices (Addison-Wesley, 2010) and his latest book Agile Application Lifecycle Management – Using DevOps to Drive Process Improvement (http://agilealmdevops.com).

Behaviorally Speaking: Dysfunctional Ops

+
-
0

Behaviorally Speaking – Dysfunctional Ops
By Bob Aiello

IT Operations plays an important role in any organization by ensuring that systems are always functioning as they should and all services meet the needs of customers and other end users. Some developers get frustrated when trying to work with their operations colleagues and may even try to bypass IT Ops whenever possible. My experience has been that developers may loose faith in the competency of the operations team – leading to doubts about their abilities. Are developers just a bunch of undisciplined wild cowboys or are there good reasons why many developers feel that they have to try to avoid working with Ops whenever possible? The truth is that operations, in many large organizations, is incapable of handling their role and developers are well-justified in trying to bypass Ops. In this article, we are going to have a tough conversation about the reality that many operations organizations lack qualified personnel, are more focused on blocking progress than delivering value to customers and exhibit many other dysfunctional behaviors. How did we get to this point where operations is often so dysfunctional?

Computer operators were once solely focused on handling decks of punch cards, data tapes and handling very large print jobs. The job focused on operating the equipment and dealing with any environment issues such as keeping the computer room nice and cold for the sake of the machines themselves. Back then developers were much more skilled than computer operators and required many more years of training in technical areas, from machine code programming to intensive work in languages including Fortran, COBOL and PL/1. By contast, many of today’s operations engineers are just as technical and skilled as any developer, often focusing on provisioning complex cloud-based runtime systems, environment monitoring while designing and implementing fully automated application deployment pipelines. Unfortunately, some operations engineers are not up to the demands of the job and developers may feel justified, and even motivated, to bypass operations whenever possible. In my own work, I have come across operations engineers who lacked even the most basic Unix command line skills, including use of the VI editor. This might seem trivial, but these same engineers had the root (e.g. super user) passwords, which means that they did have the potential to make serious mistakes, which could take down large-scale production systems. Why would IT operations management hire people who lacked the basic skills necessary to successfully administrate systems? Some organizations allocate management compensation based upon the number of people reporting into a function. When this happens, managers are incentivized to have large teams with lots of head count, without the more important focus of whether or not these engineers have the right level of knowledge, skills and abilities.

It has been my experience that when operations engineers lack sufficient technical skills, developers loose faith and resort to working around the operations group instead of partnering with them to get the work done successfully. Operations engineers themselves may exhibit counterproductive coping skills to deal with their limited expertise. Some teams focus narrowly on one specific skill which they understand and can execute successfully. They may end up adopting a siloed mentality where they maintain a very narrow focus, do not share their knowledge and procedures and consequently lack the broader systems knowledge required for success. I have seen many large operations organizations, which consisted of small, specialized siloes which could not effectively meet their objectives even within the operations organization itself.

The best operations groups consist of highly skilled engineers with strong cross-functional capabilities. Having experience with development, particularly scripting and automation, is an absolute must-have. When these colleagues work on deployment automation, we often call these technology professionals DevOps engineers. Going forward, operations must employ engineers with sufficient knowledge, skills and abilities to get the job done. Do you agree or disagree? Feel free to drop me a line to share your opinion!

Personality Matters—CM Excellence

+
-
0
Personality Matters—CM Excellence

Personality Matters—CM Excellence
By Leslie Sachs

Excellence in Configuration Management conjures up images of efficient processes with fully automated procedures to build, package and deploy applications resulting in happy users enjoying new features on a regular basis. The fact is that continuous integration servers and great tools alone do not make CM excellence. The most important resources are the technology professionals on our team and the leader who helps guide the team towards excellence. That doesn’t mean that everyone is (or can be) a top performer, even if you are blessed with working on a high performance cross-functional team. What it does mean is that understanding the factors that lead to excellence will enable you to assess and improve your own performance. A good place to start is to consider the traits commonly associated with effective leadership and successful results. This article will help you understand what psychologists have learned regarding some of the essential qualities found among top leaders and others who consistently achieve excellence!

Software Process Improvement (SPI) is all about identifying potential areas of improvement. Achieving excellence depends upon your ability to identify and improve upon your own behavior and effectiveness. It is well-known that we are each born with specific personality traits and innate dispositional tendencies. However, it is an equally well-established fact that we can significantly modify this endowment if we understand our own natural tendencies and then develop complementary behaviors that will help us achieve success!

Let’s start by considering some of the personality traits that help predict effective leadership. One of the first studies on effective leadership was conducted by psychologist Ralph M. Stogdill [1]. This early study identified a group of traits including intelligence, alertness, insight, responsibility, initiative, persistence, self-confidence, and sociability. It is certainly not surprising that these specific traits are valuable for successful leaders and achieving excellence. Being intelligent speaks for itself and of being alert (for new opportunities) and having insight into the deeper meaning of each situation and opportunity. Although general intelligence was for a long time considered static, more recent research suggests that it is possible to bolster one’s genetic inheritance. Certainly, one can consciously strive to develop the behavioral patterns, such as attentiveness to detail and novelty and thoughtful analysis of options, closely associated with intelligence.

You might want to ask yourself whether or not you are responsible, take initiative and show persistence when faced with difficult challenges. Displaying self-confidence and operating amiably within a social structure is essential as well. Do you appreciate why these characteristics are essential for leadership and achieving success and look for opportunities to incorporate these qualities into your personality? To improve your leadership profile, you must also actively demonstrate that you know how to apply these valuable traits to solve real workplace dilemmas. Upon reflection, you can certainly see why CM excellence would come from CM professionals who are intelligent, alert and insightful. Being responsible, showing initiative and being persistent along with having self-confidence and being a social being are all clearly desirable personality traits which lead to behaviors that result in CM excellence.

Stogdill conducted a second survey in which he identified ten traits which are essential for effective leadership. This expanded cluster includes drive for responsibility, persistence in pursuit of goals, risk-taking and problem-solving capabilities, self-confidence, and a drive for taking initiative. In addition to these, Stogdill also discovered that people’s ability to manage stress (such as frustration and delay) as well as their accountability for the consequences of their actions are both integral to leadership success. After all, intelligence, insight, and sharp analytic skills are not very useful if a manager is too stressed out to prioritize efficiently or authorize appropriate team members to implement essential programs Finally, you need to be able to influence other people’s behavior and to handle social interactions. [2] Other noted psychologists have also studied leadership traits and many have identified similar traits.

Jurgen Appelo lists 50 team virtues [3] which, not surprisingly, also correspond to many of the traits identified in Stogdill’s studies and I discussed many of these same traits in the book that I coauthored with Bob Aiello [4]. You need to consider each of these traits and understand why they are essential to leadership and achieving success. Keep in mind that while people are born with natural tendencies, each of us is capable of stretching beyond them if we understand our own personality and identify which behaviors are most likely to lead to the changes we desire. So if you want to achieve greater success, consider reflecting upon your own behaviors and comparing your style with those traits that have been associated repeatedly with good leaders and CM excellence. For example, being proactive in solving problems and having the self-confidence to take appropriate risks can help you achieve success. Remember also that being social means that you involve and interact with your entire team- full team participation maximizes the power of each member’s strengths while minimizing the impact of individual weaknesses.

Configuration Management excellence depends upon the skilled resources who handle the complex IT demands on a day-to-day basis. The most successful professionals are able to take stock of their personality and consider the traits that experts regard as essential for effective leadership. If you can develop this self-awareness, you can achieve success by developing the behaviors that result in strong leadership and excellence in all you do!

References:
[1] Yukl, Gary, Leadership in Organizations, Prentice Hall, 1981, p. 237
[2] Northouse, Peter G., Introduction to Leadership Concepts and Practice Second Edition, SAGE Publications, Inc 2012, p. 17
[3] Appelo, Jurgen, Management 3.0 – Leading Agile Developers, Developing Agile Leaders. Addison-Wesley, 2011, p. 93
[4] Aiello, Robert and Leslie Sachs. Configuration Management Best Practices: Practical Methods that Work in the Real World. Addison-Wesley, 2010

DevOps and Segregation of Duties

+
-
0
DevOps and Segregation of Duties

DevOps and Segregation of Duties
By Bob Aiello and updated Thursday November 10th, 2016

Editor’s note – this article was originally written in response to a July 31, 2016InfoQ article, DevOps Survival in the Highly Regulated Financial Industry, written by my esteemed colleague, Manuel Pais. Many people read the original article and came to the wrong conclusion that there is no legal requirement for a segregation of duties. I wrote my response to this article and contacted Mr. Manuel Pais, asking him to comment. Please do not interpret my words as criticizing Manuel or his writing in any way. Manuel Pais is a skilled technology professional and a talented technology writer. That said, I feel compelled to set the record straight on the issue of whether or not a segregation of duties is an actual requirement. Specific references to the requirement for segregation of duties are included at the end of this article.

Here are my original comments and Mr. Pais’ response.

Bob Aiello’s original comments:
In a July 31, 2016 InfoQ article, DevOps Survival in the Highly Regulated Financial Industry, my esteemed colleague, Manuel Pais quoted a presentation by Robert Scherrer, head of application engineering at SIX, which questioned the need for some regulatory requirements such as a segregation of duties.

Pais quotes Scherrer as saying, “Particularly noteworthy was an internal rule that developers could not access production systems. Turned out there was no external regulation with such requirement, instead this was a literal interpretation of the segregation of duties requirement. “

According to the article, Scherrer reportedly suggested that “blindly following internal directives is a costly mistake that hinders the benefits of DevOps.”

Response from Manuel Pais…
“As with many, if not most, ideas in the tech world, context is key. The story was not trying to claim there are no segregation of duties requirements at all. Instead it tried to summarize a 20m talk about an organization that was able to move to a better place in terms of deployability and recoverability with advanced mechanisms for controlling access to production environments. A mechanism that complies with regulations while allowing more flexible ways of working and organizing teams and work. It is hard to summarize such a dense talk about a complex system in a highly regulated environment. It should be clear that the need for adequate controls should never be taken lightly. Instead each organization must fully understand what are the regulations applicable in their context and geography, while at the same time (hopefully) trying to reduce waste and (often self-imposed) constraints that have become obsolete due to evolutions in technology, people and understanding of those very regulations. I urge readers to watch the recording of the presentation that the story tried to summarize (with inherent shortcomings) in order to better understand its intricacies.”
Hope that helps.
Manuel Pais | @manupaisable   |   InfoQ   |   LinkedIn   |   SlideShare

Bob Aiello’s analysis…

The premise that we need to discuss is that some folks believe that there is no explicit requirement for a segregation of duties…

I disagree with this premise completely. Effective IT controls help those responsible develop secure and reliable code much faster while also minimizing costly mistakes. As discussed in my latest book on Agile Application Lifecycle Management, DevOps requires right-sized IT controls. In my opinion, the above mentioned InfoQ article could lead some of our colleagues to believe that there is no explicit requirement for a segregation of duties (actually I know for a fact that it did which is why I wrote this article).

In fact, there are many specific references to a segregation of duties.

For example, the Office of the Currency (OCC) comptrollers handbook states (page 7), “Segregation of duties to reduce a person’s opportunity to commit and conceal fraud or errors. For example, assets should not be in the custody of the person who authorizes or records transactions.” And on page 23 it states that auditors are trained to ask “Is there segregation or rotation of duties to ensure that the same employee does not originate a transaction, process it, and reconcile it to the general ledger account?”

FINRA has similar requirements for Technology management, “We have observed deficiencies in firms’ risk management practices in these areas, for example through a lack of written procedures and evidence of supervision, insufficient segregation of duties for personnel involved in the development and deployment of technology changes, as well as insufficient user acceptance testing and quality assurance.”

ISACA Cobit (used for compliance with section 404 of the Sarbanes-Oxley act of 2002) explicitly lists the requirement for a segregation of duties. For example, in PO4.11 – Segregation of Duties, we see COBIT Control Objective PO4.11 – Segregation of Duties is contained within PO4 Define the IT Processes, Organisation and Relationships.

Another Cobit 4.1 control object, AI2 – Acquire and Maintain Application Software states that:

Control over the IT Process of Acquire and Maintain Application Software that satisfies the business requirement for IT of aligning available applications with business requirements, and doing so in a timely manner and at a reasonable cost by focusing on ensuring that there is a timely and cost-effective development process is achieved by

  • Translating business requirements into design specifications
  • Adhering to development standards for all modifications
  • Separating development, testing and operational activities ***

and is measured by

  • Number of production problems per application causing visible downtime
  • Percent of users satisfied with the functionality delivered

In practice, regulatory Compliance always comes down to what (compliance) lawyers will defend, but the assertion that the requirements are not specified and clear is just not true. Both internal and external auditors will write up audit violations when there is a lack of segregation of duties – and rightfully so. If you are working in a bank or other financial services firm you may find yourself receiving a letter from regulatory authorities – telling you that your organisation must correct deficiencies or cease and desist from a particular area of business. It is also unlikely that you will pass your ISO 9000 review without effective IT controls in place.

It is time for some common sense guidance on how to implement DevOps and still comply with regulatory and audit requirements. I have good news for you. Many of us are involved with efforts to create industry standards for DevOps.  Our goal is to focus on establishing the right balance of IT controls that help to avoid human error while focusing on building secure and reliable systems. Let me know if you would like to participate in these efforts!

Bob Aiello
bob.aiello@ieee.org

DevOps Principles and Practices

+
-
2
DevOps Principles and Practices

DevOps Principles and Practices
By Bob Aiello

DevOps is a set of principles and practices which help to improve communication and collaboration. DevOps is not just between development and operations, but in fact can be practiced between any two organizational structures which need to improve how they interact with one other. My new book on Agile ALM and DevOps explains how to use DevOps principles and practices to improve communication and collaboration between each of the groups interacting to deliver software and systems to end users. In practice, DevOps principles and practices help to operationalize the DevOps approach by explaining how to improve the existing application lifecycle management (ALM) practices, including application build, package and deployment procedures, change management and much more. The journey to DevOps should begin with effective communication and collaboration.

Effective communication and collaboration

Effective communication requires that the involved parties be included in discussions that impact how DevOps practices will be implemented. Too often, key stakeholders are not included in meetings which impact them in significant ways. Effective communication might include working sessions to review working documents or working sessions intended to develop procedures related to application build, package and deployment. I am usually sharing my screen during these sessions and asking my colleagues to partner with me to fully automate all of the procedures required for reliable and security application and systems deployment.

Systems thinking

Systems thinking refers to considering the entire software and systems lifecycle from beginning to end. Very often, solutions only address a limited scope of the system. Many technology professionals are specialists and only have a narrow understanding of how the rest of the system works. For example, developers may come up with optimal methods for deploying to development machines without considering that it is in the best interests of the organization to have a consistent process across the entire software and systems lifecycle. In the same way, the interests of each of the stakeholders including development, operations, QA, testing and security all need to be considered. Systems thinking can and should eliminate the narrow perspective-taking often the favored by organizational silos. You do not have to change your organizational structure in order to eliminate the negative impact of silos.

Ops using the same practices as development – “left-shift”

Deployment procedures should be consistent across all environments from the initial development test environments through to UAT and Production. The practice of “left-shift” involves getting operations involved in the very beginning of the software and systems lifecycle so that operations professionals can understand the required procedures and help create automated procedures essential for a repeatable process.

Feedback and experimentation

Feedback loops are a fundamental approach in DevOps that facilitate experimentation and continuous process improvement. Agile-oriented retrospectives and collaborative communication between stakeholders help to communicate what works well and address areas that require improvement. Teams should be able to experiment and try different approaches to identify and implement effective best practices.

Sustainable pace – effective culture

Many teams take a defeatist attitude and assume that deployments will always be weekend-consuming activities that require everyone to work very long hours. DevOps transforms the deployment process into a routine event that is highly regarded, but executed with minimized risk and potential impact to the
firm. DevOps spends a lot more time upfront automating the entire application build, package and deployment process. Initially, it may even seem like the
deployment is taking longer than expected because automated scripts are being written to create the automated deployment pipeline. However, automating from the beginning has proven repeatedly to be a very wise investment.

Automation and tools

Robust automation is a key feature of any DevOps implementation and selecting the right tools is an essential first step. While process is more important than tools, in DevOps, tools are definitely a first class citizen. Robust toolsets often require support as well as skilled engineers who know how to use them effectively. The right organizational structure can help to make tools more effective, especially in terms of providing the right support structure to manage their usage on a day-to-day basis.

Organizational structure where collaboration is compelling

DevOps is not specifically a group, although most organizations find that it is helpful to establish a dedicated DevOps team to implement and support DevOps principles and practices. For organizations that utilize the ITIL v3 framework, the Release and Deployment Management (RDM) team is often the right group to take responsibility for establishing DevOps best practices throughout the organization. 

Agile and DevOps

Many DevOps practices existed way before agile became popular. But agile dramatically demonstrates the value of DevOps by requiring support for a frequent release cadence. Agile principles also are inherent in DevOps principles. Equally important is understanding the perspectives of each of the stakeholders especially when their approaches and goals are not completely in alignment.

Ops and Dev and the Clash of Perspectives

Development is focused on creating and implementing new features as often as possible. Their world is often characterized as being the “Wild,
Wild West” because almost anything goes and developers are frequently not as disciplined as they need to be. The operations team has a very different
perspective and is generally focused on ensuring that releases are both reliable and repeatabl. The push to deliver changes quickly versus the desire to
maintain uninterrupted services often creates a clash of priorities between development and operations. DevOps aims to address both goals and ensure that
changes can be delivered as often as desired with completely reliable systems and services.

DevOps and the ALM

DevOps should be understood within the broad scope of Application Lifecycle Management (ALM). Just as DevOps engages in broad systems thinking,
DevOps also includes all stakeholders within the software and systems lifecycle.

Getting Started with DevOps

DevOps should always begin by assessing the existing practices to build, package and deploy the application. Manual procedures should be documented into a checklist and then the manual steps should be automated – usually using scripts (e.g. Ruby, Shell, Python etc). Getting operations involved from the beginning of the deployment lifecycles across all environments (development, test through production, inclusive) is essential. The same procedures should be used across all environments with any necessary discrepancies documented, reviewed and understood by all participants. Release coordination and change control are essential functions that help to ensure that all tasks are understood and tracked as needed.  Technical risk should also be reviewed and discussed with plans created to mitigate risks that cannot be avoided. Getting started with DevOps requires an iterative approach that is a learning experience for all participants. Goals should be established early so that everyone understands exactly what the team is working towards achieving. DevOps requires complete certainty that the correct code has been baselined in the version control repository, built correctly and packaged with identifiers to ensure that the configuration item is fully traceable and identifiable. Cryptographic hashes should be created to ensure that we can definitively and uniquely identify each CI, making it child’s play to track if the wrong configuration items have been selected.

Securing the Trusted Application Base

Ultimately, we want to create a secure trusted application base. The secure trusted application base enables us to programmatically verify that the correct code has been deployed and also proactively identify any unauthorized changes. This approach will facilitate an accurate and programmatically-updated configuration management database (CMDB) by ensuring that code can be easily discovered through automated procedures. This is the only reasonable way to gurantee an updated and accurate CMDB.

DevOps Process Improvement

Successfully implementing DevOps requires an iterative approach. Frequent retrospectives (some people call them post-mortems) should be held to discuss what went well and what can be improved. The DevOps transformation itself is an agile iterative development effort that needs to involve all of the stakeholders responsible for the successful application build, package, and deployment.

Make sure that you drop me a link and tell me about your DevOps best practices!

Bob

How to implement a comprehensive API-first framework that meets the needs of agile development and DevOps

+
-
0
How to implement a comprehensive API-first framework that meets the needs of agile development and DevOps

How to implement a comprehensive API-first framework that meets the needs of agile development and DevOps
By Steve Davis, CTO of Four51

An API-first framework, or product, allows customers and developers to innovate rapidly. The API product must be capable of reacting to bug fixes and feature requests quickly. It’s an expectation of the developer who has most likely been involved in internal only software development environments. Your API is now a part of their overall development plan and you cannot be a bottleneck to their progress. Traditionally, even within SaaS software platforms, release cycle times were measured in months, not days. With the proliferation of Open APIs and microservice architecture, your API must be capable of the expected release capability or it will be rejected. The right way to enable this is through quality DevOps.

One of the pillars of a continuous integration strategy is to always be deployable. Simply put this means you should always have a repository of code that can be deployed to your production environment on a moment’s notice. Enabling this while also allowing for multi-faceted feature development can be managed in various ways. The two major distributed version control systems (DVCS), git and mercurial, support the requirements for continuous integration.

One of the more popular and effective workflows is the “Git-flow” method and a variation promoted by GitHub’s engineering team called “Git-hub flow”. If you are inclined to use mercurial, there is a similar method named “Hg-flow” that was inspired by “Git-flow”. The concepts are simple and they work for small and large teams.

With the “Git-hub flow,” the only required rule is that the master branch should always be deployable to production. Any new features should be branched and named descriptively. Once that branch is ready for deployment, you can create a pull request and merge it into master. Merging code can present conflicts at times, though, so choosing a good merge conflict tool is important. You should also strive to release the master branch as quickly as possible once the merge has occurred. The benefits are clear –– you are pushing new functionality rapidly.

Believable progress is very difficult to demonstrate with an API-first platform. Frequent production releases provide this insight. Your marketing team has something to promote and your executive team sees tangible progress.

As with any new process or methodology, there are legitimate concerns that will be brought forward. One of the primary considerations is stability of the deployment and quality assurance. The continuous integration strategy addresses these two explicitly.

First, the methodology itself offers greater stability by design. Because the feature branches deployed should be isolated and consist of less code changes, the impact of the deployment is minimized. Monitoring the activity immediately after deployment will shed light on any unintended side effects and, if a problem is discovered, your team can address the problem, create a fix and deploy rapidly. Because the release doesn’t contain a vast amount of changes, any disruption is generally minimal and the recognition of the right fix is usually obvious.

Secondly, the quality must be covered by a comprehensive unit test routine. There are a number of good unit test frameworks, but the key for your team is to implement rigid controls around how they are managed. Unit tests should all pass before any pull requests are created. Your build system should also run all unit tests during its automated build, stop the build process on any failure and notify all developers of such issues. If there are failures, the entire team should focus on creating a solution to bring the build back to a stable status. Integration testing is also extremely important. It is different from unit testing in that it’s generally more focused on testing the system as a whole rather than testing individual methods in isolation. Your team should strive to perform these tests in a clone of your production environment. You’re hitting a homerun if your integration test routine is capable of creating and tearing down environments during the build process.

An API-first framework presents challenges that are not present in application development. Applications are used by humans, and humans are generally good at adapting to small changes without much fuss. APIs are used by code, which is generally not as good at this. When an API changes, all code that uses it must also be changed, tested, and re-deployed. Applications have control over actions of users, through a UI, that shape the intended usage. APIs offer developers opportunities to stretch the intended usage and potentially to go far outside the bounds of intention. Unit tests are crucial to ensuring the integrity of the API base so developers can feel confident that major changes are less likely to disrupt users. Having quality unit and integration tests is essential to ensuring your development process is stable, reliable and capable for rapid development and deployment.

Today, the expectations for a rapid delivery of software is normal. A sound DevOps strategy is critical to support an Agile development workflow. Getting your operations right allows your development team to focus on delivering, which benefits your customers and your organization.

About the author:
Steve Davis is CTO of B2B eCommerce solutions provider Four51,
and has held the role for nearly a decade. Steve has been working in technology for about 20 years, starting in his Navy days, which helped him develop a strong but motivating leadership style. Visit www.four51.com for more information.

So How Are Industry Standards Created?

+
-
0

So How Are Industry Standards Created?
By Bob Aiello

Industry standards and frameworks provide the structure and guidance to help ensure that your processes and procedures meet the requirements for audit and regulatory compliance. For US-based firms, this may involve passing your SOX audit (for compliance with section 404 of the Sarbanes-Oxley Act of 2002) or acquiring the highly respected ISO 9000 Quality Management System certification expected by many customers throughout the world.

Industry standards are not perfect and some of the specific reasons for why they may fall short of expectations can be traced back to how they were initially created. The process of creating an industry standard is actually quite deliberate and time-consuming.. There are some excellent resources from the IEEE and other standards bodies, which describe the process to draft and implement standards. But I would like to describe some of my own personal experiences participating in the collaboration and teamwork of creating an industry standard. Working closely with other colleagues who are dedicated to excellence has been far and away the most exciting professional experience that I have been fortunate to have.

Please note that this is not an official IEEE article, but rather Bob’s recounting of a personal experience being involved with creating industry standards.

The first step is always to decide on the initial scope and focus of the standard. We then review any existing resources available – including related industry standards and frameworks or simply documents which can help educate the members of the team involved with this effort. The standards working group is a high-performance self-managing self-educating cross-functional team with subject matter experts from a variety of disciplines and perspectives. We do not always agree with each other and, in fact, the discussions can be quite confrontational at times – although always professional and collaborative. These disagreements are a natural expression of the group’s striving to come up with the best approach to advocate in the text of the standard.

We create an initial outline and list of topics to consider and then address the task of creating a working draft. The focus is on “shall” statements which are mandatory (for compliance with the standard) and “should” statements which are recommended. We hold numerous sessions to collaboratively create the initial draft. It is common to assign specific sections to individual members (or subgroups) who then go off and independently create the initial draft wording.

Once the draft is written, it is sent to a few SMEs outside of the working group for their reaction and comment. Once this feedback is evaluated and incorporated, the draft is sent out to a wider group for review and comment and, once again, feedback is incorporated. The objective is to have validation that it is a solid document before it is put out for a vote.

Above all else, the standards creation process is collaborative and transparent. Typically, contributor’s comments are recorded and the reason for their acceptance or rejection documented. We have a strong desire to ensure that the draft standard is aligned with other industry standards and frameworks and do our utmost to harmonize with the current guidance provided by other sources. Final decisions are made and sometimes folks are not happy, but they know that their views are always heard and, most often, recorded for traceability. It is customary for a standard to require a significant percentage of voter approvals for passage and acceptance by the standards body. On occasion, controversial paragraphs have to be dropped in order to obtain the required votes for approval, similar to the negotiations, aka “horse-trading”, for which politics is known. Although such modifications felt to me personally like we were “watering” down the standard just to gain the required consensus, the teams focus and mission is to produce a clear document that will be both respected and adopted.

Over the years, I have written extensively on how to comply with configuration management related standards, including the highly popular IEEE 828 (which I had the privilege to participate in updating). Lots of folks like to criticize standards, but often they are criticizing a document that they have never actually spent the time to read – let alone understand or see implemented effectively.

It has been my personal experience that implementation of a standard requires two key skills. The first is harmonizing the guidance by understanding similar industry standards and frameworks. The second is tailoring, in which we provide a rationale for why specific guidance cannot be followed, if this is in fact necessary.

Here’s your opportunity! We are starting up an effort to create a working group to write an industry standard for DevOps. Please consider getting involved now to help shape the guidance that we provide. Rest assured that I will continue writing about this exciting project in the coming months and your voice is important to us!

Bob Aiello
bob.aiello@ieee.org
http://www.linkedin.com/in/BobAiello

 

 

Call for participation – P2675 DevOps Standard for Building Reliable and Secure Systems Including Application Build, Package and Deployment

+
-
0

P2675 – DevOps – Standard for Building Reliable and Secure Systems Including Application Build, Package and Deployment
Contact Bob Aiello to join the IEEE working group.

Scope: This standard will specify practices for groups including development, operations and other key stakeholders to collaborate and communicate effectively to build, package and deploy software and systems in a secure and reliable way. All of these activities and functions shall be integrated within the complete lifecycle.

Purpose: The purpose of this standard is to specifiy required practices for operations, development and other key stakeholders to collaborate and communicate to deploy systems and applications in a secure and reliable way

Who Should Participate: DevOps, by its very nature, impacts every member of the team so technology professionals including developers, operations engineers, QA & testing, project managers and other members of the team responsible for ensuring that applications meet their business objectives.

The DevOps standard must address and incorporate existing IEEE and ISO standards related to Systems and Software engineering, Configuration Management, Verification & Validation, Information Security, Measures of Reliability and Software Testing among others. DevOps does not contradict existing industry standards, but applies new principles and practices to integrate existing processes.

Related standards include:

  • ISO/IEC/IEEE 12207, Systems and software engineering–Software engineering processes
  • IEEE 828, IEEE Standard for Software Configuration Management
  • IEEE 1012 IEEE Standard for System and Software Verification and Validation
  • ISO/IEC 27001 Information technology — Security techniques — Information security management systems — Requirements
  • IEEE 982.1 IEEE Standard Dictionary of Measures to Produce Reliable Software
  • ISO/IEC/IEEE 29119 Software and Systems Engineering–Software TestingThis standard will describe standard DevOps practices and principles in general terms as “practices” and then include specific references for detailed tools/practices in an Annex.

    Contact Bob Aiello to join the IEEE working group.

Behaviorally Speaking – CM Excellence

+
-
0
Behaviorally Speaking – CM Excellence

Behaviorally Speaking – CM Excellence
by Bob Aiello with Dovid Aiello

Configuration management (CM) is a comprehensive and awesome discipline with practices that can help improve quality and productivity throughout the entire software and systems delivery lifecycle. Configuration management is the basis of DevOps and without excellent CM you are not going to be successful implementing continuous practices such as integration and delivery.

The lack of CM Best Practices or poorly-implemented CM can be disastrous for an organization. Although, interestingly enough, disasters themselves can provide a great opportunity to improve your configuration management because they highlight the importance of getting the process right. As Winston Churchill once said, “never let a good crisis go to waste”. For example, software installation and upgrades can result in major outages that are so disruptive that they threaten the very existence of the corporation. Conversely, excellence in configuration management can be just as beneficial. Read on if you would like to truly achieve CM excellence.

It is always important to start by considering what excellence will look like. For me, CM Best Practices means that we always know exactly what version of the code (actually configuration items or CIs) is running in production (or QA) at any point in time. We also want to be able to retrieve the exact version of the code used to build and deploy the release to production (or QA) and we need to be able to make a small change to the code without any chance of the code regressing due to the wrong version of a header file or some other dependency. These three capabilities are the starting point for CM excellence. Unfortunately, many companies do not have the necessary procedures in place to guarantee these capabilities . Here’s how to implement them.

It is always essential to start with version control, including baselining your code using a tag or label. Baseling refers to identifying all of the configuration items (CIs) for a specific milestone. You also want to be able to link changesets to workitems so that you have traceability as to exactly why a change occurred and who gave the authorization. When a bugfix is necessary, it is non-negotiable to have relible variant management (using branches or streams).

Automated application builds with immutable version IDs embedded in each configuration item (CI) are also essential as is packaging the release with a manifest specifying the contents (often called a bill of materials or BOM). The package also needs to contain a procedure that can verify both the contents of the package and also the application that has been deployed. Physical Configuration Audits verify that you have the correct configuration items (CIs) and a functional configuration audit verifies that the CIs are performing as they wre intended.

Application deployment must also be fully automated and should enable you to deploy applications frequently and painlessly. It is ideal to have a complete deployment framework to help ensure a consistent and reliable methodology for automated application deployment. There are many challenges that can impact the successful implementation of configuration management. The most challenging that I have found is a lack of communication between developers and the operations folks who are responsible for ensuring reliable and continuous service. One aspect of this challenge involves the sharing of technical knowledge. Developers get to learn new technologies and spend countless (truly countless) hours working with complex technologies. Operations folks get the runtime code when it is about to go live and often have only a short period of time to get up to speed. I have experienced this exact challenge as a build engineer. It is particularly difficult when the technology changes and there is less than adequate communication and training to help operations get up to speed.

DevOps has emerged as an industry best practice that focuses on improving communication between development and operations. I often explain that in CM we mix the ingredients and in DevOps, we make the pie. Applications should be built with the tools necessary to maintain them and, equally importantly, ensure continuous and excellent service. Moving automated build, package and deployment upstream means that automated procedures required for deployment to production can also be used for deployment to development test regions, QA, and UAT. This approach allows us to achieve excellence by having our automation developed and tested early in the process.

Excellence in CM requires that you implement best practices that meet the needs of your team. Implemented well, automated build, package and deployment can help streamline the entire software and system development process, improving quality and productivity for your entire team.

 

Micro Focus Announces Intent to Merge with Hewlett Packard Enterprise’s Software Business Segment

+
-
0

Micro Focus Announces Intent to Merge with Hewlett Packard Enterprise’s Software Business Segment

Strategic Combination Furthers Company’s Long-term Strategy to Maximize Mature Technology Investments, Accelerate Innovation Opportunities for Customers
________________________________________
ROCKVILLE, Md., Sept. 7, 2016 /PRNewswire/ — Micro Focus (LSE: MCRO.L) announced today its intent to merge with HPE’s Software Business Segment in a transaction valued at approximately $8.8 billion. The merger is subject to customary closing conditions, including anti-trust clearances and shareholder approval and is expected to close in Q3 2017. Micro Focus and HPE also announced as part of the transaction the intent to enter into a commercial partnership naming SUSE as HPE’s preferred Linux partner.

The proposed merger brings together two well established enterprise software vendors with highly complementary portfolios. With revenues of approximately $4.5 billion, it creates one of the world’s largest pure-play infrastructure software companies with a truly global footprint, agility and financial strength to drive software innovation across both traditional and emerging IT market segments.

“Today’s announcement marks yet another significant milestone for Micro Focus and is wholly consistent with the long-term business strategy we have been pursuing to be the most disciplined global provider of infrastructure software. The proposed merger with HPE Software is consistent with our recent acquisitions of Serena Software and the Attachmate Group,” said Kevin Loosemore, Executive Chairman, Micro Focus. “The combination of Micro Focus with HPE Software will give customers more choice as they seek to maximize the value of existing IT assets, leveraging their business logic and data along with next-generation technologies to innovate in new ways with the lowest possible risk.”

Organizations continue to seek technology solutions that improve their time to market and create new avenues to further engender customer loyalty and improve employee productivity and more. HPE Software brings additional breadth and depth across IT Operations Management, Software Delivery & Test, Enterprise Security, Information Management & Governance and Big Data Analytics, giving Micro Focus additional advantage to deliver richer solutions that effectively bridge existing IT infrastructure with emerging technologies to meet those business demands.

“We believe that the software assets that will be a part of this combination will bring better value to both our customers and shareholders as part of a more focused software company committed to growing these businesses on a stand-alone basis,” said Meg Whitman, President and Chief Executive Officer, HPE.

At the same time, Micro Focus and HPE have announced their intent to enter into a commercial partnership naming SUSE as HPE’s preferred Linux partner as well as exploring additional collaboration leveraging SUSE’s OpenStack expertise for joint innovation around HPE’s Helion OpenStack and Stackato Platform-as-a-Service solutions. SUSE and HPE are working together to define the specifics of the commercial partnership.

“SUSE and HPE have a long history of successful strategic partnership,” said Nils Brauckmann, CEO, SUSE. “We are excited now to explore new ways of expanding upon that with a commercial partnership focused on areas such as cloud computing, software-defined networking and application platforms. The combination of SUSE’s open source expertise and OpenStack capabilities with HPE’s Helion and Stackato offerings can create best-in-class enterprise solutions for our mutual customers.”

About HPE
HPE is an industry leading technology company that enables customers to go further, faster. With the industry’s most comprehensive portfolio, spanning the cloud to the data center to workplace applications, our technology and services help customers around the world make IT more efficient, more productive and more secure.

About Micro Focus
Micro Focus (LSE: MCRO.L) is a global enterprise software company helping customers innovate faster with lower risk. Our software helps customers build, operate and secure IT systems that bring together existing business logic and applications with emerging technologies to meet increasingly complex business demands. For more information, visit: www.microfocus.com.

 

1...345...10Page 4 of 10