How is Advanced DevOps different from DevOps

Agile and DevOps: Friends or Foes?

In one corner we have the certified Scrum Master, known to his friends as "the extreme programmer" and to his children as the "LeSS SAFe DAD": Agile!

In the other corner, the lean culture professional who relies on continuous delivery and infrastructure as code. He calls his left arm Dev, the right Ops: DevOps!

I may have exaggerated a bit, but when you hear opinions about Agile and DevOps, you might mistake them for completely different ideas. Adding to the confusion is that both concepts seem to defy definition. They even have their own jargons and slogans. Agile as the older approach may be a little less vague, but at least for DevOps there are a dime a dozen definitions. This lack of clear definitions has resulted in common meanings being confused.

Agile is often equated with Scrum, DevOps with Continuous Delivery. This over-simplification creates unnecessary tension between Agile and DevOps. You might be surprised that the two are actually best friends.

The historical connection between DevOps and Agile cannot be denied. When Patrick Debois and Andrew Clay Schafer tried to come to an agreement on the "Agile Infrastructure" at the Agile Conference 2008, they built the bridge to DevOps. The term "DevOps" was coined later by Patrick Debois, but the Agile Conference still cherishes this connection with the DevOps track. If we take a closer look at Scrum and Continuous Delivery, we discover that in addition to the story, there are also practical connections between Agile and DevOps.

In some teams, Scrum makes the difference between constant, frustrating struggle and productive, successful teamwork. In other teams, Scrum replaces political moves and over-engagement with objectivity and a clear focus. It can also be represented as a dogma. If the circumstances in the company or at work require something different, an agile team uses the basic Scrum principles, but adapts the corresponding practices in order to work more effectively. This is particularly important when using Scrum outside of software development.

Planning for unplanned tasks

Agile members of the DevOps community know that Scrum is helpful in keeping track of unplanned tasks. Certain tasks of operations teams can be planned: the releases for a major system change, the migration between data centers or the implementation of system upgrades. Most of the work of ops teams, however, consists of unplanned tasks as a result of performance peaks, system failures or security incidents. Such events require an immediate response. The team cannot wait for the appropriate tasks to be prioritized in a backlog for the next sprint planning session. For this reason, many DevOps-oriented teams use Kanban instead of Scrum. In this way, you can track planned and unplanned tasks equally and analyze the interplay between the two. Some teams also use a hybrid approach, which is often referred to as "Scrumban" or "Kanplan" (Kanban with backlog).

In many respects, the fact that Scrum is so widespread is the fact that it does not prescribe any technical practices. The lean management processes of Scrum are a great advantage for many teams. Where previously the priorities of several masters competed with each other, there is now only one set of priorities in the backlog. The previous excess of ongoing work has been replaced by a specific time schedule based on experience. Together, these two changes can bring a whole new level of productivity to the team. The team may still lack technical practices such as code reviews, automated acceptance testing, and continuous integration.

Proponents of other agile processes such as Extreme Programming are convinced of the importance of technical practices for sustainable speed in the team as well as for transparency towards management and stakeholders. Some scrum teams go over to recording technical tasks in the backlog. Although this approach is in line with the guidelines of Scrum, in practice it quickly leads to problems due to the bias of the product owner with regard to the features. If the product owners are not technology experts, they are quickly overwhelmed with the cost-benefit analysis of the technical practices. This applies in particular to technical tasks that have an impact on reliability, performance and security and thus on the operations area.

Product owner and service owner

At Atlassian, we have recognized that we can cope better with two different roles for the products we operate. Our product owners know very well which features the users need, but they are less able to weigh these features against non-functional aspects such as performance, reliability and security. For this reason, Atlassian also has a service owner for some SaaS products who is responsible for prioritizing these non-functional aspects. With the exception of occasional negotiations between the two responsible parties, these roles can be represented in independent teams. There are certainly other ways to "reinforce" feedback from the Operations team, but this method helps us overcome the common feature bias of product owners. The two-person approach isn't the only path to DevOps. It is only important to consider the non-functional aspects as "features" and to plan and prioritize them in the same way as the functional user stories.

Ultimately, all of these criticisms regarding Scrum cannot be traced back to Scrum itself. As with agile methods, Scrum has integrated mechanisms for process improvement, which are referred to as "retrospectives". We therefore assume that some Scrum teams use DevOps as a source of inspiration and optimize and adapt their approaches towards DevOps with Scrum retrospectives. However, very few teams get by without food for thought from outside. As long as DevOps is not yet mainstream (and maybe even taught during studies or training), it does not naturally result from Scrum. It probably doesn't matter whether the team hires an Agile or DevOps coach, as long as they have experience automating the creation, testing, and deployment of software.

DevOps is not just about continuous delivery

When implemented correctly, Continuous Delivery (CD) helps to limit ongoing work (WIP), while automated deployment helps ease restrictions. In this way, a team can use CD to achieve more frequent, higher quality releases instead of having to choose between speed and quality. However, as with Scrum and Agile, it should be noted here that teams focused on continuous delivery may miss the broader context of DevOps.

CD does not address communication issues between the company and the software team. If the company has a year-round, budget-driven planning cycle, the team pushing the commits to production may have to wait several months for the company to respond. The reaction is then often only an emergency solution, for example canceling the project or - even worse - doubling the project team (which is to be assessed negatively because the addition of a large number of new team members leads to interruptions).

In the Agile Fluency model, the focus on value, where teams focus on transparency and coordination, is the first level of fluency. CD quickly degenerates into an endless cycle of technical improvements that do not bring the company any significant benefit. A team may get fast, high quality deliveries, but on a product with little benefit to the end user or the business. Even if many users comment positively, the small benefit may become clear when looking at the entire company portfolio. Without this adeptness, a team can find it difficult to trade off technical practices and features. This is especially true for teams with a legacy code base without automated testing and without a design designed for frequent deployments. In a legacy environment, a transformation to CD can take several years. It is therefore imperative that the team be able to highlight the benefits to the company.

DevOps is more than just an automation of the deployment pipeline. According to John Allspaw, DevOps means that members of the operations team think like developers and vice versa. With this in mind, Gene Kim explains the basic principles of DevOps as "The Three Ways":

The first waySystems thinkingThe performance of the entire system is considered instead of the performance of a single work silo or a single department. It doesn't matter whether the contribution comes from an entire division or from a single employee.
The second wayReinforcement of the feedback loopThis is about creating the structured feedback loop. The goal of almost every process improvement initiative is to shorten and expand feedback loops so that the necessary corrections can be made.
The Third WayCorporate culture of constant experimentation and learningCreating a corporate culture that encourages two things: on the one hand constant experimentation, risk-taking and learning from mistakes, on the other hand knowing that repetition and practice make perfect.

At the heart of Continuous Delivery is The First Way: automating the process from dev to ops. Automation naturally contributes significantly to the acceleration of a deployment system. However, the systems mindset does not only apply to automation.

The Second Way is based on the principle "Developers have pagers too". Pagers may not be essential, but developers should be included in the operations of the operations team. This enables them to better assess the consequences of their development decisions. With this in mind, you might want to place log messages in a more convenient place and make the messages more meaningful. It's not just about awareness. The developers also contribute to troubleshooting with their internal system knowledge, so that solutions can be found and implemented more quickly.

The Third Way refers to incremental experiments in the system as a whole, d. H. not just through changes to the application in the pipeline. It is relatively easy to see how long the automation takes and how it can be further optimized with an increasingly powerful infrastructure. Less obvious is delays caused by handovers between different roles or organizations. So the teams check and adjust the entire delivery workflow to improve human collaboration. Constant adjustments and optimizations are essential for CD. If the team does not think about how to work more effectively and therefore aligns its behavior to other points, CD will not develop any further. The team needs to be able to solve its own problems.

In Scrum, every retrospective is an opportunity to improve practices and tools. This opportunity is wasted, however, if the team does not resolve short-term and long-term technical problems in the course of the retrospectives and instead waits for the Product Owner to add CD tasks to the backlog - which will never be the case.

DevOps is like Agile, but applied beyond the software team

Scrum primarily refers to the Agile principle "Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage." (Welcome change requests, even at late stages of development. Agile processes use changes as a competitive advantage for the customer.).

Continuous delivery is mainly related to the Agile principle "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software." (The top priority is to keep customers happy by providing valuable software early and continuously.)

Above all, agile means that incoming and outgoing changes are viewed positively. Rituals like stand-up meetings and sprint planning are less important. Ten further principles are laid down in the "Agile Manifesto". However, teams should not make a selection from these principles, but should consider them all. Together, the principles represent an attitude towards change that applies to both Agile and DevOps.

These teams face the challenge of operating sensitive systems that are of enormous importance to the company. Even the most urgent change requests affect these critical systems. The positive attitude towards change that is typical of Agile does not welcome change for its own sake. Rather, the development team should take responsibility for the quality of the changes and improve the overall capacity to add value to the company. Agile and DevOps also have this focus on business benefit in common.

In conclusion, it should be noted that neither Agile nor DevOps should be understood as independent corporate goals. Both are culture-related movements designed to provide the company with inspiration for better ways to achieve goals. Agile and DevOps work better together than in rivalry. If you want to avoid confrontations between the two ideas, you have to deal with the values ​​and principles on which they are based. Fast, too narrow definitions lead to silo thinking. Now that you know that there is more to Agile than just Scrum and there is more to DevOps than just CD, you can take advantage of the powerful combination of Agile and DevOps.

Connect your tools with automation

Perhaps the biggest challenge working across multiple tools is the constant change of context and the interruption that brings. It can take hours to get back into flow if you get interrupted by a non-code task. For this reason, more and more folks are automating across their git providers and work management tools. Here are three of the most common automation rules

  1. When a commit is created and the status is To Do ’then transition this issue to ess In Progress’. Go to rule.
  2. Transition to "done" when the pull request is merged. Go to rule.
  3. If a build fails in Jenkins, add a comment to Jira and Slack the team. Go to rule.

See these automation rules and 100s more in the Jira Automation Template Library.

Go to library

While Ian has broad and deep experience with both Java and .NET, he's best known as a champion of agile methods in large enterprises. He's currently focused on the emerging DevOps culture and the tools for enabling better continuous integration, continuous delivery, and data analysis. During his career, he's successfully managed enterprise software development tools in all phases of their lifecycle. He has driven organization-wide process improvement with results of greater productivity, higher quality, and improved customer satisfaction. He has built multi-national teams that value self-direction and self-organization. When not speaking or coding, you can find Ian indulging his passions in parsers, meta-programming, and domain-specific languages. Follow Ian at @devpartisan.