Balancing Agility and Discipline at Scale

Agile , Team Software Process (TSP) Add comments

Douglas C. SchmidtBy Douglas C. Schmidt
Principal Researcher

While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical, software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum. The event brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in mission-critical environments found in government and many industries. This blog posting, the fourth installment in a multi-part series highlighting research presented during the forum, summarizes a talk by James Over, manager of the Team Software Process (TSP) initiative, who advocated the building of self-managed teams, planning and measuring project process, designing before building, and making quality the top priority, among other principles associated with applying agile methods at-scale.

Over’s Talk on Balancing Agility and Discipline

In his opening comments to the audience, Over shared his views on agility and discipline and stressed the importance of finding a balance between the two. Over said that his presentation and research on combining agility and discipline is based on his work with software teams and software projects, although it is applicable to other fields.

One of the reasons that agile methods are so popular today is their ability to respond to change. As evidence of this popularity, Over pointed out that about 40 percent of software developers use one or more agile methods, compared with 13 percent that only use traditional methods, according to a 2010 Dr. Dobbs survey on trends among global developers.

One reason that agile methods have become so popular is that the pace of change is accelerating. Organizations are seeking solutions that will allow them to become more responsive to change. Agile methods provide some key parts of that capability, Over said.
Balance is key for organizations seeking to implement agile methods. While organizations work to improve agility, they must do so in a disciplined way.  Discipline is particularly important for organizations like DoD acquisition programs and other federal agencies developing large, mission-critical software-reliant systems at scale.

It’s important for organizations to understand what is meant by agility. While several definitions exist, Over stated one he likes: Agility is responding rapidly and efficiently to change with consistency. An agile business, he told the audience, should be able to respond quickly to change, make decisions quickly, and respond quickly to customers’ needs every single time, not just occasionally.

What Does Agile Look Like

Many software organizations claim to be agile because they follow agile principles, but they often lack an understanding of what it means to be agile. To achieve a greater understanding of agile methods, Over recommended that organizations measure their agility and evaluate their success along the following factors:

  • Response time. Organizations should assess how quickly they respond to a customer’s needs. What sort of experiences are users having with the software they are producing in terms of response time?
  • Efficiency. Organizations should consider their ability to deliver software. Can they produce projects with the desired balance between cost and quality? Are processes performing efficiently? Can organizations respond to change quickly and efficiently or do costs balloon out of control when changes are made to the content or requirements of the system?
  • Consistency. Does every customer have the same experience? When customers interact with an organization, are they always seeing the same response time, the same types of efficiency, and the same types of behavior on each application?

Impediments to Agility in Software

In 2010, the authors of the Agile Manifesto reunited to hold a retrospective on Agile. Examining the state of the practice in the last decade, the authors identified 10 impediments to achieving agility. From that list, Over identified the following five impediments that he deemed critical for organizations to overcome when they seek to making agile methods work in practice:

  • Lack of a ready backlog, which is the list of features that developers prioritize to build the software, is a serious concern. The manifesto authors found that 80 percent of teams had a ready backlog, but within that backlog, only 10 percent of items were ready for implementation. As a consequence, delays would occur in projects because the backlog was not prepared.
  • Lack of being “done” at the end of a sprint means that as the project is nearing the end of the sprint and declaring the sprint completed, some work items are being deferred or skipped, which often  causes delays. The most common cause is a poorly implemented practice or a deferred practice. For example, skipping unit testing can cause delays later in the project, as well as increase cost by at least a factor of 2.
  • Lack of teamwork surfaces when the team fails to come together and operate as a team and instead focuses on its individual needs. In this type of setting, developers focus on their own user features or stories and don’t pay attention to what the team is doing. They attend their daily standups, but fail to report problems that might affect the rest of the project. When team gets to the end of a sprint and discovers that there is still a lot of work to do, therefore, sprint failure and project delay will result.
  • Lack of good design stemming from organizational structures that affects the kind of designs that software teams produce.  For example, an inflexible hierarchical structure often results in inflexible hierarchical designs, which in turn can yield code that is brittle and hard to use, excessively expensive, and prone to high failure rates.
  • Tolerating defects is an unfortunate reality for many teams as they near the end of a sprint. They are out of time and deferring defects to the end of a sprint. Teams in this situation often take a set of unit tests or other issues and defer them to later sprints. When issues are deferred (which is a form of technical debt, as discussed by Ipek Ozkaya in her Agile Research Forum presentation), the result is increased in cost and higher failure rates later in the lifecycle because the defects aren’t being addressed as they are discovered.

Avoiding the Impediments by Balancing Agility and Discipline

Over next identified work that he and other SEI researchers have conducted to remedy the impediments to agility described above. Over’s work with software teams has focused on identifying a set of principles that teams should adopt to improve agility and response time. Over highlighted the following five principles that help address impediments identified by the Agile community in their retrospective:

  • Build high-performance teams. Software is knowledge work. Build self-managed teams that make their own plans, negotiate their commitments, and know the project status precisely.
  • Plan every project. Never make a commitment without a plan. Use historical data. If the plan doesn’t fit the work, fix the plan. Change is not free.
  • Use a measured process to track progress. To measure the work, define its measures. To measure the process, define its steps. If the process doesn’t fit the work, fix the process. Tracking progress via a measured process need not require a complicated solution given the appropriate set of tools and a codified method for applying the tools effectively.
  • Design before you implement. Design is critical since it informs the software implementation. Good designs produce less code and simplify the downstream evolution of the code. Design what you know and explore the rest. When you know enough, finish the design and then implement it.   To be clear, this principle isn’t espousing “big upfront design” (BUFD). It is BUFD without the BUF, or just D. Produce enough design to build the implementation. If the design is still too challenging, explore the problem first via prototyping to reduce design risk. Applying this principle effectively depends on various factors, including the size and complexity of the application, as well as familiarity with the problem, domain, methods, and tools.
  • Make quality software the top priority. Defects are inevitable. Poor quality wastes resources. The sooner a developer fixes a problem, the better.  As Jeff Sutherland stated in his retrospective, tolerating defects, ignoring pair programming or code review practices, and inadequate unit testing will substantially reduce team velocity. Continuous integration tools and automated testing are also important, but testing alone is insufficient because poor quality software will always cost more to produce because finding and fixing defects is still expensive, and the longer you wait to fix them the greater the cost of repair.

In summary, Over told the audience that in working on 20 different projects in 13 organizations to implement these disciplined agile principles, SEI researchers found that organizations delivered more functionality than originally planned or finished ahead of schedule. Among the other benefits, Over stated, was that projects realized test costs of less than 7 percent of the total cost with an average cost of quality of only 17 percent. Also, the projects delivered software with an average of only six defects in 100,000 lines of new and modified code.

Finally, discipline is the key to agility, Over explained, adding that agility can only be achieved when everyone in an organization acts professionally and uses a disciplined, measured approach to their work.

What's Ahead

The first posting in this series summarized discussions by Anita Carleton, director of the SEI’s Software Engineering Process Management Program, and Teri Takai, chief information officer for the DoD. Carleton provided an overview of the forum and discussed areas where SEI work is focused on applying Agile methods at-scale. Takai then discussed how Agile methods have been introduced into the DoD software acquisition and development environment. The second posting summarized discussions by Mary Ann Lapham, a senior researcher in the SEI’s Acquisition Support Program, who highlighted the importance of collaboration with end users, as well as among cross-functional teams, to facilitate the adoption of Agile approaches into DoD acquisition programs. The third posting highlighted the forum presentation by Ipek Ozkaya, a senior researcher in the SEI’s Research, Technology, and System Solutions Program, who discussed the use of strategic, intentional decisions to incur architectural technical debt. The technical debt metaphor describes the tradeoff between taking shortcuts in software development to speed up product delivery and slower—but less risky—software development.

In the next—and final—posting in this series I will summarize my presentation at the forum on the importance of applying agile methods to common operating platform environments (COPEs) that have become increasingly important for the DoD. I will explain how these methods can encourage more effective collaboration between users, developers, testers, and certifiers to help the DoD successfully build more integrated, interoperable, and affordable software systems.

We look forward to hearing your thoughts on applying agile at-scale in the comments section below.

Additional Resources

To learn more about the Team Software Process (TSP), please visit

The slides and recordings from the SEI Agile Research Forum can be accessed at

Share this

Share on Facebook Send to your Twitter page  Save to  Save to LinkedIn  Digg this  Stumble this page.  Save this page on your Google Home Page 

7 responses to “Balancing Agility and Discipline at Scale”

  1. Mark Noneman Says:
    I have a problem with the concept of "balancing" agility and discipline. Agile done right is perhaps the most disciplined method of developing software I've ever experienced.
    Part of the problem is accepting teams that simply say they're agile. I see a lot of team that say they are but within 3 questions, I can tell they're a long way from truly agile.
    The definition and execution to DONE is one of the primary drivers of quality in agile and does not need (in fact, cannot be) balanced with anything (especially cost!).
    Correctly implemented agile methods require an extremely high level of discipline. No one can hide in an agile project. Failure to perform is very public. Failure to delivery quality code cannot be covered up.
    I say don't balance agile with anything. Just do agile right!
    Mark Noneman, PSM II
    Agility Software
  2. Quinton Anderson Says:
    I could not agree more with Mark Noneman. In order to implement Agile one has to be extremely disciplined. Agile is a lean type principle promoting validated learning over reliance of requirements engineering that will always be incorrect, it has nothing to do with being undisciplined.
  3. Ben Linders Says:
    I see the need for balance between Agility and Discipline mostly on an organization level, which indeed makes it an issue at scale. On a personal and team level, Agile expects and supports disciplined professionals, as Mark and Quinton already commented. This is not a balance, but a win-win combination. But in order for professionals to become and stay agile and disciplined, the organization also has to enable and support agile and discipline.

    People throughout organizations have to build up understanding what it means to become agile, and have to practice what they preach. This includes professionals at all levels in organizations. There is a lot of knowledge and experience about agile at the personal and team level, but we are still discovering better ways to implement agile in organizations, and manage them in an agile way. Looking at the 5 bullit points, these are indeed important things. But they will only succeed if managers enable and support agility, and also become agile themselves. A disciplined way to to work Agile, from the bottom to the top!
  4. Paul Oldfield Says:
    The list of impediments looks remarkably artificial. Of all the people struggling to get Agile teams working in their organization, a good 50% or more would put "Organizational Culture" as the number one impediment. An assumption of Waterfall process and "Command and Control" management is built in to so many of the other organizational processes that it feels at times that one needs to rebuild the organization completely.
  5. Bill Nichols Says:

    For context, that list of impediments from Jeff Sutherland focused on long-term success and sustainment. Organizational change management is also on that list.

    Change is hard to accomplish and make last. The larger the organization or the more engrained the culture, the more difficult it is to change. But there are well established methods that improve the odds. I maintain that empowered teams are required.

    I describe some of the approaches we have used in my posting this week and a follow up post next week.

    One aspect of change management is that company management must change its behavior as well. This involves some training and coaching as all involved adapt to new roles, new ways of setting and evaluating goals, and new ways of working.

    Let me assure you that an organization with 50 percent of the problems stemming from the organizational culture is not that unusual. I would estimate that at least 50 percent of all initiatives that require organizational change fail because of cultural resistance. One improves the odds of success by going in with a plan to manage that change.
  6. Tim Chick Says:
    The point is not to balance Agile and discipline it is to balance "agility" and discipline. Agility is the power of moving quickly and easily; nimbleness. Discipline is behavior in accord with rules of conduct; behavior and order maintained by training and control. The problem is that the pressure put on organizations, teams, and individuals to be responsive to users and customers, incentives undisciplined behavior. Agile provides methods to help achieve agility, but does not eliminate the tension between agility and discipline. When the balance is not achieve the impediments above start to appear. Over is providing a list of ways to maintain the balance and resist the pressure to take short cuts or act in an undisciplined way.
  7. Maria Says:
    I think often people get bogged down too much with the processes and not enough time is actually spent doing the actually task. I am still a great believer in planning but PM is sometime just a little over complicated and it may appear too structured if that makes sense?

Add Comment

Leave this field empty: