Using Agile Effectively in DoD Environments

Acquisition , Agile , Technical Debt Add comments

By Mary Ann Lapham
Senior Member of the Technical Staff
Acquisition Support Program

Mary Ann LaphamOver the past several years, the SEI has explored the use of Agile methods in DoD environments, focusing on both if and when they are suitable and how to use them most effectively when they are suitable. Our research has approached the topic of Agile methods both from an acquisition and a technical perspective. Stephany Bellomo described some of our experiences in previous blog posts What is Agile? and Building a Foundation for Agile. This post summarizes a project the SEI has undertaken to review and study Agile approaches, with the goal of developing guidance for their effective application in DoD environments.

The SEI’s Agile project began in 2009 in response to our recognition of the growing awareness that Agile methods help alleviate key challenges facing the DoD, such as providing competitive capabilities to warfighters in a timely manner that minimizes collateral damage and loss of lives and property. We also observed an emerging consensus that Agile methods can be applied to create systems who functionality and quality attributes can be adapted more readily over time, which may help reduce total ownership costs over long acquisition program lifecycles. Within this context, the primary activities of our project included reviewing relevant literature, interviewing programs that are using or have used Agile methods, identifying criteria to be used to determine if a program is a candidate for Agile methods and what risks exist for implementing the Agile methods, and creating guidelines to be used in implementing agile methods.

We initially focused on the question of whether Agile could be used in DoD acquisition programs, which historically follow the DoD 5000 series of guidelines that have been associated with so-called waterfall methods. An early finding of our project was that no prohibitions preclude the use of Agile in the DoD 5000 series. This result is important since some skeptics have asserted that Agile is not suited for the DoD due to inherent conflicts between Agile methods and DoD policies and regulations.

Given that there is no one size fits all Agile method, however, we also found that implementations of Agile methods must be tailored to fit the situation and context. In other words, Agile is not a silver bullet. Transitioning DoD systems—and their associated socio-technical ecosystems—to Agile therefore requires considerable work from DoD agencies and the defense industry base, and is not without hurdles.

For example, we found that adapting Agile into the DoD acquisition life cycle presents unique challenges and opportunities. The challenges are in meeting existing DoD milestone and regulatory criteria when there is little if any guidance available on how to do so when using Agile methods. The opportunities, however, provide the capability to accomplish development with frequent results. Other hurdles identified by DoD programs we interviewed include

  • providing the right team environment allowing access to end users
  • determining how to train and coach the government staff
  • instituting suitable oversight methods
  • adapting rewards and incentives to the Agile environment
  • adjusting to a different team structure

These hurdles are due to a different approach than business as usual and a general lack of specific training and guidance on Agile concepts and approaches within the DoD. Overcoming these hurdles requires changes to the waterfall-centric organizational culture that is common within DoD acquisition programs. These culture changes also require mindset changes because the underlying paradigm for implementing Agile methods is different from that used for the waterfall method.

After studying the topics described above to gain a preliminary understanding of the use of Agile within the DoD, we then studied other management, acquisition, and technical topics, as described below.

Agile Management and Acquisition Topics

Our project also is studying the following management and acquisition topics relevant to the effective adoption of Agile methods in the DoD:

  • Being Agile in the DoD. Agile methods provide promising techniques for streamlining the acquisition process for systems within the DoD. To meet the challenges of adopting Agile methods, however, DoD program management offices must take specific actions to assist in Agile adoption and even enable it. For example, to ensure successful Agile adoption, DoD organizations must plan for it, train for it, anticipate changes in their environments and business models to ensure the benefits of Agile become a reality.
  • Managing and contracting for Agile programs. Managing large-scale, complex software-reliant systems is always hard, but management in Agile programs takes on some added dimensions. For example, program managers not only must be leaders, they must also be coaches, expeditors, and champions. If they do not personally perform these roles, someone in their organizations must responsible for them. These additional roles are needed due to the paradigm associated with using Agile methods and the lack of any significant experience by current DoD personnel in that arena. A particular management concern is the selection and implementation of appropriate contracting vehicles to support the types of practices that successful Agile projects exhibit.
  • Technical milestone reviews. One sticking point in employing Agile methods is how to accommodate large capstone events, such as the preliminary design review (PDR), critical design review (CDR), and others. While many concerns exist in this area, it’s important to focus on the purpose of holding these reviews in the first place: to evaluate progress on and/or review specific aspects of the proposed software solution. Expectations and criteria must therefore reflect the level and type of documentation that is acceptable for the milestone, which is no different from business as usual. The key, however, is to define the level and type of documentation required for the specific program while working within an Agile environment.
  • Estimating in DoD Agile acquisition. Estimation done on Agile projects is typically not the same as the traditional methods used on legacy systems within DoD. Traditional methods tend to focus on estimating details up front; these details are then modified as more information is obtained. In contrast, Agile estimates are often “just-in-time,” with high-level estimates that are refined to create detailed estimates as knowledge of the requirements matures. Some tools within the traditional estimation community are now adding modules to address Agile estimation.
  • Moving toward adopting Agile practices. Change is hard—especially for large DoD ecosystems—and understanding the scope of changes is essential. Organizational change methods must be employed to help DoD organizations successfully adapt to applying Agile. There are multiple adoption factors (such as business strategy, reward system, sponsorship, values, skills, structure, history, and work practices) that must all be addressed. Change-management best practices include understanding the adopter population, understanding the cycle of change, understanding the adoption risks, and building transition mechanisms to mitigate adoption risks. Organizations we studied that had successfully adopted Agile methods typically achieved the following goals:
  • Found and nurtured good sponsors for Agile adoption
  • Understood the adoption population they were dealing with
  • Conducted a readiness assessment that addressed organizational and cultural issues
  • Analyzed what adoption support mechanisms were needed for a particular context and built or acquired them before proceeding too far into an Agile adoption

SEI Agile work continues and the following additional documents are—or will soon be—available:

  • Case Study of Successful Use of Agile Methods in DoD: Patriot Excalibur 2011-TN-019
  • Agile Methods: Changing the Viewpoint of Government Technical Evaluation 2011-TN-026
  • A Closer Look at 804: A Summary of Considerations for DoD Program Managers 2011 SR-015
  • “DoD Agile Adoption—Necessary Considerations, Concerns, and Changes” in the Jan/Feb 2012 issue of CrossTalk

In addition, the SEI has created an Agile Collaboration Group to advise, review, enhance, and validate SEI acquisition work. The SEI is working with this group to create, calibrate, and validate a contingency model that will help acquisition professionals determine when to use Agile techniques, as well as how to identify potential risks if Agile methods are adopted. We are also creating guidelines that summarize best practices and instruct users of Agile methods on how to apply these methods effectively in DoD environments.

Agile Technical Topics

Agile software development has historically succeeded in small-scale (largely IT-based) commercial environments due largely to its easy-to-apply practices for tracking project status and allocating the development resources to those activities that deliver the most potential customer value. A key technical challenge for DoD projects, however, involves balancing the short-term and long-term needs. In particular, the cliché “You aren’t gonna need it (YAGNI)” is a principle in eXtreme Programming (XP) that implies developers should not add functionality until it is necessary, thereby eliminating a considerable amount of unused code in a system. The YAGNI principle rarely seems to apply, however, in large-scale DoD environments, where systems must operate for decades with continual flux with respect to evolving requirements, technology upgrades, new partners, and different contractors.

The SEI is conducting the following technical work on successfully creating and applying Agile methods for the DoD:

  • Agile at scale. This work focuses on providing methods and techniques for applying Agile software development practices for large-scale DoD programs, with improved visibility into the release plan and the quality of the system. One of our activities to address the use of Agile development at scale was conducting a field study with organizations that deal with the challenges of Agile and architecture practices at scale. Based on our observations, we developed a readiness, best practices, and risk analysis technique. Striking the proper balance between developing the system and the architecture in an agile manner while providing enhancement agility for maintaining the system is key to success. We observed that tactics that help organizations to succeed within an Agile environment include paying close attention to architecture-centric practices where a balance of feature development and architecture development is achieved.
  • Technical debt. Our work in technical debt analysis again focuses on architecture and looks at strategically incurring technical debt (such as applying architectural short-cuts) to improve agility in the short-term. This work focuses on developing techniques to monitor and respond to emerging rework, as well as the need to refactor or rearchitect the system to pay back the debt. The need to refactor or rearchitect DoD systems arises in several ways. For instance, system quality degradation, such as unacceptable end-to-end performance, might require refactoring. Such quality degradation-related rework can appear if the development teams focused solely on feature-oriented decomposition of the system to deliver features at early iterations, but didn’t provide the necessary architecture for the infrastructure in a timely manner. Refactoring would require the restructuring of the existing body of code to alter its internal structure (architecture) but not change its external behavior to address the decrease in quality.
  • Modeling decision impact on agile development. Our work also provides guidance and techniques that enhance the applicability of mainstream Agile and lean software development methods to DoD stakeholders by balancing their acquisition and technical needs. In a recently started project, for example, we are investigating acquisition and architecture activities during the pre-engineering manufacturing and development phase of the acquisition lifecycle. This work closely examines modeling decision dependencies and analyzes their impact on the ability to conduct effective Agile system development. This work targets the perspective of reducing integration risks in large-scale DoD systems.

In summary, our projects have found that Agile methods can indeed provide both tactical and strategic benefits in the DoD. The tactical benefits of lower cost, on-time delivery, and increasing quality are clearly important as the DoD places a growing emphasis on greater efficiency in its acquisition processes. The strategic benefits of responsiveness and more rapid adaptability to the current situation, however, may be of even greater value in today’s world, where the DoD must get results faster and be better aligned with changing needs to prepare for an uncertain future. As our work progresses, we will periodically post our progress, ask questions, and request feedback. If you have any questions or feedback on the current work, please post in the comments below.

Additional Resources

Please see the following SEI technical reports and notes for more on Agile development:
Considerations for Using Agile in DoD Acquisition
Agile Methods: Selected DoD Management and Acquisition Concerns 
Documenting Software Architectures in an Agile World
CMMI or Agile: Why not Embrace Both!  
Incorporating Security Quality Requirements Engineering (SQUARE) into Standard Lifecycle Models 
Secure Software Development Life Cycle ProcessesA Technology Scouting Report
Integrating Software-Architecture-Centric Methods into Extreme Programming (XP)

Share this

Share on Facebook Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

7 responses to “Using Agile Effectively in DoD Environments”

  1. Geoff Draper Says:
    Thank you for this excellent article. I look forward to studying the references you cited in further detail.

    The NDIA Systems Engineering Division accepted an action from our DoD partners at our recent 2013 planning session to better reconcile agile development with the DoDI 5000.02 acquisition system. We will be putting together a task group to investigate many of these issues, which sound very similar to what your research already seems to be capably addressing. We would be interested in discussing how we might collaborate further on these issues together.

    Geoff Draper
    Harris Corporation
    Vice-Chair, NDIA Systems Enginering Division
  2. Mary Ann Lapham Says:
    You're very welcome. I'm glad the article was useful. I would also be interested in discussing how we might collaborate. Let's plan on contacting each other after the New Year.

    Mary Ann Lapham
    Principal Engineer
    SEI
  3. Adrian Jasso Says:
    Mrs Lapham,
    Here is another article worth reading reference Agile methods in DoD Acquistion from Ward and Kennedy.

    http://www.dau.mil/pubscats/PubsCats/AR%20Journal/arj64/Kennedy_Ward_ARJ63.pdf

    In addition, while the next article does not explicitly document Agile, it uses Agile Methods as a means to successful requirements development. In this case, relevant and continuous input from the user in support of documents like the JLTV CDD.

    http://www.dau.mil/pubscats/PubsCats/AR%20Journal/arj64a/Pflanz-Yunker_ARJ64.pdf

    Thanks again for your hardwork with Agile research.

    V/R,

    MAJ Adrian Jasso
    US Army Aquisition Officer
  4. Mary Ann Lapham Says:
    Thank you for the additional references. I'm always interested in other perspectives on the agile space within DoD.
  5. Michael DeKort Says:
    My concern is that Waterfall and Agile methods each cause problems relative to cost and schedule

    Discovery

    Discovery = discovery of scope, risk, architecture, design, code, test etc

    Approaches – Serial and parallel

    Waterfall - uses parallel Discovery mostly upfront. Thinking the one time upfront effort is so good that no significant Discovery remains

    Agile - uses serial discovery. Thinking it will Discover as it goes when it needs to.

    Both methods think they are the best approach to handling Discovery for most software projects regardless of product or industry.

    Both methods are exposed by levels of complexity and challenges. Challenges like newness of technology, scope etc. If the project has very low complexity and is not challenged either method can work well. However as those factors go in the opposite direction each method experiences problems for opposite reasons. (This all assumes quality/scope, cost and schedule all matter regardless of priority)

    Execution Reality

    Waterfall starts of knowing much more and will get a much faster head start. However it will soon be stymied or undone by changes its Discovery missed

    Agile starts off much slower since it does not have enough information to start any efforts in parallel. It may churn unnecessarily to catch up to where Waterfall started – creating waste of time and money.


    Resolution

    Combine the best of the upfront parallel discovery with the serial execution of that.

    Perform enough high level Discovery to give Agile a running start without going so deep as to waste time and money on areas where change is possible or even probable
  6. Mary Ann Lapham Says:
    Thank you for your insight. As you've pointed out, both methods have challenges. Neither is a silver bullet. Many proponents of the Agile methods use what some call a Sprint 0 to do as you suggest - perform more discovery up front before starting. We are also finding that many organizations are creating hybrid solutions to meet their specific needs - applying the "best of both" to meet their particular project/environment needs.
  7. Adrian Jasso Says:
    Ms Lapham,

    After over a year, its good to see the article above ties to your recent technical note below. BTW, your note mentions the new upcoming DoDI 5000.2. I wanted to let you know that there is a interim version (perhaps with true intent of agile) released from the USD(ATL) available for review. I think it is worth looking at as your studies move forward. If your don't have it, it can be obtained open source at no charge.

    https://resources.sei.cmu.edu/asset_files/TechnicalNote/2014_004_001_77799.pdf

    Thanks.

    V/R,
    Adrian Jasso

Add Comment


Leave this field empty: