Reducing Project Failures by Aligning Acquisition Strategy and Software Architecture with Stakeholder Needs

Acquisition , Architecture Add comments

First in a Two-Part Series
By Lisa Brownsword
Acquisition Support Program

Lisa BrownswordMajor acquisition programs increasingly rely on software to provide substantial portions of system capabilities.  Not surprisingly, therefore, software issues are driving system cost and schedule overruns.  All too often, however, software is not even a consideration when the early, most constraining program decisions are made.  Through analysis of troubled programs, SEI researchers have identified misalignments between software architecture and system acquisition strategies that lead to program restarts, cancellations, and failures to meet important missions or business goals. To address these misalignments, the SEI is conducting new research on enabling organizations to reduce program failures by harmonizing their acquisition strategy with their software architecture.  This blog posting—the first in a two-part series—motivates the problem of misalignment and describes the SEI’s current research for addressing this problem by analyzing program-specific quality attributes associated with business and mission goals.

A Real Problem with Competing Goals

Complex acquisition programs have diverse sets of stakeholders whose goals and priorities may be misaligned. Operational users, combatant commanders, funding authorities, and acquisition team members often think they have the same priorities, but when interviewed, their answers vary widely in terms of the goals and features they consider most important. In too many cases, solutions are created based on the goals of one set of stakeholders, whose goals often conflict with those of other stakeholders.

For example, an organization we encountered in our independent technical assessments of Department of Defense (DoD) acquisition programs showed how failures stemming from misalignment can occur. This organization was going to rebuild a significant system to replace a critical capability. The program manager gave the requirements to the software architect who returned with an architecture he thought was well-suited to the requirements he received. The architect was baffled when the program manager cited the lack of a database as a problem. The architect had intentionally eliminated the database because it resulted—in his opinion—in a more elegant solution. As it turned out, the program manager had an excellent database group who were dependent on work from this program. Though this was a business goal, it wasn’t captured in any of the requirements given to the architect.

In another example, a program with a business goal to reduce the time to field a new system saw reusing an existing software component as a viable approach to provide a significant part of the system’s capability. The structure of this component, however, was not consistent with the envisioned architecture for the new system.  This program fortunately recognized this mismatch early enough to reconsider their approach, but not all programs are this lucky.

Pedigreed Attribute eLicitation Method (PALM): Our Starting Point

The genesis of our research project came about as the SEI was working with a government organization involved in a major system modernization project.  Our prior experience taught us that business and mission goals are key sources of architecturally significant requirements.

We needed a way to identify the requirements that would have the most impact on the software and system architecture to ensure that these requirements were adequately captured, prioritized, and potential conflicts resolved. My former SEI colleagues Len Bass and Paul Clements had recently finished working on the Pedigreed Attribute eLicitation Method (PALM), which provided a good starting point.

PALM enables organizations to systematically identify the high-priority, mission- and business-goals from the stakeholders of a system. The architectural impact of those goals is then captured by determining any quality attribute requirements they imply.  For example, if a high-priority goal of an organization was to ensure seamless continuity of operations in the face of regional disasters or cyber attacks, the architectural impact might be to mirror critical information systems in geographically distributed locations so failovers could be performed in real-time.

We used PALM to run an informal pilot with the organization responsible for the system modernization. PALM revealed a number of gaps between the identified program goals and what had been previously thought to be the architecturally-significant requirements. One gap concerned the program’s business goal to stay within a yearly fixed budget.  The program responded by periodically determining (with the user community) operational requirement priorities and possible deferrals to future releases. On the surface this goal appeared to have little impact on the architecture. On closer inspection, however, the goal had profound impacts on the required flexibility (and modularity) of the architecture to accommodate these changes.

PALM also illustrated conflicts among the goals. If these conflicts were not resolved they would have led to different architectural decisions with different solution implementations. For example, one stakeholder’s mission goal was to establish a next generation operational superiority.  In contrast, a second stakeholder’s business goal was to reduce operational costs by replacing the legacy system (retaining current capability only) as rapidly as possible.

For the program to achieve the first stakeholder’s mission goal, it would need a sophisticated, highly scalable architecture leveraging state-of-the-art technologies. For the program to achieve the second stakeholder’s mission goal, it would require far less sophistication in the architecture and employ state-of-the-practice technologies with known capabilities and performance characteristics.

To our surprise, these competing goals also drove different acquisition strategies.  The goal for operational superiority was supported by an acquisition strategy with an emphasis on architectural and technology prototypes, robust simulations, and continuous integration. The acquisition strategy for the goal to reduce operational costs, conversely, emphasized rapid deployment and tight control of requirements creep. 

Beyond PALM

The application of PALM for our modernization project codified many of its requirements and clarified their impact on the architecture. Discovering examples where architecture decisions affected an acquisition strategy was exciting. Although this type of analysis was outside the original scope of PALM, we could see the possibilities of extending it to provide better acquisition support. This insight became the basis for our decision to develop a new systematic analysis method that leveraged PALM to better align acquisition strategy and software architecture approaches with stakeholder needs. The first phase of this research project involved identifying and articulating the relationships among four key aspects:

  • business and mission goals
  • architectures
  • quality attributes
  • acquisition strategies

In our research we used an interview-based approach to discover and document patterns of alignment among these aspects or misalignment that tend to pull them apart. Since most of our data comes from troubled programs, to date we have discovered evidence of misalignment or anti-patterns.  We are studying these anti-patterns to develop a method that will help programs avoid these misalignments by explicitly identifying and deconflicting mission and business goals through analysis of their qualities attributes that will drive the definition of aligned acquisition strategy and software architecture.

What’s Next

The next blog posting in this series will discuss a number of the anti-patterns we’ve identified, a number of the relationships that are beneficial for alignment, and how we are applying the results of our initial research to extend PALM to enable acquisition programs to better align their mission and business goals. 

Additional Resources

For more information about the Pedigreed Attribute eLicitation Method (PALM), please visit
www.sei.cmu.edu/architecture/tools/establish/palm.cfm

Share this

Share on Facebook Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Save this page on your Google Home Page 

5 responses to “Reducing Project Failures by Aligning Acquisition Strategy and Software Architecture with Stakeholder Needs”

  1. Don O'Neill Says:
    Advocating the alignment of stakeholder goals is spot on.

    Rather than referring to competing goals, it may be advisable to view such occurrences as unreconciled goals to avoid the emergence of unnecessary winner/loser outcomes.

    An example if how goal alignment was done to good effect can be found in the IBM Federal Systems Division performance on the Global Positioning System Ground Station where accuracy was the rallying point for both goal alignment and performance incentives. This is documented in the recent May/June 2012 issue of Defense AT&L Magazine in my article entitled, "A Disruptive Game Changer to Achieve DOD Austerity".
  2. Nicholas Says:
    Aligning requirements with architectural goals has always been a challenge for me. As a business systems analyst with a technical background I often find myself with a conflict of interest - in order to meet short term goals of business stakeholder, the technical architecture is jeopardized. If the misalignments are treated with the same respect/consideration as alignments then I can see plenty of upside to the PALM model.
  3. Lisa Brownsword Says:
    Thank you for your interest in our work! You’ve touched on number of excellent issues. First, for our project, we generally use the term “conflicting goals”, rather than the more emotionally-laden “competing” – for exactly the issue you highlighted (and you found our slip!). The term competing does tend to evoke a winner/loser mindset, which is counterproductive to resolving conflicts. In any program of size, stakeholders will have goals that conflict. One of the major patterns of misalignment that we've found in our research is that these conflicting goals are NOT explicitly identified and then reconciled. Reconciling conflicting goals does not necessarily imply there must be a winner and a loser. We would like to advocate for more of a win-win resolution approach.

    Your comment about the GPS program raises the question of WHEN in the life of an acquisition program should the potential misalignments of goals, architecture, and acquisition strategy be identified? The ideal response is as early as possible in a program’s formation, particularly as the acquisition strategy is developed and BEFORE an RFP is released by the government and contractors submit their proposals. (Although, we think this is something that will need continually attention throughout a program’s life.) In the case of GPS, which was a fixed-price contract, the IBM’s development approach your article highlights two doctrine tenets that are particularly relevant to our discussion. These are “requirements and incentives known from the beginning” and “explicit project goals and readiness to peform”. Underlying both of these tenets is an assumption that all business and mission goals and the quality attributes reflective of those goals are explicitly identified and any conflicts identified and resolved. We would be very interesting to know when and how the program did this.
  4. Don O'Neill Says:
    Lisa,

    You point out an important factor in the program management, systems engineering, and software engineering work environment and culture when you stated, "One of the major patterns of misalignment that we've found in our research is that these conflicting goals are NOT explicitly identified and then reconciled."

    Rest assured that this misalignment is very likely to be known by the stakeholders affected, but reluctance to openly acknowledge the conflict is motivated by the risk of losing something in the resolution process. In a well managed program, each of these stakeholders has a cost, schedule, and requirements baseline plan to which each has made a commitment to perform. In the resolution process, these baselines are subject to review and possible change, that is, there may be winners and losers. So mums the word!

    Programs like GPS are managed along the lines of major milestones like system requirements review, preliminary design design review, critical design review, and initial operational capability. Each of these reviews provides the opportunity to surface oversights and correct defects as well as to revisit goals and cost, schedule, and requirements baselines. So it is at these major milestones that reconciling conflicting goals is best attended to. Between major milestones everyone has their head down performing on the task at hand. Reconciling conflicting goals during these periods is to risk chaos on the project.
  5. Eltjo Poort Says:
    Lisa, I spent last week with some of your SEI colleagues at the WICSA conference. I presented a paper there that highlights the relationship between software quality requirements and acquisition strategy - from a supplier's point of view. It has been known for a long time that the only way to specify quality attribute requirements in an economically optimal way is to develop them together with the architecture. Classical acquisition strategies like sealed bidding, however, do not allow this because of the communication constraints between client and supplier. In spite of this, sealed bidding is still used in 95% of software acquisition - leading to economically suboptimal architectures and trouble projects.
    PS You can find the presentation slides on the WICSA website (it appears I cannot insert a URL in a comment...).

Add Comment


Leave this field empty: