Needed: Improved Collaboration Between Software and Systems Engineering

Systems Engineering Add comments

By Sarah A. Sheard
Senior Engineer
Software Solutions Division
This post is the first in a series on this topic

Sarah SheardThe Government Accountability Office (GAO) recently reported that acquisition program costs typically run 26 percent over budget, with development costs exceeding initial estimates by 40 percent. Moreover, many programs fail to deliver capabilities when promised, experiencing a 21-month delay on average.  The report attributes the “optimistic assumptions about system requirements, technology, and design maturity [that] play a large part in these failures” to a lack of disciplined systems engineering analysis early in the program. What acquisition managers do not always realize is the importance of focusing on software engineering during the early systems engineering effort. Improving on this collaboration is difficult partly because both disciplines appear in a variety of roles and practices. This post, the first in a series, addresses the interaction between systems and software engineering by identifying the similarities and differences between the two disciplines and describing the benefits both could realize through a more collaborative approach. 

Origins of Systems Engineering and Software Engineering

Systems engineering is an interdisciplinary field of engineering that focuses on how to design and manage complex engineering projects over their lifecycles. System engineering textbooks first appeared around 1960, long before the advent of software engineering. However, systems engineering only emerged as a discipline (complete with journals, documented practices, and academic departments) in the 1990s, well after software engineering had been an established field in computer science. As part of the maturation of systems engineering, two capability models were created that documented its practices; by the time they had been merged and then folded into the Capability Maturity Model Integration (CMMI) model, they had become mainstream. Standards were written in the 1990s that document systems engineering, and in the 2000s they generally have been harmonized with software standards. 

The discipline of software engineering traces its roots to a 1968 NATO conference when Mary Shaw named “software engineering” as an aspiration, a field that was needed but not yet developed. Over the next several decades, software engineering both became better defined and grew in new directions. In 1996 Shaw categorized software engineering eras as “Programming-any-which-way,” (1955-1965), “Programming-in-the-small,” (1965-1975), “Programming-in-the-large” (1975-1985), and as “Programming-in-the-world” (1985-1995). The field continues to evolve, with recent focus on topics such as mobile programming and big data. 

Present Day

Software engineers today have adopted many of the principles and practices of systems engineering. For example, developers do not merely react to requirements; they elicit, prioritize, and negotiate changes to them. Developers do not just build code; they architect it, plan it, and establish rules to ensure data is appropriately shared and processed. Software project managers do not just distribute development tasks among individual coders and hope they integrate a seamless final product; they ensure that developers collaborate using plans, processes, and daily standup meetings. Software engineers do not just write programs; they design modular pieces, define integration principles, build for interoperability and modifiability, and perform integration testing.

Meanwhile the systems whose development programs employ systems engineers are increasingly dependent on and controlled by software. For example, the effort to write and maintain software on fighter aircraft now exceeds all other costs. Systems engineers are responsible for ensuring that every part of the system works with every other, and because software represents the intelligence of human-made systems, it is especially important to the integration of the system. Yet many chief systems engineers and program managers have had more experience in mechanical or electrical engineering than in software or software engineering. 

Overlap and Uniqueness

Today, systems engineering and software engineering activities overlap to a great extent. The figure below shows activities common to both disciplines in the middle (purple), activities related more to systems engineering than software on the left (red), and activities related more to software than systems on the right (blue). The term “hardware” appears only on the right, because software engineers use it to mean anything that isn’t software, but systems engineers generally do not use it. Instead, they call physical subsystems by name (e.g., power, propulsion, structures) or describe disciplines or domains (e.g., mechanical engineering, survivability, software, orbit analysis, launch sequence).

Software and Systems Engineering Tasks

It’s important to note in the figure above that systems engineers’ responsibilities tend to be broad but not deep, certainly not deep in software. Conversely, while software engineers’ responsibilities, may be broad across the software, they are deep in areas necessary to create working code. While most of the comparisons in the figure are fairly intuitive, three points require some elaboration: customer interface, non-functional requirements, and what each group does not do.

  1. Although customer interface is a systems engineering role, the software group must participate when discussing software requirements and design. For example, software engineers in Agile sprints often interface one-on-one with a customer representative to select the next development chunk, based on customer priorities. 
  2. Systems engineers are responsible for ensuring that non-functional requirements are met across the system (qualities they call “ilities” that include reliability, usability, and producibility, for example). Software engineers focus on software “quality attributes” (such as reliability, usability, and maintainability, for example.) The two concepts are clearly similar, but there are some differences; for example, software's counterpart to hardware producibility, namely creating the second and later copies, is trivial, and security and maintainability are more important.
  3. The contrast between systems and software engineers is most evident in what they do not do. The role of systems engineers is to balance concerns and perform tradeoff analyses at a high level and not get immersed in details in any area; they consistently delegate details to specialists. System engineers cannot say a task is not their job: if no one else is doing it and it needs to be done, the task defaults to systems engineers. In contrast, software engineers (and mechanical, electrical, etc. engineers) focus their attention inside a domain, mastering details and keeping up with evolving practices and technologies.

Theory and Practice

It is important to note that what systems and software engineers should do is not always the same as what they actually do. In fact, in our experience at the SEI, the majority of complaints each group has about the other results from a failure to meet best practices, rather than from significant differences in best practices. If the waterfall model or a Vee lifecycle model is applied as a rigid sequential process, software engineers can think systems engineering does not apply to software. If the program organizes software engineers only into integrated product teams other than the team responsible for systems engineering, integration and test, then software engineers can get frustrated with their lack of leverage. But good systems engineers also find these situations to be frustrating. Similarly, both systems and software engineers get frustrated if “agile” is used as an excuse to address only nominal-case functional requirements and not system level requirements like security and scalability. 

Needed Knowledge

Most of today’s systems engineers were not software engineers or computer scientists first, nor do many of them have significant background or experience with software engineering techniques, tools, and methods. What software concepts they need to know to perform systems engineering on software-reliant systems effectively is an interesting question, which I will address in a future blog post. Few universities offer a joint degree in systems and software engineering (exceptions include Cornell University and Carnegie Mellon, which offer a joint certificate in systems and software engineering).  Moreover, systems engineering programs ordinarily do not require a foundational understanding of software. 

The Graduate Software Engineering (GSWE) 2009 guidelines provide recommendations for how to integrate systems engineering into software engineering education. The Systems Engineering Research Center/University Affiliated Research Center (SERC-UARC) published a Systems Engineering Body of Knowledge and Curriculum to Advance Systems Engineering (BKCASE) in 2012, which includes a graduate reference curriculum for a systems engineering master’s program, but its relationship to software is minimally discussed (in two pages). 

The Road Ahead

The earlier part of this blog identified the similarities and differences between the disciplines of software engineering and systems engineering.  The remainder of this blog outlines the benefits both could realize through a more collaborative approach. At this time, there are many disconnects between software and systems engineers that could be minimized through better collaboration. Software engineers could get involved in key systems architecting decisions so that the best software architecture could be easier to build. Systems engineers could reduce work that the software engineers find to be non-value added, and would receive help from software engineers in analyzing tradeoffs. In sum, a breadth+depth combination would identify innovative ideas to solve problems and exploit opportunities.

To ensure that future software-reliant systems (and systems of systems) can be built effectively, a new collaborative relationship between systems engineering and software engineering must become the norm. Moreover, there is a clear and compelling need to understand which roles must be filled by which types of professionals and which actions must be performed to ensure the two disciplines work together effectively. Improving the relationship will require overcoming the negative emotions that sometimes appear. (Both groups at a talk on systems and software that I presented at INCOSE Three Rivers Chapter in July 2013 agreed that the others “are arrogant and not as smart as they think they are.”) By learning to work together effectively, systems engineers and software engineers can help increase the probability of success of new and planned ultra-large-scale software-reliant systems and systems-of-systems. In particular, they must understand each other’s strengths and objectives and work together to support them. 

In future posts in this blog series, I will address the questions systems engineers must ask software engineers at the earliest program phases, and vice versa. I will examine the knowledge that systems engineers should have about software and propose how to provide them with that knowledge. I will also show that typical software engineering tasks map to systems engineering tasks and fulfill much of what customers are looking for when acquiring software-reliant systems.   

What do you think systems engineering should know about software engineering? What should software engineers know about systems engineering? Please leave us feedback in the comments section below.

Additional Resources

To read the SEI technical report, The Business Case for Systems Engineering Study: Results of the Systems Engineering Effectiveness Survey, please visit
http://resources.sei.cmu.edu/library/asset-view.cfm?assetid=34061.

Share this

Share on Facebook Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Save this page on your Google Home Page 

11 responses to “Needed: Improved Collaboration Between Software and Systems Engineering ”

  1. Jim Kirby Says:
    Sarah,

    Thanks for a very interesting and compelling discussion of software engineering and systems engineering.

    I look forward to hearing more.

    I have to challenge you on your statement in Overlap and Uniqueness that "producibility is less important for software."

    Perhaps you have definitions of the term for systems and software that make the statement true, but that would deserve a discussion all by itself.

    I understand software producibility to be the capacity to design, produce, assure, and evolve software-intensive systems in a predictable manner while effectively managing risk, cost, schedule, quality, and complexity. This definition comes from the National Resource Council report, "Critical Code: Software Producibility for Defense."

    Considering how important software is to Defense, national critical infrastructure, the economy, and government, I think it's hard to overstate the importance of software producibility.

    Best regards,

    Jim Kirby
  2. Teresa Says:
    A lot of the problems that occur seem to be cultural ones--in other words, they vary by company, by division and/or group--so, it's entirely possible that what is a problem for one group is not for another. In my experience, the software architecture tends to be overlooked. It can be generated by the software group, but the requirements should be nearly as firm as those in systems and its development must precede that of the rest of the software. Systems engineers need to learn to appreciate how the architecture can affect the modularity and portability of a software system.
  3. Daniel Villani Says:
    First off, my own career experience tells me that the passage in this blog that should be written across the sky in letters of fire is "System engineers cannot say a task is not their job." On a single multi-year project I at one time found myself personally printing and applying labels to the front panel of a flight electronics unit, and at another time digging deep into the depths of some C++ code to find out why it wasn't writing output when we expected it to. And my title was "Mission Systems Engineer." So I think the "don't do details" comment might be more correctly expressed, "don't do details if you can get anybody else ... ANYBODY else ... to do them!" Looks like a well-thought-out blog, I'm looking forward to the follow-ups.
  4. Sarah Sheard Says:
    Jim, thanks for your feedback. As you can see we have changed the word "producibility" in the blog, as you are right, it has a different meaning in the software world, namely the entire engineering and development process.
  5. Sarah Sheard Says:
    Teresa, absolutely organizations differ, and differ markedly. I think the (relative) neglect of software architecture is a fairly common, in all but the most software-intensive and mature organizations. I am quite interested in what systems engineers need to learn about software architecture so thank you for the input.
  6. Sarah Sheard Says:
    @Daniel, thank you for the examples!
  7. Jim Kirby Says:
    Agreed that "software's counterpart to hardware producibility, namely creating the second and later copies, is trivial." However, for a variety of reasons, copies are often not identical to a first one. It's well-known how error-prone are today's software practices. Depending on the nature of the system, an international infrastructure delivers changes to operational software to fix errors, patch vulnerabilities, and improve the software for a variety of reasons. Of course, subsequent software copies reflecting these changes will not be identical to their predecessors.

    Further, the hope that we can freeze requirements is a vain one. They will change, not just during development, but during operation. As will infrastructure, technology, and environment of use. This software reality is one of the motivators for software product lines, software families, and design for change.
  8. Darragh McCurragh Says:
    "focusing on software engineering during the early systems engineering effort" - the problem as I see it and experienced it in consultancy at large and medium software projects, some very time-critical where an "average 21 month delay" would simply have been deadly was always that during the requirements phase the future users (i.e. those that did the same work today, albeit using other means, e.g. pen and paper, spreadsheets, an outdated legacy system etc.) were not given time off their usual duties to formulate requirements nor did they see how their input would actually get recorded in a way that made a difference. Plus, and this actually seems to be the main problem (!) the software engineers and developers to this day seem not to grasp that to formulate a prototype you must not go "IPO" (Input-Processing-Output), but the opposite: you have to ask what is the desired output, then construct the software (the "black box") that can produce that output, thus describing the input. Rather they take requirements in the form "Well, user A said, he wants to capture X then get Y as an output" which does not mathematically prove that Y can be derived from Y in all cases.
  9. John Hubbell Says:
    Common practice has rendered the phrase "systems engineering" ambiguous. For the purpose of this post I'll define it as the "science of the superficial", most concerned with the cost-efficient performance, availability and sustainability of the "system". Please recall that a single system may be decomposed into subordinate systems in the classic "System of Systems" view. What appears to be most lacking in the systems engineering process is adequate analysis of the operational scenarios in which the system must perform. These scenarios frame requirements, guide integration, provide the framework for verification / validation as well as operational procedures and transition to operations. Many times the customer / user doesn't understand how the system will be used, a condition that guarantees at least a serious challenge and at worst total failure. Also, there is not a consistent framework to allocate resources between 'bending metal (or bits)' and simulating the system's performance in the operational scenarios. As we used to say in the big time 'sim biz' "... fake it before you build it ...", lest you build the wrong beast. A natural failure mode resulting from absent or faulty scenario analysis is disjoint engineering efforts, including a fracture between systems engineering and software engineering. However, the fractures are not limited to this noteworthy fault.
    If you know how to use the system before you build it and 'fake it' in the process to make sure you're building the right beast, then $uccess is with you, at a price.
  10. Sarah Sheard Says:
    Darragh,

    Thank you for your comment.

    I agree that output (or perhaps, "effect on the world that you wish the system to produce"), then black box, then input is a valid view, maybe the best one to start with. At some point, you also have to have the forward-looking or process view, to ensure what you are attempting will get you what you need to get, and because documented processes are a good way to incorporate lessons learned from the past into our current work.

    The concept of how do you ensure software engineers (or any others, actually) have time to think about long term issues such as formulating robust requirements is perennial in this area.

    On the one hand, software engineers (or more specifically, programmers) are the "manual labor" of the 21st century and without their focused attention on the task, nothing gets built, so managers have a right to ask them to focus on immediate tasks.

    On the other hand, the analogy with assembly lines really fails because software engineering is knowledge work, not "insert this screw and tighten 1/4 turn" repetitive work.

    Today's software *is* knowledge and demands knowledge. Without the time to think, the products may exist, but they won't be right and they won't be robust. Agile development concepts help a bit in that they aren't expected to be right and they evolve, but today allocation of resources to the big picture is a continuing challenge.
  11. Sarah Sheard Says:
    Jim, thank you. Another good point regarding copies not being identical.

    I am working right now on looking at how to capture the “life cycle” of software. I don’t think there is a “life cycle” of software any more than there is a “life cycle” of human DNA. There is an instance (version, or person) and that instance is initiated, grows, and dies, but another version (or more likely many other versions) lives on with nearly identical information in it. And each version can continue to evolve and have many children. Microsoft feels obligated to manually “pull the plug” on old operating systems just to keep the “population” of versions needing support to a reasonable number.

    Especially in the world of systems-of-systems (SoS’s), an SoS does not so much live and die as it evolve. Development efforts occur for constituent systems which are conceived of (as you say, with respect to a particular set of [static] requirements), and when complete they are merged into the operating software environment.

    As you have suggested, by the time this change occurs, the requirements have changed, because the world has changed. Thus we have no choice but to continue to address the changes, hopefully proactively, but often reactively. I wrote a somewhat odd paper for the International Council on Systems Enigneering (INCOSE) in 2013 (“Systems Engineering Complexity in Context”) that touches on this process. The paper is available from INCOSE if you are a member, or I can send you a copy.

    In addition to this motivating product lines, families, and design for change (modifiability as a quality attribute), I think the reality of this problem provides an opportunity for breakthrough research results in the future.

Add Comment


Leave this field empty: