Revitalization of systems engineering in DoD: implications to product cost control.

Many systems approaching an acquisition milestone review come before the Defense Acquisition Board (DAB) without demonstrating sound management practices firmly based in systems engineering. Our analyses of a sampling of major acquisition programs show a definite linkage between escalating costs and the ineffective application of systems engineering. It is clear to me that our budgets are only going to become tighter, public scrutiny is only going to become stronger, and demands for our services are only going to come faster.

[ILLUSTRATION OMITTED]

Making Revitalization a Priority

Consequently, we've made the revitalization of systems engineering a priority within the U.S. Department of Defense. We have taken the first steps to reinvigorate policy, guidance, education, and training, as well as to develop program support and outreach. We expect to see a reduction in acquisition risk, which ultimately translates to improved product cost control over the entire life cycle.

Our primary goal is to re-establish DoD's systems engineering prowess and to let that expertise flow down to our industry. We will accomplish this through systemic, effective use of systems engineering as a key acquisition management planning and oversight tool. In addition, we will promote systems engineering training and best practices among our acquisition professionals.

Policy Shows Way

In our review of existing systems engineering policy, we identified specific gaps in policy that required immediate attention. In my Feb. 20, 2004, policy memorandum, I directed that:

  All programs responding to a capabilities or requirements document,
  regardless of acquisition category, shall apply a robust systems
  engineering approach that balances total system performance and total
  ownership costs within the family-of-systems, systems-of-systems
  context. Programs shall develop a Systems Engineering Plan (SEP) for
  Milestone Decision Authority (MDA) approval in conjunction with each
  Milestone review, and integrated with the Acquisition Strategy. This
  plan shall describe the program's overall technical approach,
  including processes, resources, metrics, and applicable performance
  incentives.

With this policy, we have established the SEP as the cornerstone of the systems engineering revitalization effort. "Early and persistent systems engineering" is a theme now emphasized by policy; and the SEP, mandated at a program's earliest milestone decision, does just that. For systems coming before the Office of Secretary of Defense (OSD) DAB review, the OSD staff is responsible for providing an assessment of readiness based on the program's achievements against the planned activities documented in the SEP.

[ILLUSTRATION OMITTED]

On Oct. 22, I issued an addendum to this policy, focusing on two aspects. First, I directed each program executive officer or equivalent entity to revitalize systems engineering. Each must have a chief systems engineer who will review assigned programs' SEPs, oversee the SEP implementation, and assess the performance of subordinate chief systems engineers. Next, I demanded further rigor in the procedures for technical reviews: reviews must be event-driven, instead of schedule-driven. In other words, reviews should be conducted when the system meets review entrance criteria as documented in the SEP. Additionally, unless waived by the SEP approval authority, reviews must include participation by subject matter experts independent of the program.

Guidance Provides Reinforcement 

The policy has been reinforced by explicit guidance from my systems engineering flag bearers. Mark D. Schaeffer and Dr. Glenn Lamartin, director of defense systems (DS), have emphasized that the SEP should convey the core information needed to understand the systems engineering approach planned for a program and how that approach is integrated with the overall program management activities, including risk management, contract management, and financial management. The SEP should answer the following questions:

* What are the technical issues?

* Who has responsibility and authority for managing the technical issues?

* What processes and tools will be used to address the technical issues?

* How will that process be managed and controlled?

* How is that technical effort linked to the overall management of the program?

Guidance documents recently released include the DS interim guidance memorandum (March 30, 2004); a systems engineering chapter in the new Defense Acquisition Guidebook <http://akss.dau.mil/dag/>; and the SEP Preparation Guide <www.acq.osd.mil/ds/se/publications.htm>. These documents emphasize the changes in the Department's approach to systems engineering, which specifically addresses:

* SEP purpose, contents, use, integration with other program documents

* Phased systems engineering activities with new emphasis on pre-Milestone A and post-Milestone C systems engineering processes

* Systems engineering leadership from senior technical leaders in a component down to technical staff on a program

* Event-driven technical reviews' timing, critical questions to be answered, participation by technical experts from outside the program (i.e., peer review).

Emphasis on Systems Engineering Overdue 

"Early and persistent" is our clear message, and it is reflected in these documents. We believe that the earlier in a program's life cycle that requirements are intensively managed by the systems engineering processes, the greater the likelihood that the program's cost and schedule estimates will be on target. And when these steps are documented in a SEP, the program will be supported by quantified technical data that can be scrutinized in a program's technical reviews.

We have reviewed many cases where programs have not delivered as promised. These programs failed to conduct the required systems engineering analyses before setting requirements, and the programs were prematurely launched. Gaps between resources and requirements were not discovered until well into product development. Many programs trace their rising costs and lagging schedules to requirements-based problems such as poor program definition, lack of traceable allocations, and incomplete or weak verifications.

[ILLUSTRATION OMITTED]

Our studies show that in cases where programs were started with requirements that exceeded resources, costs increased from 55 percent to nearly 200 percent, and schedule delays jumped an estimated 25 percent. Early application of systems engineering will give DoD's top decision makers the necessary confidence in a program's ability to define and match technical requirements with resources--in other words, to stay on budget and on schedule--and to define, understand, and manage program risk.

In addition, several programs we reviewed had completed less than 26 percent of their engineering drawings prior to their critical design reviews. These programs experienced cost overruns from 23 to 182 percent and schedule delays of 18 months to more than three years. Contrast this with commercial firms that typically have more than 90 percent of engineering drawings available prior to a critical design review.

These facts clearly show that our renewed emphasis on systems engineering and the concomitant technical review planning and conduct are way overdue. Technical reviews, particularly with peer participation from outside a program office, are more likely to identify immature technologies and proscribe intensive risk mitigation and technology maturation efforts before a "fragile" technology becomes disruptive. We have found programs that were at low maturity levels, and yet the acquisition program was launched despite a significant gap between technology maturity and weapon system requirements. For example, in one program this gap was not closed until well into the development, and problems with technologies were a main contributor to the program's 88 percent cost overrun and 62 percent jump in schedule.

Education and Training Updated and Expanded

Policy and guidance need reinforcement throughout the extended acquisition workforce. We are introducing a number of changes that will re-emphasize the teaching of sound technical program management. The formal training available for our acquisition workforce will soon include a new introductory course in systems engineering, and the intermediate and advanced systems engineering courses are getting substantial revision.

Defense Systems, along with the Defense Acquisition University, is designing this new introductory course to address basic systems engineering processes and their relationship to other acquisition and program management processes. Intermediate and advanced systems engineering courses are undergoing revision to reflect the new policy and guidance. In addition, they are refocusing on application of systems engineering processes by life cycle phase, as well as on systems engineering leadership and technical program management.

These formal courses are enhanced by a number of new online continuous learning modular courses. This year, we introduced two new ones: Reliability and Maintainability and Technical Reviews, both accessible from <www.dau.mil/>; a third, System Safety, is in development.

Key to the successful implementation of systems engineering is the relationship between program management, contract management, and financial management. It is vital that program managers, contracting personnel, and finance personnel understand that effective "early and persistent" application of systems engineering contributes to program success. Thus, we are also working with DAU to make sure that the acquisition, program management, contract management, and financial management curricula answer the question, "Why systems engineering?"

Outreach and Partnerships Essential

We are reaching out with program support in two key areas. First, we are changing the way we conduct program reviews. We have developed a tailorable common assessment process methodology that serves two purposes. One, it provides systems engineering support to program managers at their request. Two, it supports the DoD's decision makers prior to milestone reviews by providing a context for technical decisions on individual programs. Early results from these program assessments indicate that most of the issues could have been avoided through rigorous systems engineering improvements. Program offices have overwhelmingly accepted the recommendations made to date, resulting in lower program risk and added cost savings. We will continue to drive sound systems engineering into programs through our reviews.

Next, we are reaching out and supporting our programs with a Systems Engineering Forum, first convened in April 2004. Meeting almost monthly, the forum provides a venue for planning and discussing the Department's systems engineering initiatives. This gives members from across DoD and other government acquisition agencies the opportunity to share ideas at the senior executive level.

Systems engineering reinvigoration would not be complete without strong industry involvement. Among the most active of our external partners is the National Defense Industrial Association. The NDIA Systems Engineering Division has the mission "To promote the widespread use of systems engineering in the DoD acquisition process in order to achieve affordable and supportable weapon systems that meet the needs of the military users, and to provide a forum for the open exchange between government and industry to trade ideas and concepts, and develop a new understanding of a streamlined process."

Members come from across the full range of the defense contractor community, from largest systems integrators to smallest small businesses. We work with industry associations such as NDIA to share ideas, concerns, and best practices. We join with them in annual conferences, special-topic forums, and fact-finding reports. Partnering in systems engineering reinvigoration with industry is a key to program success. We believe program offices must set expectations regarding the sound application of systems engineering and work with contractors to comply with our new expectations. So far, we have had a positive response from our industry partners.

From Unique to the Norm

The goal of systems engineering is to see problems on the horizon so we can address them before they hit and potentially destroy a program. We will need everyone--at DoD and in industry--to drive systems engineering back into acquisition programs. It will be up to each and every one of you to implement our new policy and guidelines, as well as to apply the guidance appropriately to your program.

You are the people in the trenches. You are the people who will be held accountable. And you are the people who can make or break a program.

Just in case you were wondering how I define program success. Let me give you a wonderful example: Aegis Ballistic Missile Defense Long Range Surveillance and Track Development and Deployment Team.

The team fully embraced systems engineering by successfully integrating a new capability into the Aegis weapon system that detects and tracks both long-range and intermediate-range ballistic missiles. The system then reports that information to the nation's ballistic missile defense system. This was not a simple achievement. It was a Herculean effort made possible through a sound, disciplined systems engineering approach. It should come as no surprise to you that the Aegis Team was the 2004 winner of the Team NDIA Systems Engineering Award. That is what I expect from all our programs. I want to see that practice go from being unique to being the norm.

Wynne is the acting under secretary of defense for acquisition, technology and logistics. Schaeffer serves in a dual capacity as the principal deputy, defense systems, and as the director, systems engineering in the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics.

Comments and questions should be addressed to [e-mail].