An assessment of the systems engineering continuum.

The Department of Defense continues to place great significance on systems engineering activities, believing these activities are critical to the success of acquisition programs. However, this greater significance must be well-placed throughout the life cycle. Prior to program initiation at Milestone B, appropriate levels of engineering activities must be performed and managed. One such activity, concept strategy formulation, can be accomplished and managed only by a "capability planning" organization with appropriate staffing and cross-domain expertise.

Disconnects at the Seams

Recently, a series of historical systems engineering case studies have been produced by the Air Force Center for Systems Engineering at the Air Force Institute of Technology, Wright-Patterson AFB, Ohio. They were written on a number of programs including the Hubble Space Telescope, Theater Battle Management Core Systems, the C-5 Galaxy, the F-111, the B-2 Spirit, and the Joint Air-to-Surface Standoff Missile. Others studies under way are the A-10 Thunderbolt II, the Global Positioning System, and Peacekeeper Missile Systems <www.afit.edu/cse>. Many important learning principles can be drawn from the studies; one that transcends this body of work is that systems engineering and analysis exist as a continuum across the life cycle. Mission and systems analyses start well before a Milestone B program initiation and must be part of the entire systems engineering continuum. To ensure a continuous and strong set of integrated systems engineering activities, it is necessary to apply rigorous processes and tools early, from the conceptual solutions throughout system developmental and operational life. The cases point out how this thread can break at many points for many different reasons, and show that there are no shortcuts. In particular, the case studies often highlight disconnects at the seams in the continuum, as roles and responsibilities transition between requirements (user), acquisition (product center), and developer (contractor) communities.

Another conclusion drawn from the case studies is that the needs of the program vary and, therefore, different tools, organizations, and skills are required. It will be necessary to continually develop the needed skills and expertise and to educate, train, and retain the people associated with implementing the systems engineering and analysis processes.

Office of the Secretary of Defense and Air Force goals should ensure that early capability planning and analysis are as comprehensive as possible to guarantee that cost-effective solutions are being pursued; early cost estimates for the concepts and systems are reasonably accurate; technical and projected programmatic risks are identified; schedules for concept implementation are realizable; and the concepts will actually deliver the right operational capabilities.

The systems engineering and analysis processes have generally been proved to be effective to meet these goals in the post-Milestone B life cycle phases--during systems design and development, during production and deployment, and during operations and support to sustain and modernize systems. However, the systems engineering processes and tools should also be used during the preconcept and concept refinement phases prior to program initiation.

Pre-Milestone B Systems Engineering Activities

The current DoD Acquisition Management System, described in DoDI 5000.2, is fed by early operational utility analysis, called out in the Joint Capabilities Integration and Development Systems (JCIDS), CJCSI/M 3170. There are numerous events that trigger the start of the JCIDS process. Examples, shown in Figure 1, are a unified look at a mission analysis, a joint examination of a new operational concept, operational shortcomings, technology opportunities, and a broad look at a functional area. The SE-relevant activities prior to concept decision are the Functional Area Analysis (FAA), Functional Needs Analysis (FNA), and Functional Solutions Analysis (FSA). These analyses are documented in the Joint Capability Document (JCD) and Initial Capability Document (ICD).

[FIGURE 1 OMITTED]

The pre-Milestone A activities to address these triggers are as follows:

* Understand problems in joint terms and in terms of attributes and measures of effectiveness.

* Identify and prioritize capability gaps using a joint perspective. Establish basis for concept strategy.

* Identify candidate solutions (not candidate systems) using a joint perspective looking across doctrine, organization, training, materiel, leadership and education, and personnel and facilities (DOTMLPF).

* Evaluate candidates against attributes, measures of effectiveness, and early cost estimates.

* Perform trade analysis on top candidate concepts. Identify candidate measures of performance and key performance parameters, and conduct formal cost/risk assessments.

* Decide on concept(s) and develop a concept strategy. Select preferred system(s).

* For each system, establish user key performance parameters (including determining operational requirements) and define the acquisition strategy (including defining system requirements and performing risk reduction and mitigation).

* Manage the set of possible concepts throughout their life cycles.

These activities are supported by a vast systems engineering toolset including requirements analysis, and model-driven design and development and its formal inclusion in enterprise- and system-level architecture. Also applicable are numerous decision-analysis methodologies incorporating risk inclination, operational scenario modeling and simulation, utility theory, and parametric and analogy-based cost estimation techniques. Lastly, early optimization within competing concepts may be employed. But who will accomplish these pre-Milestones A and B activities?

A broad variety of different organizations is currently responsible for conducting, supporting, and documenting the analyses for each pre-Milestone B activity. This list, which changes throughout the eight previously listed activities, includes organizations such as the Joint Staff; functional capability boards; combatant commanders and their staffs; "user/sponsor" requirements and planning sub-organizations; the acquisition community (product centers and logistics centers); Service acquisition staff offices; laboratories; analysis organizations (Organization for Aerospace Studies); the U.S. Joint Forces Command's director for joint capability development (J8); the under secretary of defense for acquisition, technology and logistics; Office of the Secretary of Defense director of program analysis and evaluation; and even industry.

Concept Strategy

Examination of this end-to-end systems engineering thread uncovered issues regarding the activities leading up to, and surrounding, the concept decision. For example, the ability to achieve improved precision strike capabilities could be accomplished by current manned aircraft with new munitions, new aircraft with either existing or new weapons, new targeting sensors, ballistic missiles, or modifications to artillery. How one makes this early conceptual decision and how one tracks and manages progress of these decisions is critically important.

Most systems engineers are taught that these early decisions have huge ramifications on cost and schedule. Smart decisions can directly determine the majority of life-cycle costs, so the importance of this decision point cannot be overemphasized. The majority of the trade space is given up once the concept decision has been made. Currently, the concept decision is not a major milestone; it precedes concept refinement and analysis of alternatives, which then lead to a Milestone A decision. Often, the concept decision is made with relatively little analysis of the full solution space and even less cost analysis of candidate concepts and systems.

[FIGURE 2 OMITTED]

Applying improved systems engineering and analysis to pre-Milesone A activities needs to be a managed process (across the life cycle) in concert with a concept strategy. This idea is consistent with a recent Government Accountability Office best practice for portfolio management (GAO-07-388). Necessary concept strategies would initially precede and then continue to parallel acquisition strategies. A concept strategy would start during pre-Milesone A, carry into technology development, and be updated throughout the entire life cycle. This would also be a time of risk reduction with technology forecasting inputs and analysis of multiple concepts and solutions.

What could be included in a concept strategy? Extending our previous example of precision strike, let's suppose a decision is made to pursue both new stealthy unmanned aircraft in addition to modified artillery systems. This strategy would reference maturing technologies from the Service laboratories, and it would drive system acquisition. It might address integration of various systems that make up the concept. For example, perhaps some Web services or a networked information system are also required to more effectively push information targeting to both the new unmanned air vehicle and modified artillery. This system might be either a new system unto itself or a modification of existing systems. The strategy should begin to document how requirements flow down from this system of systems into the respective systems. It might also capture the interfaces, information flows, organizations, and operational activities in enterprise or broad mission-area architecture. By managing this early operational analysis, combinations of solutions with high risk/high payoff and/or low risk/lower payoff can be brought forward. Concepts have different technical and programmatic risks and may include both materiel and non-materiel (DOTMLPF) aspects. In our example of precision strike, perhaps various acquisitions regarding the unmanned air system, the artillery, and the network service are part of the mid-term or long-term solution. Perhaps there are immediate non-materiel, short-term solutions. All of these characteristics become the elements of a concept strategy. But who will manage this strategy of integrated capability improvements?

Implementing the idea of concept strategies creates a new approach that is a structured activity for government and industry. It will require collaboration and mutual understanding, resulting in some changes to the overall process. These include moving the concept decision to the Milestone A timeframe so that all the necessary work is completed to define the concepts and refine the concept strategies. The concepts to be pursued are still decided at Milestone A. We suggest an additional part of the solution lies in expanding the role of capability planning and creating an approach and organization to manage the concept strategies.

The Capability Planning Organization

Development planning (today often called "capability planning" or "capability-based assessment") existed in robust forms in the past when the DoD was primarily system- and platform-centric. The development planning mission in the past would seem narrowly defined today and is certainly not adequate for solutions that demand system-of-systems concepts. Further, organizations performing this function were usually adequately funded.

Today, the capability planning function is still vital, but none of those former organizations exists. The analyses must span multiple domains and Services. Operational users have the responsibilities in today's process but neither the time nor the requisite skills. There are some efforts under way to reconstitute some of these capability planning groups within the Air Force. But those that have been reconstituted have no clear role in the process, are excessively domain-specific, and are at too low an organizational level. Lastly, these efforts are struggling to find adequate and consistent funding for their activities. This missing organizational role needs to include the management of concept strategies, in addition to a broad architecting and engineering responsibility, as depicted in Figure 2 on the previous page.

Concept strategies must be managed; this makes it important to discuss who would be the program manager and chief engineer equivalents at the concept level. In many ways, the role of managing the integration across several systems and conceptual solutions is that of a portfolio manager. The concept strategy often dictates a system-of-systems perspective. So the chief enterprise architect (or capability architect) who supports the portfolio manager will need to engineer in this system-of-systems context. The chief architect's main products will be enterprise-level or mission-level architectures, program and capability roadmaps, levels of performance, and coordinated user requirements/capabilities documents.

A top-tier organization should be created at the DoD level, reporting directly to the Joint Staff at the senior level. Note that this places the concept strategy, managed by a portfolio manager and supported by a chief enterprise architect, as reporting within the user as opposed to the acquisition community. Creation of sub-tier organizations should follow at the Service and product center levels, also reporting to their senior staffs. This construct will not work effectively without the creation and maintenance of stable sources of funding for the organizations.

The concept formulation and decision process needs a strategy and a robust management organization to support it. We recommend that capability planning organizations be created, funded, and empowered to manage the enduring joint warfighting capabilities. These capabilities would then be realized through numerous creations, modifications, and disposals of component weapon systems. Adoption of these recommendations by the DoD and the Services would enable the delivery of effective capabilities that come from a sound application of systems engineering and robust analysis.

The authors welcome comments and questions. Contact them at [e-mail], [e-mail], [e-mail], and [e-mail].

Lt. Col. John M. Colombi, USAF * David R. Jacques * Mark K. Wilson * John M. Griffin

Colombi is an assistant professor of electrical engineering at the Air Force Institute of Technology. Wright-Patterson AFB, Ohio. Jacques is an assistant professor of aeronautical engineering at the Air Force Institute of Technology. Wilson is president of Mark Wilson Consulting, providing systems engineering services to aerospace organizations. Griffin is president of Griffin Consulting, providing systems engineering and program management services to large and mid-sized aerospace firms.