Investigating schedule slippage.

Past research shows that schedule slippage within the acquisition community often adversely affects the cost and performance characteristics within a program. To minimize the risk of underestimating schedule growth, a program manager needs a reliable initial schedule estimate. Statistical models can provide such estimates; however, they require accurate historical data and predictive drivers. Many archival studies have investigated potential drivers of schedule growth. In this article, we review several of those studies that investigated schedule slippage and highlight common potential drivers of schedule growth, ending with a list of variables for estimators to consider for incorporating into future predictive models.

**********

One could compare a System-of-Systems (SOS) approach to building a pyramid. As with any pyramid, the top is only as strong as the foundation. With an SOS approach, that foundation often rests with the individual systems. This in turn implies that any risk that an individual system may be subject to also makes the overall system vulnerable as well. In the acquisition world, two of the biggest risks of any system are cost growth and schedule slippage, which often results in cost growth.

Developing a major weapon system is risky and full of uncertainty. Requirements, politics, economics, and the system's technological design are just a few of the uncertainties that create risk in this venture. This uncertainty manifests itself in the form of variance between the planned schedule and the actual schedule. "Excessive schedules have two significant negative effects: U.S. forces may be left without needed capabilities, and longer schedules often mean higher costs" (Tyson, Harmon, & Utech, 1994, p. S-1). The accuracy of the program schedule estimate, therefore, is critical in planning for future capability needs and correctly accounting for the cost in developing the weapon system.

To gain a handhold on minimizing the risk of underestimating a program's actual acquisition schedule, one first needs an accurate estimate of a program's schedule. Statistical regression models can generate such estimates, but these predictive tools need reliable data and historical trends in order to be viable. For this requisite information, researchers often pull past studies; however, this can be cumbersome if not easily found or at least grouped together. In this vein, we provide an overview of some past research conducted on schedule variance. We describe the reasons given for schedule slippage within the framework of the acquisition environment followed by both descriptive and inferential statistical studies relating to schedule overruns.

From these studies, we go on to tabulate some of the key drivers associated with schedule slippage. These enable researchers to perhaps gain a better handhold on what could cause schedule slippage. This in turn could assist them in building a database foundation from which to develop models to predict more accurate acquisition schedule estimates for major weapon systems. This information could also serve as a possible flag or indicator of when a SOS may experience schedule slippage itself.

POTENTIAL REASONS

The time required to create a new weapon system from idea inception to a fielded system is an important element to understand in the overall acquisition process. Cost and schedule overruns in the development of major weapon systems are continuing problems that plague the acquisition environment. The following studies address and discuss some of the reasons for these overruns.

DREZNER AND SMITH STUDY, 1990

Although knowing the duration and variability of schedule is important, understanding what factors make up the duration and affect the variability are imperative. Based on statistical analysis of 10 programs, the results of a 1990 RAND study by Drezner and Smith suggest the following influences on the original schedule estimate: 1) competition and prototyping lengthens schedule, and 2) concurrency and adequate funding shortens schedule. Results also suggest the following influences on schedule slips: 1) unstable funding, 2) technical difficulty, 3) external guidance, and 4) external events (Drezner & Smith, 1990, p. 33). Two commonly held hypotheses that prove inconclusive for Drezner and Smith, though the small sample size of the study may be a limiting factor, are that longer planning phases incur less slippage and that cost and schedule growth are interrelated. Figure 1 lists the major attributing factors addressed by this study.

FIGURE 1. DREZNER AND SMITH'S 16 SCHEDULE FACTORS

FACTORS AFFECTING SCHEDULE

Factors Affecting Original Plan

1. Competition

2. Concurrency (overlap of effort between development and
production phase)

3. Funding adequacy

4. Inclusion of prototype phase

5. If the program's phases were contracted separately

6. Service priority

(Drezner & Smith, 1990, pp. 21-22)

Factors Affecting Program Deviation

1. Contractor performance

2. External events

3. Funding stability

4. Major requirements stability

5. Program manager turnover

(Drezner & Smith, 1990, pp. 23-24)

Factors Affecting Original Plan and/or Program Deviation

1. External guidance

2. Single service or joint management

3. Program complexity

4. Technical difficulty

5. Concept stability (system specification maturity)

(Drezner & Smith, 1990, p. 23)

TYSON, NELSON, OM, AND PALMER STUDY, 1989

The study, conducted by the Institute for Defense Analyses (IDA), examined schedule variances and its causes. The study's database consisted of 9 tactical aircraft, 9 electronic aircraft, 5 helicopters, 8 other aircraft, 16 air-launched tactical munitions, 18 surface-launched tactical munitions, 10 electronic systems, 10 strategic missiles, and 4 satellites. The main focus of this study was to determine the effect, if any, on schedule overruns from: 1) prototyping, 2) competition, 3) multiyear procurement, 4) design-to-cost, 5) sole-source procurement and fixed-price development, and 6) contract incentives. This is what the researchers found.

Use of prototyping showed a reduction in the development phase and the overall schedule by 11 and 15 percent respectively (Tyson, Nelson, Om, & Palmer, 1989, pp. VIII-6-VIII-7). Competitive programs produced 43 percent more design-schedule growth and 39 percent more production-schedule growth compared to noncompetitive programs (Tyson et al., 1989, p. VII-7). Programs using multiyear procurement experienced 7 percent less production schedule growth (Tyson et al., 1989, p. VI-8). Design-to-cost exhibited development schedule growth of 12 percent and production schedule growth of 2 percent (Tyson et al., 1989, p. IX-11). Production schedule growth was reduced by 27 percent when sole-source procurement is used (Tyson et al., 1989, p. X-7). Under a fixed-price contract strategy, development schedule growth showed a reduction of 6 percent (Tyson et al., 1989, p. X-13). Even though contract incentives were one of the focuses for this study, no comparison was made between contract incentives and schedule.

JOINT STRIKE FIGHTER (JSF) STUDY, 2000

The Department of Defense (DoD) selected the Joint Strike Fighter (JSF) program as a flagship program for acquisition reform with a principle objective to demonstrate a low level of technical risk for critical technologies in the concept demonstration phase (Rodrigues, 2000, p. 1). This strategy recommends focusing on risk reduction efforts by maturing critical technologies prior to entering Engineering and Manufacturing Development (EMD), and being allowed to do so without the penalty of reduction in funding support (Rodrigues, 2000, p. 3).

This study addressed the JSF's acquisition strategy and noted it as encouraging, but also stated that program managers are hesitant to embrace this concept. Once in the development environment, external pressure to keep the program moving becomes dominant, such as preserving cost and schedule estimates to secure budget approval. If a program manager decides that an additional year is needed in the risk reduction or concept development stage to reach the desired level of technical maturity, they run the risk of reduced funding for the development stage, which could lead to program cancellation (Rodrigues, 2000, p. 6). They are more likely to accept a lower level of technology rather than risking the program. Unfortunately, low levels of maturity lead to increased risks, which in turn lead to the likelihood of schedule delays, increased costs, and/or quantity reduction.

DESCRIPTIVE STUDIES

The previous section highlighted a few studies that identified reasons for schedule slippage. They essentially answered the question of what causes schedules to slip. Next, we proceed to relay both descriptive and inferential statistical studies that answer the questions of how and why schedule growth occurs within the acquisition community.

REIG STUDY, 1995

In a study to identify the track record of success for acquisition programs based on the number of Low Rate Initial Production (LRIP) articles used for testing, the office of the Director of Operational Test and Evaluation asked the Defense Systems Management College to research the current acquisition system (Reig, 1995, p. 27). The question that the researchers wanted to answer was the following: Is there a relationship between the number of LRIP articles used for testing in EMD and the success of that program?

The researchers analyzed 24 programs of all types, concentrating exclusively on the EMD phase. None of the 24 programs studied completed the EMD phase of acquisition within the planned schedule, and the average schedule slippage was 63 percent (Reig, 1995, p. 28). For data they used the Selected Acquisition Report (SAR) database and a review of the Blue Books, which are summary program plans and data compiled for the use of Defense Acquisition Board principles and staff assistants immediately prior to milestone meetings (Reig, 1995, p. 27).

The results of the study showed that the percentage of LRIP test quantity (LRIP test quantity divided by total planned production quantity) is an effective predictor of the success of that program. That is, programs using three percent or more LRIP test articles never exceeded a 50 percent slip and the percentage of slip steadily decreased as the three-percent mark was approached (Reig, 1995, p. 28). The LRIP was not the only variable to potentially explain the success of a program. The study identified six other explanatory variables. These included the degree of risk identified at Milestone II, use of competition at Demonstration/Validation (DEM/VAL) phase, use of competition during EMD phase, contract type, number of associate contractors, and joint- or single-Service program (Reig, 1995, p. 29).

For these variables, this is what the researchers concluded. Programs that were medium-risk at Milestone II had a higher average program success than those that were low-risk. Programs that did not use competition in DEM/VAL or in EMD had a higher success rate. As far as contract type, Cost Plus Incentive Fee (CPIF), Fixed Price Incentive (FPI), and Firm Fixed Price (FFP) had equally higher program success in comparison to Cost Plus Award Fee (CPAF). The data also showed that programs using one or fewer associate contractors (besides the Prime) had a higher program success than those that used more than one associate contractor. Similarly, single-Service programs had higher program success compared to joint programs (Reig, 1995, p. 29).

JSF STUDY, 2000

In the JSF study previously mentioned, it also discussed the use of technology readiness levels (TRLs). Pioneered by the National Aeronautics and Space Administration (NASA) and adopted by the Air Force Research Laboratory (AFRL), TLRs are used to determine the readiness of technologies incorporated into a weapon or other type of system (Rodrigues, 2000, p. 9). When measured on a scale of one to nine, the lower the level of maturity when a technology is included in a development program the higher the risk that it will cause problems such as schedule delays in the future (Rodrigues, 2000, p. 8). According to NASA, AFRL, and others in DoD, a level of seven enables a technology to be included in a development program with acceptable risk (Rodrigues, 2000, p. 9).

Prior to the JSF, TRLs were also used to assess the impact of technological maturity on product outcomes. The JSF study reviewed 23 different technologies incorporated into new product and weapon systems designs, either within DoD or the commercial sector, and found the following. First, that cost and schedule problems arise when programs start with technologies at low readiness levels. And second, it conversely showed that programs met product objectives when the technologies are at higher levels of readiness.

GALLEY STUDY, 2002

As explained earlier, the 1995 Reig study examined preliminary data using information from 24 acquisition programs to determine if any relationship between LRIP quantities and the success of the program in EMD existed. This 2002 study expanded the dataset to include 46 programs that had completed EMD (Gailey, 2002, p. 5). Contrary to the results of Reig (1995), which used a smaller database, this study detected no correlation between LRIP quantities and the probability that the schedule will slip (Gailey, 2002, p. 5). Additionally, no differences were noted in EMD success attributable to whether EMD is competed, how many associated contractors are present, or whether the program is joint-Service (Gailey, 2002, p. 9). Lastly, the findings of this study also reiterated that programs that did not compete the DEM/VAL phase had higher program success and narrowed the best contract type to CPIF. That is, CPIF contracts in EMD produced greater success than CPAF, FPI, or FFP contracts (Galley, 2002, p. 9).

INFERENTIAL STUDIES

THE ANALYTIC SCIENCES CORPORATION (TASC) STUDIES, 1986 & 1987

In the 1980s, the Air Force's Aeronautical System Division (ASD) called upon The Analytic Sciences Corporation (TASC) to improve ASD's independent schedule assessment capabilities for the Full Scale Development (now known as EMD) phase. Although various techniques using models based upon Cost Estimating Relationships (CERs) provide a cost estimate from data of analogous systems, the same level of attention was not given to Scheduling Estimating Relationships (SERs) (Nelson, 1986, p. 1-1).

TASC developed a schedule database consisting of 17 aircraft, conducted a literature review, and identified potential drivers. They noted that the popular drivers used in CERs, such as continuous physical parameters, rather than discrete categorical variables, showed little significance in the use of SERs (Nelson, 1986, p. 3-2). The TASC broke the EMD phase into six duration intervals. These intervals were: 1) Milestone II to EMD contract award, 2) EMD contract award to first flight, 3) first flight to first production unit, 4) first production unit to Initial Operational Capability (IOC), 5) EMD contract award to IOC, and (6) Milestone II to IOC (Nelson & Trageser, 1987, pp. 2-6). The TASC also grouped potential schedule drivers into six distinct categories: 1) technical complexity, 2) degree of technological change, 3) system mission, 4) period of procurement, 5) acquisition strategy, and 6) funding profile (Nelson & Trageser, 1987, pp. 2-8). The TASC then conducted a correlation analysis between these durations and category drivers.

For the two schedule durations, Milestone II to IOC and EMD contract award to IOC, the researchers suggested using mission type (cargo, tanker, attack, or fighter), avionics complexity (low, medium, or high), and the period of acquisition for the analysis (Nelson & Trageser, 1987, pp. 4-5). Their findings: A positive correlation exists between mission type and schedule duration where the schedule durations increase from cargo to fighter aircraft (Nelson & Trageser, 1987, pp. 2-17). Additionally, the period of acquisition also showed a positive correlation between the year development started and the schedule duration.

Much like the findings in the 1990 RAND study, schedule duration within EMD overall showed an increase from past to present (Nelson & Trageser, 1987, pp. 2-18; Drezner & Smith, 1990, p. 11). Another finding of this study, which contrasts that of later studies by RAND and IDA, stated that no significant relationships exist between prototyping and EMD schedule durations (Nelson & Trageser, 1987, pp. 3-13; Tyson et al., 1989, pp. VIII6-VIII-7; Harmon, Ward, & Palmer, 1989, p. 47; and Drezner & Smith, 1990, p. 30).

FITCHER, ARNOLD, AND ALLEN STUDY, 1992

This study presented a historical perspective of DoD program schedule performance, based on 35 Army, 46 Navy, and 24 Air Force programs from the December 1991 Selected Acquisition Reports (Fitcher, Arnold, & Allen, 1992, p. 1). The purpose of the study was to provide a point estimate and range for the expected schedule duration of future programs by creating probability distributions of past schedule durations within certain intervals. The intervals were: 1) Milestone I--Milestone II, 2) Milestone II--Milestone III, 3) program start to first flight, 4) program start to first unit equipped, and 5) program start to IOC (Fitcher, Arnold, & Allen, 1992, p. 1).

Although this study in no way tried to predict the schedule duration of a specific interval based on predictor variables, it did provide an ability to check the realism of schedules proposed by the program managers. The probability distributions were compared by service and by intervals, to give a range of values as "most likely" and schedule expectations considered overly optimistic or pessimistic (Fitcher et al., 1992, p. 2). Results from this study showed that no marked difference exists among the data from within each Service. However, the study noted that the only significant difference among the Services (at an alpha level of 0.05) is a longer than average time for Air Force programs compared to the Army and Navy between Milestone II and Milestone III.

HARMON, WARD, AND PALMER, 1989

The IDA attempted to provide methods for assessing the reasonableness of proposed acquisition schedules in the first of four studies. This particular study, consisting of data collected from nine tactical aircraft programs, analyzed schedule intervals and provided a schedule assessment tool that spanned the period from Full Scale Development (FSD) (now referred to as EMD) start through full-rate production (Harmon, Ward, & Palmer, 1989, p. 1). The programs chosen with development occurring from the early 1970s to early 1980s were based on the newness of the program, its importance in historical perspective, and the expected availability of data (Harmon, Ward, & Palmer, 1989, p. 17). The researchers obtained cost and technical data from government sources and prime contractors, while schedule data were acquired from SARs, contractors, and the Services' sources (Harmon, Ward, & Palmer, 1989, pp. 17-18).

The researchers also gathered data on certain program and aircraft characteristics, such as contractor, mission, and prototyping. The study revealed that the program attributes play an important role in explaining variations in interval length. For instance, under the program attribute of the prime contractor, it is estimated that McDonnell Aircraft programs require 15 percent more time than other contractor types considered (Harmon Ward, & Palmer, 1989, p. 47). Additionally, the data also showed that prototype programs require 11 percent less time than programs that do not develop prototypes (Harmon Ward, & Palmer, 1989, p. 47).

HARMON AND WARD, 1989

The second IDA study was a follow-up to the previous IDA study on tactical aircraft acquisition schedules. The approach used in this study in many ways paralleled that used for the previous study. The data consisted of 14 air-launched missile programs (seven air-to-air and seven air-to-surface systems) that involved substantial developments from the mid 1960s to the late 1980s (Harmon & Ward, 1989, p. 7). The data were collected from military services, prime contractors, and third parties (studies and databases at IDA, RAND, and others), with schedule and missiles characteristic data obtained from SARs, numerous government sources, and secondary data sources (Harmon & Ward, 1989, p. 8). The researchers also collected data on program and missile characteristics.

The researchers originally wanted to develop a single equation to predict the interval of FSD, which was defined as the period from FSD start to delivery of the first production missile (Harmon & Ward, 1989, p. 23). However, due to the fact that the determinants of time to first launch and time from first launch to first production are vastly different, the researchers chose the interval between first guided-launch and the first production delivery to model (Harmon & Ward, 1989, p. 36). From their analysis, the researchers believed that the most important determinant of overall development program length is length of the flight test program. Because flight test duration is determined by the number of test missiles launched and the rate at which test launches are accomplished, it is no surprise that the most important program attribute in determining development effort length was the number of missiles launched during flight tests (Harmon & Ward, 1989, p. 13).

HARMON AND OM STUDY, 1993

The third of four studies by IDA, this study's approach was consistent with the previous two studies but differed in what was studied. The data collected consisted of the most important 26 unmanned spacecraft programs spanning the timeframe of the late 1960s to the early 1990s. Included in the dataset were operational DoD spacecraft, four commercial communications spacecraft, and selected NASA and DoD experimental and scientific unmanned spacecraft (Harmon & Om, 1993, p. I-2). Although the original approach to this analysis of development schedule data was to partition the schedule into multiple intervals, spacecraft development program schedule intervals were not characterized consistently enough to yield satisfactory estimating relationships (Harmon & Om, 1993, p. I-2). Instead, the researchers chose to estimate development schedule as a whole. Development schedule duration was defined as the interval between the authority to proceed (ATP) date and delivery of the first flight-model spacecraft (Harmon & Om, 1993, p. I-3).

In order to compare across all programs, the milestone dates were standardized. For DoD spacecraft, ATP generally equated to the start of EMD. For NASA spacecraft, it meant the start of phase design/development C/D (1) contract (Harmon & Om, 1993, p. III-7). The 26 programs used in this study were grouped into one of three categories: DoD sensor/navigation (8 spacecraft), NASA scientific/experimental (9 spacecraft), or operational communications (9 spacecraft) (Harmon & Om, 1993, p. III-2). In analyzing the schedule interval data, the researchers tested relationships for intervals at more disaggregated levels than EMD as a whole, such as time from development start to critical design review (CDR), and time from CDR to first production delivery (Harmon & Om, 1993, p. IV-2).

Unfortunately, the regression models for decomposed timeframes proved less than satisfactory, due to missing data, inconsistency of data between the three program categories, and large unexplained variability for the NASA data points (Harmon & Om, 1993, p. IV-23). On the positive side, the measures of spacecraft size and complexity proved to be good explanatory variables, with the variable, "Beginning of life power in watts," proving most satisfying (Harmon & Om, 1993, p. IV-4). The researchers also stated that all the categorical variables for sensor, navigation, planetary, and commercial spacecraft were statistically significant at an alpha level of 0.03 or better (Harmon & Om, 1993, p. IV-4).

HARMON AND OM STUDY, 1995

This IDA study was the last of four sequenced studies and built on the three previous IDA studies of tactical aircraft, air-launched missiles, and unmanned spacecraft acquisition schedules. The data collected consisted of 22 missile programs with substantial developments from the mid 1960s to the 1990s. These programs included eight surface-launched interceptors, seven air-launched interceptors, and seven air-launched surface-attack missiles (Harmon, & Om, 1995, p. I-2). Although the focus of this study was on interceptor missiles, inclusion of the attack missiles was used because attack missile programs tend to be influenced by the same drivers and the missiles' hardware also share many attributes (Harmon, & Om, 1995, p. II-1).

In the previous study of air-launched missile programs, the researchers used the delivery date of the first production missile to mark the end of development, due to wide variance in production rates associated with different types of missiles. The data for this study encountered the same variability and therefore used this same date to mark the end of development (Harmon & Om, 1995, pp. I-3-I-4). Although emphasis was placed on both the pre-EMD and the EMD phase of the acquisition cycle, schedule intervals in the concept exploration phases and the demonstration and validation phase prior to EMD were often highly dependent upon political factors and therefore not emphasized. Pre-EMD prototype intervals, on the other hand, were an exception (Harmon & Om, 1995, p. II-1).

The researchers originally wanted to develop a single equation to estimate the interval of EMD defined as the period from EMD start to delivery of the first production missile (Harmon & Om, 1995, p. III-1). "Unfortunately, the determinants of time to first launch and time from first launch to first production are just too different" (Harmon & Om, 1995, p. III-24). Therefore, the researchers instead chose the interval between first guided-launch and the first production delivery as the interval to model (Harmon & Om, 1995, p. III-1). Their findings were similar to the 1993 study by IDA on spacecraft--the most important program attribute in determining length of the development effort is the number of missiles launched during flight tests (Harmon & Om, 1995, p. II-9).

TYSON, HARMON, AND UTECH STUDY, 1994

Unrelated to the four previous studies, IDA performed an analysis on 20 tactical missile and 7 tactical aircraft programs with the objective to describe costs and schedule growth patterns associated with the acquisition of selected major systems. Moreover, they wanted to identify reasons for the growth and then develop a way to predict growth in ongoing development and early production phases (Tyson, Harmon, & Utech, 1994, p. iii). Data used for this study were obtained from SARs, historical memoranda to support DoD program reviews, and from summaries of program data (Tyson, Harmon, & Utech, 1994, p. S-1).

The study found that programs took from 50 to 137 months from Milestone II to IOC with only 2 of the 20 tactical missile programs finishing on time, and the highest development schedule growth exceeding its plan by 180 percent (Tyson, Harmon, & Utech, 1994, p. S-2). The researchers also examined the characteristics of programs with the highest and lowest schedule. Those often with the highest schedule commonly had high concurrency of interrelated activities (e.g., overlap of development and production). Additionally, the researchers stated that keys to preventing schedule growth in development are technical realism and a willingness to make tradeoffs (Tyson, Harmon, & Utech, 1994, p. S-2).

Another finding from this research was that the major determinant of development schedule growth is an increase in quantity (i.e., the need to produce more items for testing than planned) (Tyson, Harmon, & Utech, 1994, pp. S-5-S-6). Contrary to the 1990 RAND study, the researchers found a positive correlation between cost growth and schedule growth in both development and production (Tyson, Harmon, & Utech, 1994, p. S-6; Drezner & Smith, 1990, p. 45).

PREDICTIVE SCHEDULE DRIVERS IN LITERATURE

In Table 1, we provide a cross tabulation of predictive schedule drivers by study. This tabulation reveals that the main drivers that prove beneficial in determining schedule variance are technical issues, the use of competition, contract type, and the existence of prototyping. Although the studies highlighted differ in the number of programs, the source of data, and methodologies used, they prove beneficial in providing insight into possible predictors a researcher could use in estimating a program's schedule duration. This in turn could reduce the risk of underestimating a program's completion, thereby minimizing additional costs incurred.

CONCLUSION

In this article, we reviewed a multitude of studies that examined numerous databases and performed a variety of statistical procedures, all in the pursuit of explaining and predicting schedule duration and variance. It is from these studies that we can identify the characteristics that drive acquisition schedules and derive a list of predictor variables. From the past studies, we identify the following potential candidates: program size, mission type, contract type, use of competition, existence of prototyping, and number of test articles. We also identify management characteristics, such as the military service and contractor; schedule characteristics, such as maturity and concurrency measures; and other characteristics reviewed in the literature, such as technical complexity.

In an SOS approach, knowing which programs or systems may be at risk of schedule growth allows a PM to shift assets as needed to ensure, or at least attempt, to keep the overall system on track. In addition, the potential schedule drivers identified in Table 1 are just not limited to thinking in terms of an individual system. It is definitely conceivable that each individual system may not be so technologically complex as to warrant thinking that it may be a potential candidate for schedule slippage due to complexity reasons. However, when taken in the aggregate, the overall system may very well be then prone to slippage. Therefore, researchers could also look to the factors identified in the table with respect to modeling schedule slippage for an overall SOS.

TABLE 1. PREDICTIVE SCHEDULE DRIVERS IN LITERATURE

                                               Author

                                       Tyson,      Harmon,
                            TASC     Nelson, Om,   Ward and    Harmon
                           Studies   and Palmer     Palmer    and Ward

                                                Year
Predictive Schedule
Drivers                     1987        1989         1989       1989

Competition                               X
Concurrency
Contract type                             X
Design-to-cost                            X
External events/guidance
Funding
  (amount/stability)
Increase in quantity
Length of testing                                                X
LRIP/number of test
  articles
Mission type                  X
Multiyear procurement                     X
Number of contractors
Period of acquisition         X
Prime contractor                                      X
Program size
Prototyping                               X           X
Service type
  (joint or single)
Sole source procurement                   X
Technical issues              X
  (maturity/difficulty/
  complexity)

                                             Author

                                       Fitcher,              Tyson,
                            Drezner     Arnold     Harmon    Harmon,
                           and Smith   and Allen   and Om   and Utech

                                              Year

Predictive Schedule
Drivers                      1990        1992       1993      1994

Competition                    X
Concurrency                    X                                X
Contract type
Design-to-cost
External events/guidance       X
Funding
  (amount/stability)           X
Increase in quantity                                            X
Length of testing
LRIP/number of test
  articles
Mission type                                         X
Multiyear procurement
Number of contractors
Period of acquisition
Prime contractor
Program size                                         X
Prototyping                    X
Service type
  (joint or single)                        X
Sole source procurement
Technical issues               X                     X          X
  (maturity/difficulty/
  complexity)

                                        Author

                                                            Occurrence
                                                                of
                                                            Predictive
                                                             Schedule
                           Harmon          JSF              Drivers in
                           and Om   Reig   Study   Gailey   Literature

                                        Year

Predictive Schedule
Drivers                     1995    1995   2000     2002

Competition                          X               X           4
Concurrency                                                      2
Contract type                        X               X           3
Design-to-cost                                                   1
External events/guidance                                         1
Funding
  (amount/stability)                                             1
Increase in quantity                                             1
Length of testing            X                                   2
LRIP/number of test
  articles                           X                           1
Mission type                                                     2
Multiyear procurement                                            1
Number of contractors                X                           1
Period of acquisition                                            1
Prime contractor                                                 1
Program size                                                     1
Prototyping                                                      3
Service type
  (joint or single)                  X                           2
Sole source procurement                                          1
Technical issues                             X                   5
  (maturity/difficulty/
  complexity)

ENDNOTES

(1.) There are five phases in the life cycle of a NASA major system acquisition:

a. Phase A (Preliminary Anaysis).

b. Phase B (Definition).

c. Phase C (Design).

d. Phase D (Development).

e. Phase E (Operations).

REFERENCES

Drezner, J. A., & Smith, G. K. (1990). An analysis of weapon system acquisition schedules (R-3937-ACQ). Santa Monica, CA: RAND.

Fitchter, L. S., Arnold, D. K., & Allen, M. K. (1992). DoD weapon system schedule risk profiles based on an analysis of schedule data presented in the December 1991 selected acquisition reports (SARs) (TO-92-007). Santa Barbara, CA: Tecolote Research, Inc.

Gailey, C. K. (2002). Predictive power for program success from engineering and manufacturing development performance trends (TR 1-02). Alexandria, VA: Defense Acquisition University.

Harmon, B. R., & Om, N. I. (1993, March). Assessing acquisition schedules for unmanned spacecraft (P-2766). Alexandria, VA: Institute for Defense Analyses.

Harmon, B. R., & Om, N. I. (1995, August). Schedule assessment methods for surface-launched interceptors. (P-3014). Alexandria, VA: Institute for Defense Analyses.

Harmon, B. R., & Ward, L. M. (1989). Methods for assigning acquisition schedule for air-launched missiles (P-2274). Alexandria, VA: Institute for Defense Analyses.

Harmon, B. R., Ward, L. M., & Palmer, P. R. (1989, February). Assessing acquisition schedules for tactical aircraft (P-2105). Alexandria, VA: Institute for Defense Analyses.

Nelson, E. K. (1986). Independent schedule assessment FSD study (TR-5300-2-2). Fairborn, OH: The Analytic Sciences Corporation.

Nelson, E. K., & Trageser, J. E. (1987). Analogy selection methodology study (TR-5306-7-2). Fairborn, OH: The Analytic Sciences Corporation.

Reig, R. W. (1995, January-February). A decade of success and failures in the DoD acquisition system. Program Manager, 24(1), pp. 27-29.

Rodrigues, L. J. (2000). Joint strike fighter acquisition: Development schedule should be changed to reduce risks. (GAO/T-NSIAD-00-132). Washington, DC: Author.

Tyson, K. W., Harmon, B. R., & Utech, D. M. (1994). Understanding cost and schedule growth in acquisition programs (P-2967). Alexandria, VA: Institute for Defense Analyses.

Tyson, K. W., Nelson, J. R., Om, N. I., & Palmer, P. R. (1989, March). Acquiring major systems: Cost and schedule trends and acquisition initiative effectiveness (P-2201). Alexandria, VA: Institute for Defense Analyses.

AUTHOR BIOGRAPHY

Captain James V. Monaco, USAF, is currently assigned to the F/A-22 System Program Office (SPO) at Wright-Patterson AFB, Ohio, in their cost analysis branch. He is a distinguished graduate from the Air Force Institute of Technology and received in 2005 an M.S. in cost analysis. He has served as both the Financial Services Officer end Financial Analysis Officer at Francis F. Warren AFB, WY.

(E-mail address: [e-mail])

Dr. Edward "Tony" White is an associate professor of statistics at the Air Force Institute of Technology. His research interests include design of experiments, biostatistics, growth curves, linear and nonlinear regression, categorical data analysis, log-linear models, statistical simulation, and response surface modeling. White's current projects involve developing models to assess the risk or likelihood of a program experiencing cost and schedule growth. White received his Ph.D. in statistics from Texas A&M University in 1998.

(E-mail address: [e-mail])