Printer Friendly

What's in your estimate at completion?

In this era of increased emphasis on cost constraints and affordability, we often ask, "How much will this cost?" Within an Earned Value Management (EVM) context, the answer is found in the Estimate at Completion, or EAC. EACs are discussed in various venues from a technical manager's office at a prime contractor facility to a crowded conference room at the Pentagon. All eyes become fixated on "the number."

It is important for government program managers (PMs) to realize, especially as budgets tighten, that the EAC is more than just a number. The EAC is a gateway to insight on past, present and future program performance. The EAC numbers are the tip of an iceberg. Below the surface is a rich story describing why the numbers are what they are, how they were derived, and what they reflect. EACs provide insight and contributions to both government and contractor planning and management process execution.

Used as a leading indicator, an EAC provides a PM an opportunity to make proactive decisions. It is a pathway to explore not only whether a provider is likely to deliver on its promise, but whether the government PM can deliver within the cost, schedule, performance (C/S/P) constraints of the program's Acquisition Program Baseline (APB). EAC discussions in the government program management office (PMO) are also excellent chances for the government PM to listen to what his or her staff thinks.

EAC Insights

EACs are tied directly to the contract portion of the program threshold and objective costs in an APB. In addition, EACs provide a current estimate against the contract portion of the schedule and performance parameters in the APB. EACs therefore become a key consideration in exploring APB trade space with respect to C/S/P, to retain program affordability. Thus, EACs tell a story of real program integration.

EACs help clarify the influence of contractor overhead costs, general and administrative costs, and cost of money. Add in profit/fee and you get price, which takes you directly into budget territory: totals, phasing, obligations and expenditures. Thus an EAC discussion should be married with evaluation of a contractor's Contract Funds Status Report (CFSR) to assess whether the current time-phased budget and execution plan will support the latest EAC.

Given that usually two-thirds of a program's total ownership costs (TOC) reside "downstream" in the program life-cycle process, "upstream" EACs become important leading indicators of ultimate costs. Significant EAC changes demand nearimmediate revisits to TOC estimates.

EACs also should reflect the reality of performance trends and forecasting. It is of paramount importance to compare EAC results with indices such as current and projected cost variance, schedule variance, Cost Performance Index (CPI), Schedule Performance Index (SPI) and Variance at Completion (VAC). Every EAC drives a particular To-Complete Performance Index (TCPI), which is an important additional "reality check."

The derivation and reporting of a contractor EAC typically is associated with a detailed "bottoms up" analysis of work to go, which takes significant planning and assessment effort on the part of contractor control account managers (CAMs). Some organizations update their "bottoms up" EACs annually, others semiannually, some even more frequently. EACs also are expected to be assessed frequently (monthly) by each CAM via less-detailed means (in terms of planning) but with thorough consideration of performance to date using a variety of metrics and indicators.

The credibility of EACs can also be reflections of incentives, culture and/or trust. There have been cases of EACs "adjusted" based in part on a desired story or outcome rather than entirely on performance to date, work to go and associated risk. For further exploration, see Table 1.

EACs Should Capture Risk

Most importantly, EACs give us a healthy look at risk assumptions.

Contractor-derived EACs are part of formal reporting artifacts such as the Integrated Program Management Report, or IPMR (formerly called the Contract Performance Report or CPR). Government PMO-derived EACs are intended to be compared to the contractor EACs. There ought to be some reasonable degree of traceability in each, from the initial assumptions through the derivation and into the final result.

Of all potential factors, the most significant differences between contractor and government EACs lie in risk and opportunity assumptions, which mean there is a range of potential EACs, typically reported as Most Likely (ML), Best Case (BC) and Worst Case (WC). The ML is the most commonly reported EAC, and is exactly what it sounds like: the "best guess" final result after considering all dynamics, risks and opportunities. A WC is the result anticipated if most risks become issues and few opportunities, if any, are captured. By contrast, a BC reflects most risks not materializing and most opportunities being capitalized upon (i.e., things mostly going right the first time). The range, or spread, among these EACs reflects directly on the uncertainty associated with the program, starting with the first EAC on Day One of program execution. From a cost estimator's perspective, the range between BC and WC is typically the broadest toward the beginning of a program and typically the narrowest toward the end. The PM therefore should consider the cost estimator's perspective relative to EACs and their program budget. In addition, the PM can use the EAC range to be continually apprised of risk and opportunity and how much of each is assumed in the most likely EAC.

A "90-Second Back-of-the-Envelope" PM EAC Cross-Check

Many EVM training seminars and courses within and outside the Department of Defense (DoD) encourage students to focus on standard formulas when it comes to calculating EACs. The Defense Acquisition University EVM "Gold Card" is one of the most well-known references for widely accepted EAC formulas. One chooses a formula based on local conditions of cost performance, schedule performance and/ or degree of program completion. As a general rule, these formulas have two components--actual costs to date and anticipated costs going forward. These anticipated costs are derived through dividing the budget of work remaining by an efficiency factor reflecting schedule and/or cost performance to date. While these formulas are effective and highly capable of forecasting, they are not necessarily geared to foster a direct conversation on risk.

Therefore, this article offers a "90-second back-of-the-envelope" formula for government PMs. This formula is a simple cross-check that enables the government PM to discuss key entering arguments of a proposed EAC. This "back-of-the-envelope" formula and explanation of its terms is below. A specific example is shown in Table 2.

A Program Manager's 90-Second
"Back-of-the-Envelope" EAC

Costs    Budget   Remaining    Dollarized   Marching
       +        +            +            +
So Far   to Go    Management    Risks         Army
                   Reserve                    Costs


90-Second Formula Components Defined

Each of the 90-second formula components is defined as follows:

Costs So Far: An EAC should include program costs to date. These otherwise are known as "actual costs," "actuals" or ACWP (Actual Costs of Work Performed). They represent "sunk costs." The PM starts here by recording ACWP as reported in the CPR/IPMR.

Budget to Go: Second, an EAC should consider the budget for the remaining work. This is calculated by taking Budget at Completion (BAC) and the cumulative value for Budgeted Cost of Work Performed (BCWP) from the latest CPR/IPMR (i.e., BAC minus BCWP cumulative). This value is added to the ACWP.

Remaining Management Reserve (MR): DoD acquisition history for medium- to high-risk programs tells us odds are a contractor has planned insufficient MR and will consume whatever it does have. Thus, we add all the remaining MR. The MR value can also be found in the CPR/IPMR.

Dollarized Risks: When it comes to quantified cost risk, contractors put a dollar value against each risk. These are dollarized risks. Furthermore, some contractors assess a probability value against each risk. They then develop a factored risk by multiplying the dollarized risk value by the probability. The resulting number is added into the EAC. By contrast, the back-of-the-envelope approach adds the full dollarized (not factored) risk value. The assumption here is that, though many risks may not be realized, other risks will continue to emerge until the end of the contract. Total dollarized risks can be found on the risk list or risk register. They also may be found in Format 5 of the CPR/IPMR. If there is not a dollarized risk list, try to get a rough estimate of the total value.

Marching Army Costs: When the contract end date schedule slips, everything slips, but people typically still get paid and overhead dollars still accrue. Multiplying anticipated contract end date schedule slip (in months) by an estimate of the contractor's "burn rate" when program slip is anticipated will do the trick for this term. A quick way to select the burnrate value is to use the ACWP of the current period from the CPR/IPMR.

Discussing the 90-Second Formula Components

These back-of-the-envelope formula components are not the final word for an EAC. Rather, each sets the stage for a robust and systematic EAC discussion.

Costs So Far: The reported actual costs might not truly reflect sunk costs. In some cases, portions of the reported actual costs are, in fact, estimated costs. This presents an opportunity for the PM to clarify the impact, if any, of these "estimated" actuals. For example, prime contractors typically utilize a large array of subcontractors, and subcontractors employ subcontractors. Accounting periods and invoicing across these "layers" often are not synchronized. The final "actuals" number likely could differ from the estimated number.

Budget to Go: Work remaining directly reflects work accomplished to date. A PM uses this term to discuss the contractor's measurement of "percent complete." How reasonable and/ or robust is that measurement? What is really meant by 35 percent complete, 40 percent complete? An integrated master plan (IMP) becomes an excellent supporting tool for this part of the discussion.

Remaining MR: This term kicks off the program risk discussion. How did the contractor derive the MR? Was it pulled "across the board" using some factor like 8 percent or 10 percent (i.e., a management challenge) of negotiated cost? Does the factor differ between material and labor? How does this MR reflect the risk register content and nature of the work? How has the contractor been burning MR to date and for what purpose? How does the contractor factor into risk handling and forecasting? Has "disbursed" or "allocated" MR been reflected directly as work in the integrated master schedule (IMS)?

Dollarized Risks: This is an excellent opportunity to ask the hard questions about risk (and opportunity) assumptions and impacts. In particular, it sheds light on how risks actually are quantified in different organizations. The end result should be a mutual understanding of risk perspectives, even if the parties differ in their bottom line.

Marching Army Costs: Time is money, so this is an excellent opportunity for the PM to inquire how the contractor schedule risk analysis (SRA) was conducted, and to what degree its results influenced the EAC. An SRA accomplished in a manner consistent with industry and DoD expectations is an exceptionally powerful tool for PM insight and proactive decision-making.

Points of Order

A concern with this approach might be, "Wait a minute: MR plus dollarized risk plus schedule burn (marching army)? You are double and even triple-counting risk!" Where highly disciplined approaches to program management are encouraged and practiced, a "double" or "triple"

count concern might be warranted. This formula assumes that this exceptional degree of discipline is not the case in most organizations.

It is often observed that not all risk considerations are created equal. MR, for example, often appears as a top-down "challenge" to managers, an across-the-board percentage cut at the beginning of the program. Derivation of MR levels and risk management assessments often are performed in separate venues and times. As a program's risk management process spins up, risk impacts and mitigation costs are developed in various ways and often are loosely connected to MR levels.

In practice, the ubiquitous "risk cube" tends to be used as a basis for quantitative risk calculations. However, that particular risk management tool is qualitative, not quantitative. One person's "high probability" is another person's "low probability," and it is hard to distinguish between how 35 percent versus 45 percent versus 55 percent was derived.

The same goes for assessment of impact. It is nearly impossible to predict program-level risk impacts without the aid of program-level models such as a program cost model and an IMS. In short, true quantitative risk analysis results, factored into no-kidding quantitative numbers such as EACs, need more than a risk cube. A back-of-the-envelope tool merely opens the door for that discussion.

Best practice to the contrary, risk decision points, handling approaches and forecasting often are not directly reflected in the IMS in a way that models their respective influence on the program. Not all programs perform SRAs, and those that do so will not always integrate the results into the risk register, let alone the EAC. This 90-second approach delineates schedule risk and other risk considerations separately not because that is how it ought to be, but instead as a reminder to PMs to bring IMS considerations to the forefront of EAC discussions.

Opportunities could, and should be, assessed in a detailed EAC review. However, we assume strictly for our "back of the envelope" purposes that people are optimistic by nature and therefore we focus on risk and do not factor in opportunities, per se. Therefore, this approach fosters a discussion on opportunity (within which resides the topic of "should cost").

The "Blind Side"

Acute awareness of the EAC and its derivation is important beyond the government and contractor PM conversation, because they aren't the only ones who see, and interpret, an EAC. While we know EACs typically are reported within a contract through a deliverable such as an IPMR, EACs also find their way elsewhere. The contractor PM has his or her own command chain, all the way up to the chief executive officer, that will have an interest in--and influence upon--the EAC. On the government side, the EAC finds its way into program dashboards and briefings "up the chain" to program executive offices and acquisition executives in the Service or agency and in the Office of the Secretary of Defense (OSD). EACs are contained within Defense Acquisition Executive Summary (DAES) quarterly reports and DAES briefing charts. Selected Acquisition Reports contain EACs. EAC information gets pulled into other databases such as the Central Repository at the OSD level. The Defense Contract Management Agency (DCMA) assesses and reports on EACs. In short, EAC discussions take place in venues beyond the government PM's zone of control, and lack of awareness can result in a PM being "blindsided" and unprepared at a most inopportune time.

The PM is expected to know the EAC and what it represents. By asking targeted and insightful questions about contractor EACs, he or she can help make EAC generation and review more efficient and effective. A "back of the envelope" approach enables tremendous insight into how the contractor accommodates risk within its performance measurement and forecasting functions and is one way for a PM to assess whether risk has been reasonably factored into a contractor's EAC. Smart questions can "cut to the chase" quickly and discover elements or indicators that reflect process issues, clarify perspectives, prevent late-to-need information and foster proactive decision-making. A 90-second EAC is not the last word on a program EAC. It is but a cross-reference for a PM, a starting point for discussion, and a catalyst for more detialed government PMO staff analysis. Powerful EAC questions and robust EAC discussions put "affordability" in the forefront in a very real and productive way.

The authors can be reached at patrick.barkeradau.mil and roberta. tomasiniadau.mil

Table l. Warning Signs for EVM (EAC) Reporting Issues

SOMETHING YOU MIGHT SEE THAT         WHAT IT MIGHT MEAN (AND IS WORTH A
SHOULD "RAISE A FLAG"                QUESTION OR TWO FOR CLARIFICATION)

A PM deferring to the "cost lead"    Lack of PM comfort with EAC
or "EVM lead" to explain an EAC      derivation, lack of PM familiarity
                                     with EVM.

EACs stagnant, not being updated     EAC development/assessment process
at regular intervals                 that is either immature or
                                     atrophied, Lack of PM attention to
                                     EAC.

A volatile and "upwardly             Weak (and disconnected) program
accelerating" EAC (changes           planning, scheduling and EVM
significantly each month)            activities. Lack of anticipation
                                     and proactive decision-making.

Best Case (BC), Worst Case (WC)      Significant EVM process execution
and Most Likely (ML) EAC are equal   problems. Complete lack of
                                     risk/opportunity integration.
                                     Lack of understanding of what EAC
                                     represents.

ML EAC < BC EAC                      EAC development/assessment process
                                     that is either immature or
                                     atrophied. Lack of PM attention to
                                     EAC.

ML EAC > WC EAC                      EAC development/assessment process
                                     that is either immature or in
                                     atrophy. Lack of PM attention to
                                     EAC.

Exact or near-exact agreement        Government staff lacks comfort,
between contractor EAC and           experience with EVM. Likely lack
government EAC                       of integration/interface between
                                     government EVM analysts and rest
                                     of govern ment staff, especially
                                     technical staff.

Government EAC(s) consist solely     Government staff lacks familiarity
of "Gold Card" formula results       with how to integrate risk and
with little or no accompanying       EVM. Likely lack of
explanation/rationale                integration/interface between
                                     government EVM analysts and rest
                                     of PMO staff, especially technical
                                     staff.

Lack of robust explanation of EAC    Lack of comfort, experience with
cost drivers                         EVM, and especially risk and EVM.
                                     Likely disconnect between
                                     Integrated Master Schedule,
                                     Schedule Risk Analysis (SRA) and
                                     EVM.

Exact or near-exact match of EAC     EAC development/assessment process
to prominent cost-related contract   that is either immature or
elements such as BAC, TAB,           atrophied, PM lack of comfort with
Negotiated Price, PMO budget         EAC derivation and/or lack of
                                     familiarity with EVM. Could
                                     reflect organizational culture of
                                     risk aversion. Could also reflect
                                     bud get constraints.

Sustained high SPI                   Premature claim of work
                                     accomplished. High percentage of
                                     "level of effort" earned value
                                     technique.

Sustained high CPI                   High percentage of "level of
                                     effort" earned value technique
                                     combined with understaffing and/or
                                     substitution of lower labor rate
                                     personnel for higher rate
                                     personnel. Premature claim of work
                                     accomplished. Premature opening of
                                     work packages.

Contractor expenditures different    Contract Funds Status Report
than anticipated                     (CFSR) misalignment with reported
                                     Actual Cost of Work Performed from
                                     CPR/IPMR. Must reconcile CFSR
                                     (with fee) to CPR/IPMR (without
                                     fee).

No reference in EAC to schedule      Disconnect between scheduling
forecast derived from SRA            discipline, EVM and risk
                                     management. Lack of cost
                                     adjustment to pay for
                                     "standing/marching army."

To-Complete Performance Index        EAC might be unrealistic. A TCPI
(TCPI) associated with the EAC is    exceeding the CPI by 5% warrants
greater than the cumulative Cost     pointed questions, and a 10% or
Performance Index (CPI) reported     greater difference warrants
to date                              concern. It is rare for
                                     performance efficiency to increase
                                     by such a large margin unless
                                     fundamental changes to program
                                     management and execution are
                                     assumed.

Table 2. Back-of-the-Envelope Example

Assumptions: Let's say a contractor has developed an EAC and reports
its composition as noted in the table below. Taking out a small sheet
of paper, the government PM can apply the 90-second
back-of-the-envelope approach as shown below.

                          WHAT A         THE GOVERNMENT
                          CONTRACTOR     PM'S "BACK-OF-THE-
ROW   TERM                MIGHT REPORT   ENVELOPE" NOTES

1     Actual Costs to     $50 million    million
      Date

2     Budget of work to   $50 million    $50 million
      go (BAC-BCWP)

3     MR                  --             $3 million

4     Risk/Opportunity    $5 million     $8 million (dollarized
                          (factored)     risks only)

5     Marching Army       --             $6 million

6     Total               $105 million   $117 million

      NOTES ON GOVERNMENT PM'S ACTIONS
ROW

1     Extracted directly from Format 1 CPR/
      IPMR

2     Extracted directly from Format 1 CPR/
      IPMR

3     Extracted directly from Format 1 CPR/
      IPMR

4     Extracted directly from contractor Risk
      Register and/or Format 5 CPR/IPMR

5     Extracted from contractor SRA results
      for contract end date and current "burn
      rate" from CPR/IPMR. This is based on a
      6-month contract end date slip at a burn
      rate of $1 million per month

6

As a result, the PM's back-of-the-envelope EAC is $12 million higher
than the contractor EAC. This becomes a starting point for robust EAC
conversation and an important catalyst for more detailed PMO staff
analysis. The true goal is to reach a mutual understanding of the
influence of risk in EAC derivation, not to determine that one EAC is
more accurate than the other. The government PM's staff can, and
should, perform a more thorough analysis using its own methodologies
while keeping the PM dis cussion questions at the forefront. Note
that it may be necessary to replace contractor risk information (such
as dollarized risk and SRA) with government information should the
contractor information be determined to be inadequate.
COPYRIGHT 2014 Defense Acquisition University Press
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2014 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Barker, Pat; Tomasini, Roberta
Publication:Defense AT & L
Date:Mar 1, 2014
Words:3435
Previous Article:Just in time: expecting failure: do JIT principles run counter to DoD's business nature?
Next Article:Bombs and books: building a better air force military acquirer.
Topics:

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters