What's in your estimate at completion?
It is important for government program managers (PMs) to realize, especially as budgets tighten, that the EAC is more than just a number. The EAC is a gateway to insight on past, present and future program performance. The EAC numbers are the tip of an iceberg. Below the surface is a rich story describing why the numbers are what they are, how they were derived, and what they reflect. EACs provide insight and contributions to both government and contractor planning and management process execution.
Used as a leading indicator, an EAC provides a PM an opportunity to make proactive decisions. It is a pathway to explore not only whether a provider is likely to deliver on its promise, but whether the government PM can deliver within the cost, schedule, performance (C/S/P) constraints of the program's Acquisition Program Baseline (APB). EAC discussions in the government program management office (PMO) are also excellent chances for the government PM to listen to what his or her staff thinks.
EACs are tied directly to the contract portion of the program threshold and objective costs in an APB. In addition, EACs provide a current estimate against the contract portion of the schedule and performance parameters in the APB. EACs therefore become a key consideration in exploring APB trade space with respect to C/S/P, to retain program affordability. Thus, EACs tell a story of real program integration.
EACs help clarify the influence of contractor overhead costs, general and administrative costs, and cost of money. Add in profit/fee and you get price, which takes you directly into budget territory: totals, phasing, obligations and expenditures. Thus an EAC discussion should be married with evaluation of a contractor's Contract Funds Status Report (CFSR) to assess whether the current time-phased budget and execution plan will support the latest EAC.
Given that usually two-thirds of a program's total ownership costs (TOC) reside "downstream" in the program life-cycle process, "upstream" EACs become important leading indicators of ultimate costs. Significant EAC changes demand nearimmediate revisits to TOC estimates.
EACs also should reflect the reality of performance trends and forecasting. It is of paramount importance to compare EAC results with indices such as current and projected cost variance, schedule variance, Cost Performance Index (CPI), Schedule Performance Index (SPI) and Variance at Completion (VAC). Every EAC drives a particular To-Complete Performance Index (TCPI), which is an important additional "reality check."
The derivation and reporting of a contractor EAC typically is associated with a detailed "bottoms up" analysis of work to go, which takes significant planning and assessment effort on the part of contractor control account managers (CAMs). Some organizations update their "bottoms up" EACs annually, others semiannually, some even more frequently. EACs also are expected to be assessed frequently (monthly) by each CAM via less-detailed means (in terms of planning) but with thorough consideration of performance to date using a variety of metrics and indicators.
The credibility of EACs can also be reflections of incentives, culture and/or trust. There have been cases of EACs "adjusted" based in part on a desired story or outcome rather than entirely on performance to date, work to go and associated risk. For further exploration, see Table 1.
EACs Should Capture Risk
Most importantly, EACs give us a healthy look at risk assumptions.
Contractor-derived EACs are part of formal reporting artifacts such as the Integrated Program Management Report, or IPMR (formerly called the Contract Performance Report or CPR). Government PMO-derived EACs are intended to be compared to the contractor EACs. There ought to be some reasonable degree of traceability in each, from the initial assumptions through the derivation and into the final result.
Of all potential factors, the most significant differences between contractor and government EACs lie in risk and opportunity assumptions, which mean there is a range of potential EACs, typically reported as Most Likely (ML), Best Case (BC) and Worst Case (WC). The ML is the most commonly reported EAC, and is exactly what it sounds like: the "best guess" final result after considering all dynamics, risks and opportunities. A WC is the result anticipated if most risks become issues and few opportunities, if any, are captured. By contrast, a BC reflects most risks not materializing and most opportunities being capitalized upon (i.e., things mostly going right the first time). The range, or spread, among these EACs reflects directly on the uncertainty associated with the program, starting with the first EAC on Day One of program execution. From a cost estimator's perspective, the range between BC and WC is typically the broadest toward the beginning of a program and typically the narrowest toward the end. The PM therefore should consider the cost estimator's perspective relative to EACs and their program budget. In addition, the PM can use the EAC range to be continually apprised of risk and opportunity and how much of each is assumed in the most likely EAC.
A "90-Second Back-of-the-Envelope" PM EAC Cross-Check
Many EVM training seminars and courses within and outside the Department of Defense (DoD) encourage students to focus on standard formulas when it comes to calculating EACs. The Defense Acquisition University EVM "Gold Card" is one of the most well-known references for widely accepted EAC formulas. One chooses a formula based on local conditions of cost performance, schedule performance and/ or degree of program completion. As a general rule, these formulas have two components--actual costs to date and anticipated costs going forward. These anticipated costs are derived through dividing the budget of work remaining by an efficiency factor reflecting schedule and/or cost performance to date. While these formulas are effective and highly capable of forecasting, they are not necessarily geared to foster a direct conversation on risk.
Therefore, this article offers a "90-second back-of-the-envelope" formula for government PMs. This formula is a simple cross-check that enables the government PM to discuss key entering arguments of a proposed EAC. This "back-of-the-envelope" formula and explanation of its terms is below. A specific example is shown in Table 2.
A Program Manager's 90-Second "Back-of-the-Envelope" EAC Costs Budget Remaining Dollarized Marching + + + + So Far to Go Management Risks Army Reserve Costs
90-Second Formula Components Defined
Each of the 90-second formula components is defined as follows:
Costs So Far: An EAC should include program costs to date. These otherwise are known as "actual costs," "actuals" or ACWP (Actual Costs of Work Performed). They represent "sunk costs." The PM starts here by recording ACWP as reported in the CPR/IPMR.
Budget to Go: Second, an EAC should consider the budget for the remaining work. This is calculated by taking Budget at Completion (BAC) and the cumulative value for Budgeted Cost of Work Performed (BCWP) from the latest CPR/IPMR (i.e., BAC minus BCWP cumulative). This value is added to the ACWP.
Remaining Management Reserve (MR): DoD acquisition history for medium- to high-risk programs tells us odds are a contractor has planned insufficient MR and will consume whatever it does have. Thus, we add all the remaining MR. The MR value can also be found in the CPR/IPMR.
Dollarized Risks: When it comes to quantified cost risk, contractors put a dollar value against each risk. These are dollarized risks. Furthermore, some contractors assess a probability value against each risk. They then develop a factored risk by multiplying the dollarized risk value by the probability. The resulting number is added into the EAC. By contrast, the back-of-the-envelope approach adds the full dollarized (not factored) risk value. The assumption here is that, though many risks may not be realized, other risks will continue to emerge until the end of the contract. Total dollarized risks can be found on the risk list or risk register. They also may be found in Format 5 of the CPR/IPMR. If there is not a dollarized risk list, try to get a rough estimate of the total value.
Marching Army Costs: When the contract end date schedule slips, everything slips, but people typically still get paid and overhead dollars still accrue. Multiplying anticipated contract end date schedule slip (in months) by an estimate of the contractor's "burn rate" when program slip is anticipated will do the trick for this term. A quick way to select the burnrate value is to use the ACWP of the current period from the CPR/IPMR.
Discussing the 90-Second Formula Components
These back-of-the-envelope formula components are not the final word for an EAC. Rather, each sets the stage for a robust and systematic EAC discussion.
Costs So Far: The reported actual costs might not truly reflect sunk costs. In some cases, portions of the reported actual costs are, in fact, estimated costs. This presents an opportunity for the PM to clarify the impact, if any, of these "estimated" actuals. For example, prime contractors typically utilize a large array of subcontractors, and subcontractors employ subcontractors. Accounting periods and invoicing across these "layers" often are not synchronized. The final "actuals" number likely could differ from the estimated number.
Budget to Go: Work remaining directly reflects work accomplished to date. A PM uses this term to discuss the contractor's measurement of "percent complete." How reasonable and/ or robust is that measurement? What is really meant by 35 percent complete, 40 percent complete? An integrated master plan (IMP) becomes an excellent supporting tool for this part of the discussion.
Remaining MR: This term kicks off the program risk discussion. How did the contractor derive the MR? Was it pulled "across the board" using some factor like 8 percent or 10 percent (i.e., a management challenge) of negotiated cost? Does the factor differ between material and labor? How does this MR reflect the risk register content and nature of the work? How has the contractor been burning MR to date and for what purpose? How does the contractor factor into risk handling and forecasting? Has "disbursed" or "allocated" MR been reflected directly as work in the integrated master schedule (IMS)?
Dollarized Risks: This is an excellent opportunity to ask the hard questions about risk (and opportunity) assumptions and impacts. In particular, it sheds light on how risks actually are quantified in different organizations. The end result should be a mutual understanding of risk perspectives, even if the parties differ in their bottom line.
Marching Army Costs: Time is money, so this is an excellent opportunity for the PM to inquire how the contractor schedule risk analysis (SRA) was conducted, and to what degree its results influenced the EAC. An SRA accomplished in a manner consistent with industry and DoD expectations is an exceptionally powerful tool for PM insight and proactive decision-making.
Points of Order
A concern with this approach might be, "Wait a minute: MR plus dollarized risk plus schedule burn (marching army)? You are double and even triple-counting risk!" Where highly disciplined approaches to program management are encouraged and practiced, a "double" or "triple"
count concern might be warranted. This formula assumes that this exceptional degree of discipline is not the case in most organizations.
It is often observed that not all risk considerations are created equal. MR, for example, often appears as a top-down "challenge" to managers, an across-the-board percentage cut at the beginning of the program. Derivation of MR levels and risk management assessments often are performed in separate venues and times. As a program's risk management process spins up, risk impacts and mitigation costs are developed in various ways and often are loosely connected to MR levels.
In practice, the ubiquitous "risk cube" tends to be used as a basis for quantitative risk calculations. However, that particular risk management tool is qualitative, not quantitative. One person's "high probability" is another person's "low probability," and it is hard to distinguish between how 35 percent versus 45 percent versus 55 percent was derived.
The same goes for assessment of impact. It is nearly impossible to predict program-level risk impacts without the aid of program-level models such as a program cost model and an IMS. In short, true quantitative risk analysis results, factored into no-kidding quantitative numbers such as EACs, need more than a risk cube. A back-of-the-envelope tool merely opens the door for that discussion.
Best practice to the contrary, risk decision points, handling approaches and forecasting often are not directly reflected in the IMS in a way that models their respective influence on the program. Not all programs perform SRAs, and those that do so will not always integrate the results into the risk register, let alone the EAC. This 90-second approach delineates schedule risk and other risk considerations separately not because that is how it ought to be, but instead as a reminder to PMs to bring IMS considerations to the forefront of EAC discussions.
Opportunities could, and should be, assessed in a detailed EAC review. However, we assume strictly for our "back of the envelope" purposes that people are optimistic by nature and therefore we focus on risk and do not factor in opportunities, per se. Therefore, this approach fosters a discussion on opportunity (within which resides the topic of "should cost").
The "Blind Side"
Acute awareness of the EAC and its derivation is important beyond the government and contractor PM conversation, because they aren't the only ones who see, and interpret, an EAC. While we know EACs typically are reported within a contract through a deliverable such as an IPMR, EACs also find their way elsewhere. The contractor PM has his or her own command chain, all the way up to the chief executive officer, that will have an interest in--and influence upon--the EAC. On the government side, the EAC finds its way into program dashboards and briefings "up the chain" to program executive offices and acquisition executives in the Service or agency and in the Office of the Secretary of Defense (OSD). EACs are contained within Defense Acquisition Executive Summary (DAES) quarterly reports and DAES briefing charts. Selected Acquisition Reports contain EACs. EAC information gets pulled into other databases such as the Central Repository at the OSD level. The Defense Contract Management Agency (DCMA) assesses and reports on EACs. In short, EAC discussions take place in venues beyond the government PM's zone of control, and lack of awareness can result in a PM being "blindsided" and unprepared at a most inopportune time.
The PM is expected to know the EAC and what it represents. By asking targeted and insightful questions about contractor EACs, he or she can help make EAC generation and review more efficient and effective. A "back of the envelope" approach enables tremendous insight into how the contractor accommodates risk within its performance measurement and forecasting functions and is one way for a PM to assess whether risk has been reasonably factored into a contractor's EAC. Smart questions can "cut to the chase" quickly and discover elements or indicators that reflect process issues, clarify perspectives, prevent late-to-need information and foster proactive decision-making. A 90-second EAC is not the last word on a program EAC. It is but a cross-reference for a PM, a starting point for discussion, and a catalyst for more detialed government PMO staff analysis. Powerful EAC questions and robust EAC discussions put "affordability" in the forefront in a very real and productive way.
The authors can be reached at patrick.barkeradau.mil and roberta. tomasiniadau.mil
Table l. Warning Signs for EVM (EAC) Reporting Issues SOMETHING YOU MIGHT SEE THAT WHAT IT MIGHT MEAN (AND IS WORTH A SHOULD "RAISE A FLAG" QUESTION OR TWO FOR CLARIFICATION) A PM deferring to the "cost lead" Lack of PM comfort with EAC or "EVM lead" to explain an EAC derivation, lack of PM familiarity with EVM. EACs stagnant, not being updated EAC development/assessment process at regular intervals that is either immature or atrophied, Lack of PM attention to EAC. A volatile and "upwardly Weak (and disconnected) program accelerating" EAC (changes planning, scheduling and EVM significantly each month) activities. Lack of anticipation and proactive decision-making. Best Case (BC), Worst Case (WC) Significant EVM process execution and Most Likely (ML) EAC are equal problems. Complete lack of risk/opportunity integration. Lack of understanding of what EAC represents. ML EAC < BC EAC EAC development/assessment process that is either immature or atrophied. Lack of PM attention to EAC. ML EAC > WC EAC EAC development/assessment process that is either immature or in atrophy. Lack of PM attention to EAC. Exact or near-exact agreement Government staff lacks comfort, between contractor EAC and experience with EVM. Likely lack government EAC of integration/interface between government EVM analysts and rest of govern ment staff, especially technical staff. Government EAC(s) consist solely Government staff lacks familiarity of "Gold Card" formula results with how to integrate risk and with little or no accompanying EVM. Likely lack of explanation/rationale integration/interface between government EVM analysts and rest of PMO staff, especially technical staff. Lack of robust explanation of EAC Lack of comfort, experience with cost drivers EVM, and especially risk and EVM. Likely disconnect between Integrated Master Schedule, Schedule Risk Analysis (SRA) and EVM. Exact or near-exact match of EAC EAC development/assessment process to prominent cost-related contract that is either immature or elements such as BAC, TAB, atrophied, PM lack of comfort with Negotiated Price, PMO budget EAC derivation and/or lack of familiarity with EVM. Could reflect organizational culture of risk aversion. Could also reflect bud get constraints. Sustained high SPI Premature claim of work accomplished. High percentage of "level of effort" earned value technique. Sustained high CPI High percentage of "level of effort" earned value technique combined with understaffing and/or substitution of lower labor rate personnel for higher rate personnel. Premature claim of work accomplished. Premature opening of work packages. Contractor expenditures different Contract Funds Status Report than anticipated (CFSR) misalignment with reported Actual Cost of Work Performed from CPR/IPMR. Must reconcile CFSR (with fee) to CPR/IPMR (without fee). No reference in EAC to schedule Disconnect between scheduling forecast derived from SRA discipline, EVM and risk management. Lack of cost adjustment to pay for "standing/marching army." To-Complete Performance Index EAC might be unrealistic. A TCPI (TCPI) associated with the EAC is exceeding the CPI by 5% warrants greater than the cumulative Cost pointed questions, and a 10% or Performance Index (CPI) reported greater difference warrants to date concern. It is rare for performance efficiency to increase by such a large margin unless fundamental changes to program management and execution are assumed. Table 2. Back-of-the-Envelope Example Assumptions: Let's say a contractor has developed an EAC and reports its composition as noted in the table below. Taking out a small sheet of paper, the government PM can apply the 90-second back-of-the-envelope approach as shown below. WHAT A THE GOVERNMENT CONTRACTOR PM'S "BACK-OF-THE- ROW TERM MIGHT REPORT ENVELOPE" NOTES 1 Actual Costs to $50 million million Date 2 Budget of work to $50 million $50 million go (BAC-BCWP) 3 MR -- $3 million 4 Risk/Opportunity $5 million $8 million (dollarized (factored) risks only) 5 Marching Army -- $6 million 6 Total $105 million $117 million NOTES ON GOVERNMENT PM'S ACTIONS ROW 1 Extracted directly from Format 1 CPR/ IPMR 2 Extracted directly from Format 1 CPR/ IPMR 3 Extracted directly from Format 1 CPR/ IPMR 4 Extracted directly from contractor Risk Register and/or Format 5 CPR/IPMR 5 Extracted from contractor SRA results for contract end date and current "burn rate" from CPR/IPMR. This is based on a 6-month contract end date slip at a burn rate of $1 million per month 6 As a result, the PM's back-of-the-envelope EAC is $12 million higher than the contractor EAC. This becomes a starting point for robust EAC conversation and an important catalyst for more detailed PMO staff analysis. The true goal is to reach a mutual understanding of the influence of risk in EAC derivation, not to determine that one EAC is more accurate than the other. The government PM's staff can, and should, perform a more thorough analysis using its own methodologies while keeping the PM dis cussion questions at the forefront. Note that it may be necessary to replace contractor risk information (such as dollarized risk and SRA) with government information should the contractor information be determined to be inadequate.
|Printer friendly Cite/link Email Feedback|
|Author:||Barker, Pat; Tomasini, Roberta|
|Publication:||Defense AT & L|
|Date:||Mar 1, 2014|
|Previous Article:||Just in time: expecting failure: do JIT principles run counter to DoD's business nature?|
|Next Article:||Bombs and books: building a better air force military acquirer.|