Candid voices: metrics in Air Force Calibration Laboratories: a case study.

Introduction

Under the direction of the Air Force Metrology and Calibration (AFMETCAL) Program, the Air Force employs a network of 75 calibration laboratories across the country and overseas to support its inventory of test measurement and diagnostic equipment (TMDE). Some of the laboratories, known as precision measurement equipment laboratories (PMEL), are operated by government employees--military, civil service, or a combination thereof. The remaining labs are contractor operated. A summary of the geographic location and who operates the PMELs can be found in Table 1. The overseas figures include PMELs located in Alaska and Hawaii.

The TMDE supported by PMELs is defined as

   ...those devices used to maintain, evaluate, measure, calibrate,
   test, inspect, diagnose, or otherwise examine materials, supplies,
   equipment, and systems to identify or isolate actual or potential
   malfunctions, or decide if they meet operational specifications
   established in technical documents (1).

Well maintained and accurate TMDE plays a key role in ensuring aircraft are able to fly when needed, navigate and communicate as necessary, and deliver their payload with precision. When examining the skills mix of a PMEL, one finds a majority of personnel found in a PMEL are metrologists. (2) They perform calibration and repair work on TMDE. Typically there may also be several support personnel assigned to accomplish tasks such as scheduling and supply. PMELs are most commonly seen as a function within the aircraft maintenance complex.

TMDE requires periodic calibration to ensure it meets its specified parameters. "Calibration is a comparison between equipment items, one of which is a measurement standard of known accuracy, to detect, correlate, adjust and report any variation in the accuracy of the other item(s)." (3) Calibration of TMDE is a crucial element in maintaining system accuracy and reliability.

This case study examines how the use of metrics might help PMELs manage the TMDE workload with a goal of reducing equipment downtime. Shortening the time TMDE is unavailable to the owners or users could cut the cost of TMDE support to the Air Force, improve customer service, and free up resources that could be used elsewhere.

Defining the Problem

Since the early 1990's the Air Force has explored ways to cut the cost of maintaining calibrated TMDE. Most of the effort was focused on outsourcing TMDE workloads, privatization of calibration facilities and their associated workloads, eliminating weapon system specific PMELs, and allowing torque calibrations to be performed outside a PMEL by nonmetrology personnel. The process the organization uses to deliver TMDE to the PMEL, the workflow through the PMEL, and the means to return equipment to the customer was only given a cursory look. Little effort was expended measuring or determining an adequate measurement to gauge the efficiency of PMEL operations. Measuring backlog, the one metric that has been used for 30 years, does little to measure the actual performance of a PMEL. The Air Force supports 608,000 items of TMDE and performs 754,000 maintenance actions involving TMDE calibration each year. Because of the number of items supported, the costs associated with the workflow are significant and warrant further investigation.

The major objectives for this study are as follows.

* Review existing metrics used by PMEL managers

* Develop notional metrics for the PMEL function

* Identify the advantages and disadvantages of each proposed metric

Literature Review

The literature review for this project is divided into two categories--government documentation and commercial standards. The government documentation section is comprised of Air Force instructions, Air Force technical orders, government reports, and contract statements of work. All instructions, technical orders and contractual documents require compliance by the user while the other documents offer guidance unless specified otherwise.

Government Documents

In December 2001 the Air Force Logistics Management Agency published the handbook Maintenance Metrics US Air Force. In the foreword, the Director of Logistics, Air Force Materiel Command, Lieutenant General Terry L. Gabreski states:

   Aircraft maintenance metrics are important. Don't let anyone tell
   you differently! They are critical tools to be used by maintenance
   managers to gauge an organization's effectiveness and efficiency.
   In tact, they are roadmaps that let you determine where you've been,
   where you're going, and how (or it) you're going to get there. Use
   of metrics allows you to flick oft your organizational autopilot and
   actually guide your unit. But they must be used correctly to be
   effective. Chasing metrics for metrics' sake is a bad thing and
   really proves nothing. A good maintenance manager will not strive to
   improve a metric but will use it to improve the performance of the
   organization. (4)

Air Force Instruction 21-113, Air Force Metrology and Calibration (AFMETCAL) Program, is the primary governing document for the AFMETCAL Program. It defines the purpose of the program and recognizes Air Force Metrology and Calibration Detachment 1 as the technical authority on metrology issues. "PMELs are owned and operated by their respective MAJCOM [major command] or FOAs (field operating activity) to provide calibration and maintenance support to TMDE operated by users within a specified geographic region." (5) Air Staff responsibilities include establishing "policy for managing and operating the AFMETCAL Program." (6) The publication does not include any production standards for managing a PMEL.

Technical Order 00-20-14, Air Force Metrology and Calibration Program, provides more in-depth metrology program guidance delineating PMEL and AFMETCAL responsibilities. There is a lengthy section outlining the responsibilities of the PMEL manager. Neither timeliness of TMDE flow times nor any other laboratory operation measurements are addressed in this section of the technical order.

The philosophy General Gabreski established in the maintenance metrics handbook has since been institutionalized in Air Force Instruction 21-101, Aerospace Equipment Maintenance Management. It points out that decisionmakers must have accurate and reliable information about their operations and that using management indicators can help them make choices that will improve the organization's performance. It also emphasizes that metrics shall be used at all levels and shall adhere to certain guidelines. These guidelines are as follows. (7)

* "Metrics must be accurate and useful for decisionmaking."

* "Metrics must be consistent and clearly linked to goals/ standards."

* "Metrics must be clearly understood and communicated."

* "Metrics must be based on a measurable, well-defined process."

These guidelines form the framework used to analyze the PMEL metrics identified in this study. Finally, the instruction assigns the PMEL chief the responsibility of evaluating the adequacy of TMDE flow time through the PMEL. It sets the standard as, "workable backlog must not exceed 7 days." (8) The source document for data is the PMEL Automated Management System (PAMS) Daily Workload Report or equivalent.

A review of 19 statements of work representing 32 of the 41 contracted PMELs revealed all but one, or 97.5 percent, have metrics for number of days backlog, turnaround time, or throughput time. The terms turnaround time and throughput time are both used to measure the time period from acceptance into the PMEL until it is ready for customer pickup or shipment back to the customer. For the purposes of this study, both terms will be referred to as turnaround time. These statistics are summarized in Table 2. Both are measures of how quickly the PMEL returns equipment to its customers

Only 18.8 percent of contracts used backlog as a measure. One possible explanation is that a backlog metric may not give a true picture of a PMEL's capability to produce equipment in a timely manner. If a PMEL has a 5-day backlog, in theory, given 5 days they could complete every workable item in the laboratory and have no remaining items if nothing was added during that timeframe. In practice, however, it may take longer if items are not routinely worked on a first-in, first-out basis. When more complex items are bypassed and equipment that takes less time to repair is worked first, the measurement can result in a backlog that might take much longer than 5 days to complete. This is an example of working to improve a metric rather than using the metric to improve the performance of the organization.

The more frequently used standard of turnaround time simply measures the number of days from the time the item is scheduled into the PMEL's workload until it is complete and ready for the customer to pick it up. The contracts reviewed all used sampling to measure turnaround time. However, with the PAMS management information system available to all PMELs, it should be easy to develop an algorithm to measure a PMEL's turnaround time performance on a weekly, monthly, or annual basis.

Commercial Standards

Air Force representatives participated in the development of ANSI/NCSL Z540-1-1994, American National Standard for Calibration - Calibration Laboratories and Measuring and Test Equipment General Requirements, a voluntary standard for operating calibration laboratories. The standard requires a laboratory to maintain a quality manual, which, in part, "shall state the laboratory's policies and operational procedures established to meet the requirements of the standard." (9) In the list of specific items to be included in the quality manual, no mention is made of standards for the timeliness of completing customer equipment items.

A more recent commercial standard is the General Requirements for The Competence of Testing and Calibration Laboratories, ISO 17025, published by the International Organization for Standardization and International Electrotechnical Commission in 1999. "It contains all of the requirements that testing and calibration laboratories have to meet if they wish to demonstrate that they operate a quality system, are technically competent, and are able to generate technically valid results." (10) This standard does not require calibration laboratories to address timeliness of service with customers. It does require a laboratory to: "have arrangements to ensure that its management and personnel are free from any undue internal and external commercial, financial and other pressures and influences that may adversely affect the quality of their work." (11)

Literature Review Summary

Air Force policy strongly encourages the use of metrics as a management tool within the maintenance community. PMELs adhere to this policy through the extensive use of backlog and turnaround time as production metrics. The private sector does not have any known production measurements for calibration laboratories.

Quality and timeliness, while not considered mutually exclusive, are often deemed to be opposing forces. It is true that the speed at which a task is accomplished can affect the quality of an output; however, it should not be viewed as an obstacle. Rather it should be considered a factor in overall product quality.

Methodology

Data Collection

Secondary data, obtained from PAMS, at several PMELs, were used in the study--extracted from the Daily Workload Report over 15 consecutive workdays. Fifteen days of data enables evaluation of trends and results in a more accurate picture than just a snapshot of a single day's management data. The data were used to gauge current performance in those PMELs and evaluate the validity of proposed management metrics. Another source of information was AFMETCAL's semiannual reports. In addition, Air Force PMEL contract information was reviewed for any performance indicators or metrics already used.

Evaluation of Current Procedures

The analysis evaluated the current use of backlog and turnaround time measurements to determine the advantages and shortcomings of each measurement. It explored the following two questions:

* Do current metrics provide meaningful management information?

* Do current metrics identify specific areas for improvement?

Examination of Potential Alternatives

The data analysis also examined whether potential alternative metrics can add value to a PMEL operation. The proposed metrics are as follows.

* Average Time in PMEL. This metric would measure the average time items spend in PMEL. It differs from turnaround time because it will include every maintenance status (for example, awaiting parts or shipped) as opposed to only those categories considered workable.

* TMDE Not Mission Capable (NMC) Rate. This measurement will identify TMDE unavailable for customer use. It is more comprehensive than current metrics because it encompasses the entire process of TMDE calibration and repair. It should be noted that PMEL managers and personnel cannot control this metric alone. It will also take involvement from customers.

Data Analysis

Current Metrics

Tracking workable backlog is a measurement of PMEL performance used widely in the Air Force. This metric is consistent for military and civil service operated PMELs and is clearly linked to a 7-day standard by AFI 21-101. Several contractual documents use backlog as a performance indicator, but they vary and are not necessarily tied to any Air Force guiding publication. Backlog has proven useful in determining the need for additional effort when the standard is not met. The backlog measure is also effective. Experience has shown that backlogs exceeding 7 days result in an increased number of requests for priority service causing the backlog to climb even higher.

Workable backlog is calculated by:

Dividing the Number of Items in Workable Status by the Average Daily Production Calculated Over the Previous Year.

The Daily Workload Report produced by the latest PAMS software provides six different figures for workable backlog. It is calculated using average daily production for a year, 180 days, 90 days, 30 days, 14 days, and 7 days. The additional calculations only provide wider variations for the same basic measurement. They are merely additional ways of looking at the same data and therefore provide no real additional value. Having multiple ways to express essentially the same measurement can lead to confusion or misrepresentation when communicating results.

The advantage of using backlog as a PMEL metric is that it is well known and accepted by workers in the lab and leadership within the wing. However, as mentioned earlier, it can misrepresent the amount of work in a lab when items are not worked on a first-in, first-out basis.

Figure 1 illustrates the workable backlog reflected on the Daily Workload Report of six different PMELs over a period of 15 workdays. It shows that only three of the six PMELs, or 50 percent, achieved the standard for the entire timeframe--two of the laboratories met the standard part of the time and PMEL E did not meet the standard at any time during the 3-week period. Without additional information this would lead one to believe PMEL E is poorly managed or lacks the resources necessary to accomplish its workload in a timely manner.

[FIGURE 1 OMITTED]

Turnaround time is not a metric typically used in a government-operated Air Force PMEL, but it is widely used in contractor-operated PMELs. Seventy-eight percent of those PMELs use turnaround time as a performance measure. Unlike the backlog standard dictated by Air Force instruction, the various contracts are not standardized, and required turnaround times vary from 5 workdays to 60 calendar days. Contracts do, however, seem to be standardized within the different major Air Force commands. There are several reasons for this. First, some commands choose to use a single contract for all contracted PMELs within their command. This is the case for Air Combat Command and Pacific Air Forces. The standards are also fairly uniform between these two commands. The other command with somewhat consistent contract standards is Air Education and Training Command (AETC). AETC uses different contracts for each of its PMELs, but the contracts are reviewed and standards approved at the command level.

Air Force Materiel Command (AFMC) uses turnaround time as a production metric, but the standards vary from 5 workdays at Robins Air Force Base, Georgia, to no standard at all for the Arnold Air Force Base, Tennessee, contract. Different standards can, at times, be justified by a given PMEL's mission. For example, all operations and maintenance functions at Arnold are covered under the same contract. As such there may be no need for a turnaround time standard in the PMEL. The contractor knows the timelines for the projects he is responsible for and is therefore in the best position to set priorities for various workloads covered by the contract including the PMEL. While this appears to be an efficient way to ensure PMEL performance at a reasonable cost, it neglects the fact that PMELs are required to "provide maximum support to Air Force activities, other federal agencies, and contractors (authorized to receive such support)." (12)

Another observation during the analysis is the degree to which AFMC turnaround time standards vary at its air logistic centers. These centers support calibration standards for other PMELs within their geographical area of responsibility. The 5-workday turnaround time at Robins is on par with most PMELs in the operational commands. However the standard of 14 days turnaround time at Tinker Air Force Base, Oklahoma, allows over twice the time to complete equipment. The standard at Hill Air Force Base, Utah, is 25 days--nearly a month. Couple the turnaround time with shipment to and from the air logistics center, and the amount of time a PMEL or its customers are without a piece of equipment can be significant. This can drive the need for additional TMDE or the need for additional planning so the items supported by the TMDE do not require periodic maintenance while the TMDE is unavailable.

Air Force Space Command uses both backlog and turnaround time as contract metrics and the standards range from 5-day workable backlog to 60 calendar days.

Due to the sensitivity of contractor performance in specific areas, no turnaround time performance data was gathered for this project and therefore, no analysis was made in this area. However, there are several theoretical advantages and disadvantages to using turnaround time instead of backlog. The primary advantage is that it measures the cumulative time the item was in a workable status in the PMEL. This typically includes awaiting maintenance status and in-work status. It usually excludes statuses that can be influenced by people not working in their own PMEL such as awaiting parts, awaiting technical data, or shipped to certifying laboratory (a higher echelon PMEL). This metric is an improvement over measuring backlog because it measures how well a PMEL manages items they have total control over.

The disadvantage is that PAMS is not set up to measure turnaround time on items flowing through the PMEL. Most government personnel who monitor contractor performance use random sampling techniques to determine whether or not the contractor meets the standard in each performance period. They utilize PAMS data to determine the time a selected item spends in various statuses, then manually calculate whether or not that item meets the contract specification. Since PAMS has all of the data necessary to calculate turnaround time, it could be programmed to provide a report with turnaround times for individual items or an average turnaround time for a given period.

Why is it then that the Air Force opts to use turnaround time in lieu of backlog for the majority of its contract operations? In addition to measurement accuracy provided by the extracted PAMS data, the information is useful in determining whether or not a contractor met performance requirements set forth in a contract. The data also indicates that turnaround time is a better measure of performance than backlog.

Average Days in PMEL

Average days in PMEL is the first of two notional metrics that may be useful to PMEL managers and other Air Force leaders. It measures the timeliness of service provided by the PMEL. It is more accurate than backlog because it lessens the possibility of items being set aside in favor of easier or less time consuming workload. The advantage it has over turnaround time is the ability to measure all of the items in the PMEL and average the entire time TMDE has spent out of the customer's hands. This differs from turnaround time, which usually excludes time spent in statuses traditionally thought of as nonworkable. It identifies the need for managers to work supply problems, technical data shortfalls, shipping delays, and other issues that keep TMDE out of their customer's hands rather than passively waiting for the problems to resolve themselves.

The information necessary to determine average days in PMEL is available in PAMS and provided on the Daily Workload Report. The formula to calculate this metric is as follows.

Average Days in PMEL = Geometric Mean of Average Days in all PAMS Statuses

Correlation coefficients for backlog and average days in PMEL varied greatly in the six PMELs studied, ranging from 0.526 to 0.693. They indicate that, in some circumstances, the two measurements may be related and measure some of the same attributes of the workload.

The average days in PMEL for the six test PMELs is graphed in Figure 2. PMEL F has a coefficient of correlation of r = 0.69. This is the highest degree of correlation between backlog and average days in PMEL in the labs studied. PMEL F has the lowest backlog over the period and also has one of the lowest average days in PMEL during the same period. However, PMEL B has the second lowest backlog for most of the run with the second highest average days in PMEL. Its coefficient of correlation of r = 0.47 is the second highest. The two measurements vary enough to indicate they are not similar measurements of a PMEL's production performance.

[FIGURE 2 OMITTED]

When the metric identifies a problem, the manager must investigate the cause before initiating corrective action. Notice PMEL C had wide variations in average days in PMEL over the 2-week observation period, ranging from 14 to 30 days. During the same timeframe, average time for items in awaiting shipment status ranged from 4 to 37 days with the number of items ranging from 10 to 34. At the same time 93 to 251 items of TMDE waited for customer pickup on average from 4 to 14 days. In this case, for example, say the lab had a standard of 2 days for items waiting for shipping, and 7 days for items awaiting customer pickup. If those standards were, at worst, met, the resulting improvement in overall average days in PMEL would vary from 1 to 7 days as illustrated in Figure 3.

[FIGURE 3 OMITTED]

TMDE Not Mission Capable Rate

The other proposed metric is the TMDE not mission capable (NMC) rate. It identifies the percentage of inventory items that cannot be used because they are overdue calibration in the PMEL being serviced or waiting to be picked up at PMEL after being serviced. The formula used is as follows.

[Overdue Equipment + PMEL Total Backlog) / (TMDE Inventory] X 100

The PAMS Daily Workload Report provides all the figures necessary to make the calculation. It also provides the percentage of inventory items within each PMEL maintenance status (awaiting maintenance, in-work, awaiting parts, and so forth) allowing for the following alternate formula.

TMDE NMC Rate = Sum of All Status Percentages Including Percent Overdue

The two formulas give slightly different results because of the rounding of the status percentages. The first is easier to calculate. Examining the data used by the second formula provides a better understanding of potential problem areas that can affect overall performance--one of the premises of using metrics. Figure 4 shows the TMDE NMC Rate for the six PMELs during the 15-day period the data was collected.

[FIGURE 4 OMITTED]

This measurement gives leadership an indication of organizational capability as well as PMEL and customer performance. As with most metrics, it is not a perfect measure since a single unique TMDE item can ground the entire wing's aircraft. However, it expands on the traditional backlog or turnaround time measurements by including overdue equipment and items awaiting customer pickup. This metric differs from the others because it looks at how much equipment is in the PMEL or otherwise unavailable rather than how fast it flows through the PMEL. Theoretically, if flow time through the PMEL is quick, its customers can afford to have this number higher.

An attribute of this metric is that it can be used at the workcenter or squadron level to provide further insight into the customer's ability to perform daily missions or deploy. It can identify problem areas that the customer may need to resolve, the PMEL may need to fix, or that require a joint effort between the customer and PMEL to find a solution. For example, if a PMEL has difficulty with overdue items, this could be a sign that the users need additional equipment because the TMDE is in constant use. Another example would be an item that cannot be located because it is seldom used. This could signal the customer has excess equipment. Another advantage of this measurement is that NMC measurements are used and understood throughout other functions in the maintenance community.

A disadvantage to TMDE NMC rate is that it is easily skewed when a customer has only a few items requiring PMEL support. A rate of 50 percent indicates serious problems with a large account, but not necessarily when the customer only has two items. Whether or not a problem exists depends on what the item is used for and when it is needed. PMEL managers must work with their customers to understand the ramifications that various NMC rates have on the customer's operational capability.

The backlog and TMDE NMC measurements appear to have a slight degree of relationship with the coefficient of correlation ranging from r = 0.18 to r = 0.69. This is to be expected because even though backlog is used to estimate the amount of time necessary to service items in the PMEL, the basis for the calculation is the number of items in the lab. TMDE NMC uses the same figures but adds equipment overdue and waiting for customer pickup.

The two notional metrics have hardly any relationship. Four of the labs had coefficients of correlation ranging from r = 0.14 to r = 0.29. The other two were very slightly related-calculations were r = 0.44 and r = 0.58.

Summary, Conclusions, and Recommendations

Evaluation of Current Metrics

The analysis of current metrics is summarized in Table 3. Both backlog and turnaround time provide management with meaningful information necessary to operate an Air Force calibration laboratory. They also identify the need for improving the time it takes to process customer equipment. However, they only address the portion of workload the PMEL has direct influence over. Both exclude data from any maintenance status considered nonworkable.

Evaluation of Notional Metrics

Table 4 summarizes the evaluation of the proposed notional metrics. Both average days in PMEL and TMDE NMC rate provide managers with information about the production performance of their PMELs. In that respect they are useful for decisionmaking. Unlike with backlog or turnaround time, PMEL personnel can break both of the notional metrics into subsets that provide specific areas for improvement. Both of the metrics can be linked to standards. However, establishing standards is beyond the scope of this study. Both metrics are also easily understood and are measurable using data already collected by the management information system in place.

Conclusion

This study analyzed the existing metrics used in PMELs (backlog and turnaround time) and two notional PMEL-specific metrics (average days in PMEL and TMDE NMC rate). Both of the notional metrics met the test of two independent variables. In both cases the notional metrics identify areas for improvement. Of the existing metrics, turnaround time is a better measure of PMEL performance than backlog; however, it is only used in contractor operated PMELs. Metrics in and of themselves do not enhance the performance of the organization. However, they do provide information for PMEL personnel to improve performance.

Recommendations

Backlog and turnaround time should not be the sole measurements for determining how quickly a PMEL produces items it has direct control over.

The Air Force should establish service-wide standards for PMELs regardless of whether they are operated by government personnel or contractors. The standards should be based on calendar days for several reasons. First it would ease programming for PAMS. Since turnaround time is already widely used, PAMS should feature the capability to run turnaround time reports. The reports should provide an average turnaround time figure and turnaround time by individual item. This will eliminate the need for manual calculation of turnaround time by people monitoring contractor performance. Second, the Air Force is a 7-day per week operation. When a field commander engaging in combat operations asks how long something will take, his or her mindset is in calendar days. Even if a PMEL normally operates 5 days per week, PMEL management must have the same mindset as the customer in order to provide the best customer support.

The Air Force should adopt the notional metrics on an Air Force-wide basis. If an Air Force-wide standard cannot be agreed upon, the major commands should implement individual command-wide standards. No single base should have its own standard.

The PMEL community must include customer feedback when determining appropriate standards.

Article Acronyms

AETC--Air Education and Training Command

AFMC--Air Force Materiel Command

AFMETCAL--Air Force Metrology and Calibration

NMC--Not Mission Capable

PAMS--PMEL Automated Management System

PMEL--Precision Measurement Equipment Laboratory

TMDE--Test Measurement and Diagnostics Equipment

Notes

(1.) Air Force Metrology and Calibration Program, TO 00-20-14. Washington, DC: Department of the Air Force, 30 Jun 2006, 11.

(2.) Air Force Instruction 21-113, Air Force Metrology and Calibration (AFMETCAL) Program, 1 February 2000, defines metrology as the science or system of weights and measures used to determine conformance to technical requirements, including the development of standards and systems for absolute and relative measurements. A metrologist is one who practices the application of metrology.

(3.) Air Force Metrology and Calibration Program, 10.

(4.) Maintenance Metrics U.S. Air Force, Gunter Annex, Maxwell AFB. AL: Air Force Logistics Management Agency, 20 Dec 2001, 2.

(5.) Air Force Metrology and Calibration (AFMETCAL) Program, Air Force Instruction 21-113 [Electronic Version], Washington. DC: Department of the Air Force, 2000, 2.

(6.) Ibid.

(7.) Aerospace Equipment Maintenance Management, Air Force Instruction 21-101 [Electronic version]. Washington, DC: Department of the Air Force, 2004, 21.

(8.) Aerospace Equipment Maintenance Management, Air Force Instruction 21-101, 125.

(9.) American National Standard for Calibration-Calibration Laboratories and Measuring and Test Equipment General Requirements, ANSI/NCSL Z540-1-1994, Boulder, CO: American National Standard Institute & National Conference of Standards Laboratories, 1994.

(10.) General requirements for the competence of testing and calibration laboratories, ISO 17025 (final draft), Geneva, Switzerland: International Organization for Standardization, & International Electrotechnical Commission, 1999, iv.

(11.) General requirements for the competence of testing and calibration laboratories, ISO 17025, 3.

(12.) Air Force Metrology and Calibration Program, TO 00-20-14, 3-19.

Senior Master Sergeant William L. Maitland retired from the Air Force after 26 years in the precision measurement equipment laboratory field. At the time of the writing of this article, he was a graduate student at Central Michigan University.

Table 1. Air Force PMELs

PMELs                 Government      Contractor
                       Operated        Operated

CONUS                     25              35
OCONUS                     8               7

Table 2. Contract Production Standards

                       Number of     Percentage of
Standard              Contracted       Contract
                     PMELs Used In       PMELs

Workable backlog
of 5 days                  2             6.3%

Workable backlog
of 7 days                  4             12.5%

Turnaround time 5
workdays                   6             18.8%

Turnaround time 7
workdays                  11             34.4%

Turnaround time 7
calendar days              4             12.5%

Throughput time
14 days                    1             3.1%

Turnaround time
25 calendar days           1             3.1%

Turnaround time
60 days or as              2             6.3%
scheduled with
customer

No standard used           1             3.1%

Table 3. Evaluation of Current Metrics

                        Backlog       Turnaround
                                         Time

Does it provide
meaningful                Yes             Yes
information?

Does it identify
areas for specific        Yes             Yes
improvement?

Table 4. Evaluation of Notional Metrics

                            Average          TMDE Non-
                            Days in          Mission
                            PMEL             Capable Rate

Is it useful for                 Yes              Yes
decision making?

Does it identify
areas for specific               Yes              Yes
improvement?

Does it enhance             No (not in and   No (not in and
performance?                of itself)       of itself)

Is it linked to goals         It can be        It can be
or standards?

Is it easily
understood and                   Yes              Yes
communicated?

Is it a measurable,
well-defined                     Yes              Yes
process?