Application of a generic evaluation methodology to assess four different chiller FDD methods (RP-1275).

BACKGROUND

In the last 15 years, the development of robust automated fault detection and diagnosis (FDD) methods applicable to HVAC&R equipment has been an area of active research, and several papers have been written on the issue (Comstock et al. 1999; Katipamula et al. 2001; Katipamula and Brambley 2005a, 2005b). Despite their importance in terms of cost and energy use, large chillers have been the focus of relatively few studies. An ASHRAE-funded research report by Reddy (2006) summarizes, with respect to FDD, the various published studies of unitary cooling equipment and chillers in terms of author, year of publication, type and size of equipment, number of faults studied, and type of action performed during both fault-detection and diagnoses stages. Practitioners had identified an urgent need to develop a general testing methodology against which different FDD methods and tools could be evaluated. The ultimate objective was to develop a test standard for the industry akin to those available for testing the normal performance of different types of HVAC&R equipment and evaluating building energy analysis computer programs (ASHRAE 2001). A preliminary version of such an evaluation methodology has been proposed as ASHRAE-funded research project RP-1275 (Reddy 2006, 2007).

The suggested methodology involved developing analytical expressions for FDD evaluation cast as an objective function made up of two competing considerations: 1) cost associated with false alarms and 2) penalties associated with the onset of faults. Further, special effort was made to determine the types of penalties associated with various faults in chiller installations, such as energy increase, loss of cooling capacity, reduced life, etc. (Reddy 2007). After discussion with service personnel of a large chiller company, it was decided to limit the FDD evaluation to the energy penalty alone. It was also pointed out that, from a practical viewpoint, FDD evaluation should be based on two criteria:

1. The normalized fault detection index resulting in a normalized score or rank between 0 and 1, where the basis of evaluation is with respect to an ideal detector with a score of unity and with no false alarms:

[[PHI].sub.Detect,s] = [[N.sub.F].summation over (f = 1)][[P.sub.f]*[DELTA][E.sub.f]*(1 - [F.sub.N,f])]/[[N.sub.F].summation over (f = 1)]([P.sub.f]*[DELTA][E.sub.f]) (1)

where

[F.sub.N,f] = false negative rate for fault f (i.e., missed-opportunity rate)

f = index for fault type

[N.sub.F] = total number of possible faults in the system

[P.sub.f] = probability of occurrence of fault type f

[[DELTA]E.sub.f] = extra electric power required to provide necessary cooling due to performance degradation as a result of fault type f

2. The combined fault detection and fault diagnosis index consisting of four different diagnosis outcomes (correct and unique, correct but non-unique, unable to diagnose, and incorrect diagnosis), all of which have different implications for time taken (i.e., cost) by the technician or the serviceman to diagnose the fault, make an evaluation, and choose an appropriate course of action:

[[PHI].sub.FDD] = [[N.sub.F].summation over (f = 1)][[P.sub.f]*[DELTA][E.sub.f]*([w.sub.cu]*[r.sub.cu,f] + [w.sub.cn]*[r.sub.cn,f] + [w.sub.ic]*[r.sub.ic,f] + [w.sub.ud]*[r.sub.ud,f])]/[[N.sub.F].summation over (f = 1)][[P.sub.f]*[DELTA][E.sub.f]] (2)

where

[r.sub.cu] = correct and unique diagnosis rate expressed as a fraction of the signaled faulty data

[r.sub.cn] = correct but non-unique diagnosis rate

[r.sub.ic] = incorrect diagnosis rate

[r.sub.ud] = unable to diagnose rate

[w.sub.cu] = weighting factor for correct and unique diagnosis rate (same for each fault type)

[w.sub.cn] = weighting factor for correct but non-unique diagnosis rate

[w.sub.ic] = weighting factor for incorrect diagnosis

[w.sub.ud] = weighting factor for unable to diagnose

Numerical values of various quantities appearing in the above expressions are discussed later in this paper.

OBJECTIVES AND SCOPE

The objectives of the research summarized in this paper were to review existing literature and propose four different chiller FDD methods--either adapting existing ones to chillers or proposing new ones if necessary--in order to evaluate them on the basis of the proposed FDD methodology and thereby identify those most promising. The scope of the proposed research was limited to process fault detection and did not include sensor faults, actuator faults, or control loop or controller faults (Wang and Cui 2006). Also, the FDD processes were to rely on continuous thermal, pressure, and electrical measurements as opposed to one-time diagnostic measurements or other tests such as vibration and electrical signature analysis, visual inspection, oil-wear debris analysis, or surface and internal defect detection tests (Davies 1998). The scope of this research was limited to FDD methods based on steady-state data, which are consistent with most of the FDD work to date in the HVAC&R area with the exception of a couple of studies (Bruecker and Braun 1998a, 1998b; Stylianou 1997) that use such transient data only cursorily and in a manner lacking rigor. Finally, only centrifugal chillers were considered. This limits the size of chillers to above around 80 tons (281 kW) and excludes unitary equipment such as rooftop units. Medium-to-large chillers come equipped with elaborate safety control mechanisms for critical/catastrophic faults. This research was not targeted at these faults or the detection of hard faults, such as fan-belt breakage or a burnt motor, but rather toward incipient faults, which lead to energy wastage and gradually damage equipment. Further, medium-to-large chillers come equipped with numerous sensors (usually temperature, pressure, and electrical measurements on individual sub-components) and contain distinct loops, such as the condenser and evaporator loops, refrigerant loops, and cooling oil loops. Thus, any FDD method should explicitly make use of such a data-rich environment for which component isolation methods (McIntosh et al. 2000; Jia and Reddy 2003; Wang and Cui 2006) seem particularly appropriate. On the other hand, calibrated simulation model approaches for FDD are deemed best suited for systems where limited sensor data are available, such as unitary rooftop cooling equipment (Rossi and Braun 1997; Brueker and Braun 1998a, 1998b; Castro 2002).

DESCRIPTION OF CHILLER DATA SETS USED

The research supporting this paper makes use of the numerous experiments, under both fault-free and faulty conditions, performed within the framework of previous ASHRAE research project, RP-1043 (Comstock and Braun 1999). Specifically, experimental data on a 90-ton (316 KW) centrifugal water-cooled chiller were collected in which 1) a wide variety of chiller faults were studied--eight to be exact, but only six are considered here (Table 1), and 2) each fault was introduced at four levels of severity (10%-40% fault levels in increments of about 10%) denoted by SL1-SL4. Which physical quantities were altered and by how much in order to simulate the effect of the faults and their severity levels are also indicated in Table 1. Numerous sets of tests were performed under each of the eight different faults, introduced one at a time, under benchmark (or normal, fault-free, or baseline) conditions and under four different fault-severity conditions. Note that several replicate sets of tests had to be performed under fault-free conditions in order to re-establish the baseline each time a specific fault, previously introduced, had to be rectified prior to introducing another fault. Each experimental test set consists of 27 performance points obtained by varying the following three control variables: 1) chilled-water outlet temperature from chiller evaporator, [T.sub.evo]; 2) condenser water inlet temperature, [T.sub.cdi]; and 3) chiller thermal load, [Q.sub.ev] (derived measurement). So as not to introduce bias, a careful analysis was performed on the numerous baseline data sets available in order to identify four that seem to be least noisy and most consistent with each other (Reddy 2006). One of these was used to calibrate the FDD methods, while all four test sets were then used during the FDD evaluation phase to tune the fault-detection thresholds to pre-selected false alarm rates.

Table 1. Summary of RP-1043 Lab Chiller Data Sets (Comstock and
Braun 1999)

    Description of     Normal        SL1        SL2        SL3
       Fault         Operation

1  Reduced           270 gpm (17  0.87-0.93  0.77-0.81  0.69-0.70
   condenser water          L/s)
   flow               (0.98-1.0)

2  Reduced               216 gpm  0.90-0.91  0.81-0.82  0.72-0.72
   evaporator water   (13.6 L/s)
   flow               (0.99-1.0)

3  Refrigerant       300 lb (136        0.1        0.2        0.3
   leak                      kg)

4  Refrigerant       300 lb (136        0.1        0.2        0.3
   overcharge                kg)

5  Condenser           164 tubes       0.06       0.12       0.20
   fouling                 total

6  Noncondensables            No       0.01      0.017      0.024
   in system (by        nitrogen
   volume)

    Description of      SL4
      Fault

1  Reduced           0.59-0.61
   condenser water
   flow

2  Reduced           0.63-0.65
   evaporator water
   flow

3  Refrigerant             0.4
   leak

4  Refrigerant             0.4
   overcharge

5  Condenser              0.30
   fouling

6  Noncondensables       0.057
   in system (by
   volume)

Note: Fractional values indicate the level of fault severity. For
example, the range 0.59-0.61 under SL4 for reduced condenser water flow
indicates that the flow was reduced to about 60% of the normal value.

TYPES OF VARIABLES

Experience gained from past studies indicates that fault detection can be more sensitive if certain characteristic quantities (CQs) or characteristic parameters (CPs) representative of physical or thermal properties of the chiller sub-components are used instead of the basic sensor measurements themselves (Comstock and Braun 1999; McIntosh et al. 2000; Wang and Cui 2006). These CPs and CQs can be directly deduced from the sensor measurements using arithmetic operations and thermodynamic refrigerant property tables or correlations. Note that since CPs have physical meaning, baseline or fault-free models of their behavior identified from the performance data of the numerous sensors available are likely to yield more meaningful FDD results (Jia and Reddy 2003; McIntosh et al. 2000). However, distinction between CQs and CPs are blurred when numerical values of a parameter change during operation; for example, chiller coefficient of performance (COP) and overall heat conductance (UA) of a flooded-type evaporator usually change with operating conditions. A distinguishing trait is that CPs are those that in some manner better capture than CQs the performance of the internal state of the system or its components in response to specific values of forcing functions. Hence, we have also adopted the terminology characteristic features (CFs), which would include both CQs and CPs.

Definitions of the seven CQs and seven CPs used in this paper are provided in Table 2 along with their symbols and computational definitions. The overall chiller COP is also considered an additional CF. It was also found that the primary chiller measurements presumed to be available in this study were consistent with those presumed in allied published studies on large chillers (Reddy 2006).

DESCRIPTION OF THE FOUR FDD METHODS EVALUATED

Based on a literature review of existing FDD approaches, four were selected for evaluation that were deemed simple enough to be practical at this early stage of FDD tool implementation while exhibiting adequate diversity in terms of pre-processing of primary sensor data and in their conceptual approaches to FDD. All four methods evaluated belong to the same general class, namely data-driven methods. This choice is natural because quantitative models are usually more sensitive and better suited for engineering systems than purely heuristic ones; furthermore, chillers come equipped with a large array of built-in sensors that provide an advantageous, data-rich environment.

Would faulty data be required for the particular FDD process, and if so, how much? Only requiring fault-free data for FDD process training would be ideal, while requiring elaborate faulty data under various faults at different fault severity levels for each chiller installation would be least desirable, since such data are very hard to come by in practice. Because the basic trend knowledge captured by a fault diagnosis table or tree--such as that for unitary rooftop air-conditioning units in Rossi and Braun (1997), Chen and Braun (2001), and Li and Braun (2003)--was lacking, data from fault severity Level 4 (the highest level) for each fault along with one fault-free data set of 27 observations were used to extract fault features by determining association of a specific CF with a particular fault and codifying the diagnosis rules. In order to be consistent, it was assumed that this information is known to all four FDD methods being evaluated. The four FDD methods evaluated (Figure 1) and their variants are briefly described in the following sections (Reddy 2006).

[FIGURE 1 OMITTED]

Table 2. Characteristic Quantities and Parameters Evaluated

     Description     Symbol

--   Chiller         COP_overall
     overall COP

CQ1  Evaporator      T_evi-T_evo
     water
     temperature
     difference

CQ2  Condenser       T_cdo-T_cdi
     water
     temperature
     difference

CQ3  Refrigerant     T_cpis
     compressor
     suction
     superheat

CQ4  Refrigerant     T_cpos
     compressor
     discharge
     superheat

CQ5  Refrigerant     T_cds
     condenser
     subcooling

CQ6  Condenser       T_c-T_cdo
     approach
     temperature

CQ7  Evaporator      T_evo-T_e
     approach
     temperature

CP1  Overall         UA_cd
     condenser heat
     loss
     coefficient

CP2  Overall         UA_ev
     evaporator
     heat loss
     coefficient

CP3  Polytropic      Effy_Poly
     efficiency of
     the
     compressor

CP4  Isentropic      Effy-Isen
     efficiency of
     the
     compressor

CP5  Expansion       Cd.A0*10^6
     valve blockage
     coefficient

CP6  COP of the      COP_cycle
     thermodynamic
     cycle

CP7  Motor           Effy_motordrive
     efficiency

     Computed as

--   [Q.sub.ev]/E

CQ1  [T.sub.evi]-[T.sub.evo]

CQ2  [T.sub.cdo]-[T.sub.cdi]

CQ3  [T.sub.cpi]-[T.sub.e]

CQ4  [T.sub.cpo]-[T.sub.c]

CQ5  [T.sub.cpo]-[T.sub.c]

CQ6  [T.sub.c]-[T.sub.cpo]

CQ7  [T.sub.evo]-[T.sub.e]

CP1  [(UA).sub.cd] = [C.sub.p] [m.sub.cd]
     In([T.sub.cdo]-[T.sub.c]/[T.sub.cdi]-[T.sub.c])

CP2  [(UA).sub.ch] = [C.sub.p] [m.sub.ch]
     In([T.sub.cho]-[T.sub.c]/[T.sub.chi]-[T.sub.c])

CP3  [[eta].sub.p] = ([P.sub.2] [v.sub.2]-[P.sub.1]
     [v.sub.1])
     In([P.sub.2]/[P.sub.1])/([h.sub.2]-[h.sub.1]
     In(([P.sub.2] [v.sub.2])/([P.sub.1] [v.sub.1]))

CP4  = [h.sub.2]"-[h.sub.1]/[h.sub.2]-[h.sub.1]

CP5  [C.sub.d][A.sub.0] = [m.sub.r]
     [[upsilon].sub.3]/2[square root of
     (term)][P.sub.3]-[P.sub.4]

CP6  = [h.sub.1]-[h.sub.4]/[h.sub.2]-[h.sub.1]

CP7  [COP.sub.overall]/[COP.sub.cycle]

Note: [m.sub.r] is the refrigerant mass flow rate calculated from an
energy balance on the evaporator = [Q.sub.ev]/[h.sub.1]-[h.sub.4]. P,
v, and h are the refrigerant absolute pressure, specific volume, and
enthalpy, respectively.

FDD#1: MODEL-FREE FAULT DETECTION WITH DIAGNOSIS TABLE

The first approach evaluated is the model-free approach, which uses heuristically determined fault detection thresholds along with a diagnosis table. This is similar in concept to the simple rule-based method (SRBM) proposed by Chen and Braun (2001) for packaged rooftop air-conditioning units. No regression model is used, nor is there need to calculate normalized residuals. The fault detection thresholds were simple range limits for each variable. An exploratory data analysis identified specific CFs that were affected by different faults, while their deviations varied sufficiently with load such that it was necessary to divide them into three different load conditions for deducing operating ranges for heuristic-based FDD methods. This is illustrated in Figure 2, where a sample plot depicts how CQ5 values are impacted by fault F3 (refrigerant leak). We have divided the operating range into three regions--low, medium, and high evaporator loads (with no distinction made for the other two conditions, [T.sub.cdi] and [T.sub.evo])--to determine the range limits for each CF directly from the baseline fault-free data set. Note that these look-up table ranges are likely to be specific to a particular chiller size, make, and model; however, these would be easy to determine if baseline fault-free data were available. No normalization of the deviations has been made.

[FIGURE 2 OMITTED]

Table 3. Fault Diagnosis Table Proposed for Model-Free FDD#1

Fault        Fault         CQ1  CQ2  CQ5  CQ6  CP1
Code     Description

F1     Reduced                  +
       condenser water
       flow rate

F2     Reduced             +
       evaporator water
       flow rate

F3     Refrigerant                   -    -    +
       leak

F4     Refrigerant                   +    +    -
       overcharge

F5     Condenser                               -
       fouling

F6     Noncondensables               +    +    -
       in system

Note: The + and - signs indicate directional change of the numerical
values of the CFs with increasing fault severity. For example, as the
condenser water flow rate decreases, CQ2 will increase. Note that
refrigerant overcharge and non-condensables in system faults cannot be
uniquely distinguished.

The specific CFs affected by different faults and their fault diagnosis rules are shown in Table 3. For example, if the observed numerical value of CQ2 falls above the range stipulated as fault-free for the particular load bin, this would imply the onset of fault F1 (reduced condenser water flow rate). On the other hand, fault F3 would be signaled only if the numerical values of CQ5 and CQ6 are both lower than their stipulated ranges while that of CP1 is concurrently higher. This set of Boolean rules form the basis of our fault diagnosis. Note that the refrigerant overcharge and noncondensables in system faults cannot be uniquely identified.

From Table 3, we note that only 5 of 15 CFs are used. In case multiple CQs are affected by the onset of a particular fault, fault diagnosis may be more robust if different subsets of the identified diagnosis rules are evaluated. For example, from Table 3 we note that the refrigerant overcharge fault (F4) results in an increase in CQ5 and CQ6 and a decrease in CP1. Instead of the fault diagnosis requiring that all three fault trends be detected, one could elect to make a positive diagnosis when even two out of the three (2/3) fault trends are observed. Hence, the following two variants were evaluated:

* FDD#1-1: Diagnosis rule combination of 3/3 for faults F3, F4, and F6

* FDD#1-2: Diagnosis rule combination of 2/3 for faults F3, F4, and F6

FDD#2: Multiple Linear Regression (MLR) Black-Box Model Innovations for Fault Detection with Diagnosis Table

The second FDD method evaluated is based on the analytical redundancy approach similar to Chen and Braun (2001) and Li and Braun (2003) for roof-top units and Grimmelius et al. (1995) and Wang and Cui (2006) for chillers. Polynomial black-box multiple linear regression (MLR) models for each CF have been identified from one fault-free data set of 27 individual performance data points using forward step-wise regression to assure model parsimony. The MLR model structure assumed is the second-order polynomial model with [Q.sub.ev], [T.sub.cdi], and [T.sub.evo] as the three regressor variables. For a specific chiller operating condition, the particular models are used to predict the values of the various CFs from which the residual for each CF is normalized by the root mean square error (RMSE) of the fault-free model so as to provide the well known Student's t-statistic:

[(UA).sub.cd] = [C.sub.p][[bar.m].sub.cd]ln([[T.sub.cdo] - [T.sub.c]]/[[T.sub.cdi] - [T.sub.c]]) (3)

Fault detection was done based on a Student's t-test, which is theoretically more sophisticated than the simple-minded approach used in FDD#1. The numerical value of this statistic can be directly interpreted in statistical measures if a Student's t-distribution of the model errors is assumed.

Combinations of strong association between CFs and faults were identified using a statistical approach involving medians rather than means for increased robustness (Reddy 2006) and then framed as diagnostic rules, as shown in Table 4. It is interesting to note that although the analysis methodology to identify these rules was different from that of FDD#1, these diagnostic rules are almost identical to those of FDD#1 (Table 3). The only difference is F5, condenser fouling, which could be because the condenser model identified is poor. Finally, it was found that CP5 and CP6 would also be impacted were a noncondensable fault to occur, which was not the case for FDD#1. In Table 4, a cell value indicated as zero for a specific CF implies that the diagnosis would involve performing a statistical test to verify that the t-statistic has neither increased nor decreased. The four cells corresponding to CP5 and CP6 against faults F4 and F5 are shown italicized to indicate that different diagnosis combination schemes with and without these values were also investigated:

* FDD#2-3: Effect of limiting the entire FDD method to using five CFs only, namely CQ1, CQ2, CQ5, CQ6, and CP1, so as to be consistent with FDD#1. A statistical test is done on CQ5 to ascertain whether it is zero or not for fault F5 as per Table 4. Note that the CFs selected are consistent with those selected in FDD#1.

* FDD#2-4: All seven CFs (CQ1, CQ2, CQ5, CQ6, CP1, CP5, and CP6) are used with CP5 and CP6 set to zero for faults F4 and F5 as per Table 4.

* FDD#2-5: Same as FDD#2-4 except that no zeros are set for CP5 and CP6.

Table 4. Fault Diagnosis Table Proposed for MLR Model-Based FDD#2

Fault        Fault         CQ1  CQ2  CQ5  CQ6  CP1  CP5  CP6
Code     Description

F1     Reduced                  +
       condenser water
       flow rate

F2     Reduced             +
       evaporator water
       flow rate

F3     Refrigerant                   -    -    +
       leak

F4     Refrigerant                   +    +    -   0*    0
       overcharge

F5     Condenser                     0    +    -    0    0
       fouling

F6     Noncondensables               +    +    -    -    -
       in system

Note: The + and - signs indicate directional change of the t-statistic
with increasing fault severity. For example, as the condenser water
flow rate decreases, the t-statistic for CQ2 will increase. Except for
F5 and F6, this table is identical to that of FDD#1 (see Table 3) if we
neglect the effect of CF5 and CP6 on F6.

* The four cells corresponding to CP5 and CP6 against F4 and F5 are
shown italicized to indicate that different diagnosis combination
schemes with and without these values were investigated.

FDD#3: Principal Component Analysis (PCA) Model Innovations for Fault Detection with Diagnosis Table

A statistical analysis revealed that a lot of redundant information is contained in the 15 CFs; three principal components explain 99.4% of the variability in the data. Hence, although 15 CFs are being monitored, there is a good deal of redundancy in the variation of the original variables, implying that most of them are measuring similar attributes of the process. Being able to reduce the dimensionality of the problem without losing much information regarding variability would be very desirable, both for fault detection as well as diagnosis, because the detection logic and diagnostic rules would be much simpler. A well-known method for achieving a reduction in dimensionality of the problem while leading to non-collinear (i.e., orthogonal) quantities is to use principal component analysis (PCA) (Manly 2005). Such an approach has been used by Wang and Cui (2006) with good results for detecting chiller sensor faults.

Hence, the third FDD method evaluated was to use the principal component model with fault innovations for fault detection and an association table for diagnosis. The fault diagnosis rules have been identified in a manner analogous to that of FDD#2 and are shown in Table 5. Note that F4 (refrigerant overcharge) and F6 (noncondensables in system) cannot be uniquely identified, which is consistent with FDD#1 and FDD#2 based on five CFs only. The following variants to this method have been investigated:

* FDD#3-6: Using all 15 CFs for calculating the three principal components (PCs); performing diagnosis per the association rules shown in Table 5.

* FDD#3-7: Using only five (the same five identified by FDD#1, namely CQ1, CQ2, CQ5, CQ6, and CP1) for calculating two PCs; performing diagnosis per slightly different rules (Reddy 2006).

FDD#4: Linear Discriminant and Classification Approach

The fourth FDD approach evaluated was the linear discriminant and classification approach whereby fault detection and diagnosis could be done simultaneously. This approach is conceptually similar to the classification provided by an artificial neural network approach. Discriminant analysis and classification are multivariate techniques concerned with separating distinct objects or observations and allocating new ones to previously defined groups (Manly 2005). This is a more statistically refined version of FDD#1 and allows the association of faults and CFs to be identified simply in addition to allowing easy implementation. Discriminant analysis provides the necessary methodology to statistically distinguish differences between two or more groups when one knows beforehand that such groupings exist in the data set provided. Subsequently, it can be used to assign, allocate, or classify a future observation into a specific group. Hence, it allows one to train the data with known fault-free and faulty data sets and then conveniently determine whether a particular performance condition datum is faulty or not. Analyzing and interpreting the results from discriminant analysis is similar to MLR analysis except here the dependent variable is categorical (either 0, fault-free, or 1, faulty).

Table 5. Association Rules and Fault Diagnosis Table for FDD#3 with
All 15 CFs

            Fault                               Diagnosis Rules

                                              PC1        PC2  PC3

F1  Reduced condenser water flow rate                 +         -
F2  Reduced evaporator water flow rate                -
F3  Refrigerant leak                                       +    +
F4  Refrigerant overcharge                                 -    -
F5  Condenser fouling                                           -
F6  Noncondensables in system                              -    -

Note that the refrigerant overcharge fault cannot be distinguished
from the noncondensables in system fault.

The 15 CF variables have been used with the fault-free data set of 27 observations from fault-free baseline data along with those from each of the faulty data sets at fault severity level 4 of the RP-1043 lab chiller data (Comstock and Braun 1999) to develop linear discriminant models using forward step-wise model identification. The detection and diagnosis aspects of this FDD method involve identifying one set of seven models simultaneously using one set of baseline data in conjunction with the SL4 fault data sets of all six faults. The models are found to be excellent with 100% classification accuracy with respect to the data sets used to train them (namely, fault-free baseline and fault SL4). These functions can be used to predict the group into which a future observation can be placed. No preprocessing of data is necessary except to calculate the seven CFs that will not only indicate whether a fault has occurred but also suggest the cause. The following different variants have been evaluated:

* FDD#4-8: Based on classification functions for all 15 CFs identified using fault-free and SL4 data sets.

* FDD#4-9: Based on classification functions using five CFs (CQ1, CQ2, CQ5, CQ6, and CP1) consistent with the ones used in FDD#1.

SPECIFIC CRITERIA FOR FDD EVALUATION

The FDD evaluation is based on several presumptions, each of which is described below:

1. As stated in the Background section, the FDD methods ought to be evaluated based on two separate issues: 1) their detection capability only and 2) their combined fault detection and diagnostic capability. The merit of the former is that the robustness it would provide to service companies during the early adoption of FDD tools would be beneficial to gradual field adoption of FDD methods in general. It is more important to be sure that a fault has indeed occurred in the system so that a service technician can be dispatched than to be able to diagnose the fault with more certainty. Thus, we will adopt the normalized score given by Equation 1 to rank the FDD methods in terms of fault detection capability, while Equation 2 allows normalized ranking in terms of each method's overall fault detection and diagnosis capability.

2. Establishing thresholds for flagging the occurrence of faults is a critical issue. On one hand, being too aggressive would lead to too many false alarms (with drastic consequences in terms of the operator disabling the FDD system entirely), while being too conservative would result in excessive energy wastage and other undesirable consequences (the very issues the FDD system is meant to avoid in the first place). One reasonable approach is to first tune the detection thresholds of the relevant CFs for each of the various FDD methods being evaluated using fault-free data so that they have the same false alarm rate and then evaluate them on their detection and diagnostic capabilities separately for different faults and fault severity levels. Thus, the fault detection thresholds for each FDD method were tuned individually using four fault-free data sets containing 96 observations total. Threshold tuning was done by widening/tightening the thresholds of each of the three ranges of each CF for FDD#1 by the same amount, increasing/decreasing the numerical values of the t-statistic of each CF for FDD#2 and FDD#3, and by increasing/decreasing the numerical values of the constant term of the linear discriminate model for FDD#4. A value of 95% for the correct fault-free detection rate (approximately four to five of the 96 fault-free observations lie outside the threshold) was selected for baseline evaluations. Sensitivity evaluations of the various FDD methods were also performed for three other false alarm rates (2.5%, 7.5%, and 10%) in addition to the 5% value.

3. One needs to evaluate the FDD capability of the four methods and their variants from the perspective of six different faults at four severity levels (Table 1). In order to get a combined correct fault detection rate, one needs to weight the rates by 1) the frequency of occurrence of each type of fault and 2) the electric energy increase at different severity levels for each type of fault. Reddy (2006, 2007) discusses these issues in addition to others, such as how one includes the adverse effects of cooling capacity loss or increased wear and tear due to the onset of different types of faults. Data from the RP-1043 lab chiller (Comstock and Braun 1999) as well as simulations from an in-house computer program of a large chiller manufacturer have been analyzed, which, in conjunction with personal discussions with the service managers of a large chiller company, yielded preliminary but realistic values of energy penalties that were then used in the evaluation of the four FDD tools. The ranks or weights associated with the occurrence frequency of various faults as well as the weights for four possible outcomes of a fault diagnosis were also determined based on personal discussions. These are shown in Table 6, and the assumed diagnosis outcome weights are shown in Table 7. A value of 1.0 has been selected for correct and unique diagnosis (the most favorable outcome) while poorer ones have lower weights (for example, incorrect diagnosis has a weight of 0.4). Two additional sets of weights used for sensitivity analysis are also shown in Table 7.

Table 6. Values Used in the FDD Evaluation for Excess Electric Energy
Use (%) of Different Faults at Different Severity Levels(1)

                      Energy Penalty ([[DELTA}E.sub.f])

    Fault             SL 1  SL 2  SL 3  SL 4
                        %    %       %     %

F1  Reduced           0.70  1.9   3.0   5.3
    condenser water
    flow

F2  Reduced            0.0   0.0  0.40  0.90
    evaporator water
    low

F3  Refrigerant       0.14  0.31  0.47  0.71
    leak

F4  Refrigerant       0.80  0.94   3.8   7.6
    overcharge

F5  Condenser         0.50  0.50  0.50   1.8
    fouling

F6  Noncondensables    4.5   6.2   7.4  15.6
    in refrigerant

       Energy Penalty
     ([[DELTA}E.sub.f])

    [CFS.sup.2]             Frequency
         %                   Weights
                           ([P.sub.f])

    Fault Data   Assumed
        Set      Value %

F1  SL3              3.0        1.0

F2  SL4             0.90        2.0

F3  SL4             0.71        1.5

F4  SL3              3.8       0.25

F5  SL4              1.8        1.0

F6  SL1              4.5        0.5

1. Includes frequency weights, their occurrence, and impact.
2. CFS = composite fault severity level. Different severity levels were
selected for different faults so that a composite severity level could
be generated with an associated increased energy use as close to 1%-3%
as permitted by data. The corresponding fault data sets for different
CFS faults are also shown.
Table 7. Weights Assigned to the Four Different Diagnoses Outcomes

Outcome      Symbol    Base     Sensitivity  Sensitivity
                     Weights      D1           D2

Correct and  w_cu       1.00         1.00         1.00
unique

Correct but  w_cn       0.75         0.65         0.85
non-unique

Unable to    w_ud       0.50         0.40         0.60
diagnose

Incorrect    w_ic       0.40         0.30         0.50
diagnosis

4. The choice of the fault severity levels follows the tests performed under RP-1043 (Comstock and Braun 1999), and these severity levels seem to have been arbitrarily chosen. For example, F6 has a higher energy penalty at SL1 than do F2 and F5 at SL4. Instead of merely presuming the same severity levels arbitrarily selected in RP-1043, this research also evaluated the four FDD methods in terms of a composite fault severity (CFS) level constructed from different fault severity levels of varying faults that result in approximately the same energy penalty. We have assumed 1%-3% as a threshold of increased energy use due to a fault that would warrant operator intervention and selected the composite fault data sets for different faults (coded CFS) as shown in Table 6.

DISCUSSION OF RESULTS

Evaluating Fault Detection Capability Using Severity Level Categories

Three cases have been run corresponding to 1) no energy penalty weights and no occurrence frequency rank, 2) energy penalty weights but no occurrence rank, and 3) energy penalty weights and occurrence rank, the results of which are plotted in Figures 3a-3c. For example, a value of 0.7 would imply that the particular method only catches a fault 70% of the time when applied to a faulty data set consisting of 27 data points. We note the following:

[FIGURE 3 OMITTED]

1. As expected, fault detection capability in all cases improves as the fault severity increases.

2. The effect of including energy penalty weights and occurrence frequency weights does alter the normalized rank values of the various FDD methods but not to the extent that we need to alter our conclusions regarding the relative performance of each FDD method.

3. The two variants of FDD#1 are the best under the higher fault severity levels (SL3 and SL4), while their performance degrades appreciably under lower severity levels. However, if no frequency weights are applied (Figure 3b), FDD#1 and FDD#2 are a close tie.

4. The fault detection capability of FDD#4-8 is excellent for SL4 (the state used to identify the linear discriminant models), while it serves very poorly for other severity levels. This suggests that the model approach has poor predictive ability when applied to data sets other than those used to train it--a drawback also widely attributed to artificial neural networks, which are nonlinear versions of the linear discriminant model. Further, the distinction between fault-free and faulty values of the CFs is reduced at lower fault severity levels, and a model trained with SL4 data appears not to have the sensitivity necessary to robustly distinguish between states with closer classification boundaries.

5. The two variants of FDD#3 are generally poor and have no redeeming features. A physical explanation as to why the PCA method is poor may have to do with the manner in which the principal components are determined. They are linear weighted measures of several CFs as opposed to one or a few individual CFs that are directly impacted by a specific fault. Since specific faults are associated with directional changes of certain CFs only, the fault signal represented by the principal components is weakened by the presence of CFs, which are not directly impacted by the onset of the particular fault. This could be the reason why the principal component model results in reduced sensitivity compared to using only the pertinent CFs directly.

6. The three variants of FDD#2 are generally the best among all the other methods since they are close to FDD#1 while being more stable at lower fault severity levels. Further, all three variants have almost identical fault detection scores. Hence, overall we would conclude that the simpler FDD#2-3 using only five CFs (CQ1, CQ2, CQ5, CQ6, and CP1) can be deemed to rank the best in terms of fault detection of the four FDD methods.

Evaluating Overall FDD Capability Using Severity Level Categories

Comparative results in evaluating the overall FDD process are plotted in Figure 4. Again, the values are normalized scores, which can be evaluated against an ideal FDD (one that detects and diagnoses perfectly) with a score of 1.0. As in Figure 3, three cases have been run, as shown in Figures 4a-4c. The same general conclusions can be drawn as previously except that now FDD#1-1 is the best of all the FDD methods evaluated, with the three FDD#2 methods coming second. However, it is our belief that the success of FDD#1 in the tests is entirely an artifact of the way the test-data loads match the training-data loads. The experimental design adopted by RP-1043 lab chiller tests (Comstock and Braun 1999) result in the data points scattering neatly in three chiller load ranges (minimum, medium, and maximum), which are rather distinct from one another with no overlap between boundaries (Figure 2). This was the case for the fault-free data sets as well as the fault data at the four severity levels. Since this would not happen in practice, we feel that the above evaluation is not fully valid and may have biased the evaluation in favor of FDD#1. Thus, it can be argued that FDD#2-3 (based on five CPs only) should be deemed the best choice. Analyses of other chiller data sets would clarify this issue.

[FIGURE 4 OMITTED]

Finally, as previously, the effect of including energy penalty weights and occurrence frequency weights do alter the normalized rank values of the various FDD methods but not to the extent that we need to alter our conclusions regarding the relative performance of each FDD method.

Evaluating Overall FDD Capability Using CFS Categories

The evaluations involving fault detection capability only and the combined FDD capability were repeated using the energy penalty weights corresponding to the CFS levels rather than the arbitrarily selected severity level values used earlier. The energy penalty weights under the CFS evaluation are shown in Table 6, while the base values for the diagnostic weights are the same used previously (Table 7). The results of this analysis for fault detection capability only and for the combined FDD capability are summarized in Figures 5a and 5b for the same three earlier cases, depending on whether energy penalty weights and fault frequency occurrence weights were considered during the fault detection phase. Note that the patterns for all three cases are very similar, indicating that the selection of specific values of energy penalty weights and occurrence frequency weights is not a crucial issue. Again, we basically arrive at essentially the same conclusions as previously: FDD#2-3 is the best for detection, while in terms of overall FDD it is superseded by FDD#1-1. Methods FDD#2 and FDD#4 are similar in terms of overall FDD, but the latter requires much more elaborate faulty data in order to train the linear classification models, while FDD#2 does not.

[FIGURE 5 OMITTED]

In addition, two types of sensitivity analyses have also been performed:

1. Instead of using a pre-selected false-alarm rate of 5% (i.e., a correct fault-free detection rate of 95%), all analyses were repeated assuming three different false alarm rates (2.5%, 7.5%, and 10%). Given that we have only 97 fault-free data points to tune the detection thresholds, we could not go lower than 2.5% because of the resulting lack of robustness. The results are shown in Figure 6.

[FIGURE 6 OMITTED]

2. We have selected two different sets of diagnoses weights, rather than the one assumed for all the above analyses, and repeated the analyses (Table 7 assembles the values chosen for these weights under sensitivities D1 and D2). The results of these sensitivity analyses are not provided in this paper but can be found in Reddy (2006).

Essentially, the conclusions of all these sensitivity analyses were found to be consistent across the choices of 1) the energy penalty weights, 2) the fault occurrence frequency weights, and 3) the diagnoses weights. Further, we note that these conclusions are also similar to those reached using security level categories.

CONCLUSIONS

The evaluation of the four FDD methods (actually, nine variants were evaluated) was performed based on both fault detection capability only and on the overall FDD capability. Most of the evaluation was done based on a false-alarm rate of 5% (i.e., the fault detection thresholds for each FDD method were tuned to achieve this rate), but sensitivity analyses were also made based on 2.5%, 7.5%, and 10% false-alarm rates. Further sensitivity analyses were also made by selecting different weights for energy use penalties and the four diagnoses outcomes. The evaluation results consistently pointed to FDD#1-1 as the best overall both for fault detection and for combined FDD performance. However, it was pointed out that this could be an artifact of the way the test-data loads match the training-data loads and that further studies are needed to clarify this issue. Overall, FDD#2 was found to be the best. Possible reasons why FDD#3 and FDD#4 performed poorly were also provided. In conclusion, this paper serves to illustrate the application of the FDD methodology, highlight the benefit of the FDD evaluation tool in identifying the most promising FDD method for practical evaluation, and identify the most promising chiller FDD tool suitable for field evaluation.

ACKNOWLEDGMENTS

This research was sponsored by ASHRAE Technical Committee 7.5, Smart Building Systems. Constructive comments and suggestions by the project monitoring committee members are greatly appreciated. We acknowledge advice and helpful feedback from Ashish Singhal, John Seem, Bill McQuade, Justin Kauffman, Dennis Dietz, and John House. The assistance of Chanin Panjapornpon and Wei Jiang, former graduate students, is also acknowledged.

NOMENCLATURE 

[C.sub.d][A.sub.0] = expansion valve blockage coefficient 

[C.sub.d] = specific heat at constant pressure

E = electric power input to compressor 

[F.sub.N,f] = false negative rate for fault f

[F.sub.P] = false positive rate

f = index for fault type

h = enthalpy 

m = mass flow rate

[N.sub.F] = total number of possible faults in system

[P.sub.f] = probability of occurrence or frequency weight of fault type f

P = pressure

[Q.sub.ev] = thermal heat load or capacity

[r.sub.cu] = correct and unique diagnosis rate expressed as a fraction of the signaled faulty data

[r.sub.cn] = correct but non-unique diagnosis rate

[r.sub.ic] = incorrect diagnosis rate

[r.sub.ud] = unable to diagnose rate

T = temperature

[T.sub.c] = saturated refrigerant temperature in condenser

[T.sub.cdi] = condenser water inlet temperature

[T.sub.cdo] = condenser water outlet temperature

[T.sub.co] = refrigerant temperature leaving condenser

[T.sub.cpi] = refrigerant temperature entering compressor or leaving evaporator

[T.sub.cpo] = refrigerant temperature at compressor discharge

[T.sub.e] = saturated refrigerant temperature in evaporator

[T.sub.evi]i = evaporator water inlet temperature

[T.sub.evo] = evaporator water outlet temperature

t-value = Student's t-statistic

UA = overall heat conductance of heat exchanger 

[w.sub.cu] = weighting factor for correct and unique diagnosis rate (same for each fault type)

[w.sub.cn] = weighting factor for correct but non-unique diagnosis rate

[w.sub.ic] = weighting factor for incorrect diagnosis

[w.sub.ud] = weighting factor for unable to diagnose

x = regressor variable

y = response variable

[[DELTA]E.sub.f] = extra electric power required to provide necessary cooling due to performance degradation as a result of fault f

[upsilon] = specific volume

[eta] = efficiency

[phi] = normalized rating index for fault detection and FDD defined by Equations 1 and 2

1,2,3,4 = refrigerant state points on the pressure-enthalpy diagram indicating inlet to compressor, exit from compressor, exit from condenser, and inlet to evaporator

Subscripts

cd = condenser

ch = chiller, evaporator

ev = evaporator

r = refrigerant

s = index for fault severity level

Acronyms

CF = characteristic feature

CFS = composite fault severity

COP = coefficient of performance of chiller

CP = characteristic parameter

CQ = characteristic quantity

FDD = fault detection and diagnosis

MLR = multiple linear regression

PCA = principal component analysis

RMSE = root mean square error

REFERENCES

ASHRAE. 2007. ANSI/ASHRAE Standard 140-2001, Standard Method of Test for the Evaluation of Building Energy Analysis Computer Programs. Atlanta: American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc.

Breuker, M.S., and J.E. Braun. 1998a. Common faults and their impacts for rooftop air conditioners. HVAC&R Research 4(3):303-18.

Breuker, M.S., and J.E. Braun. 1998b. Evaluating the performance of a fault detection and diagnostic system for vapor compression equipment. HVAC&R Research 4(4):401-25.

Castro, N.S. 2002. Performance evaluation of a reciprocating chiller using experimental data and model predictions for fault detection and diagnosis. ASHRAE Transactions 108(1).

Chen, B., and J.E. Braun. 2001. Simple rule-based methods for fault detection and diagnostics applied to packaged air conditioners. ASHRAE Transactions 107(1).

Comstock, M.C., and J.E. Braun. 1999. Development of analysis tools for the evaluation of fault detection and diagnostics in chillers. ASHRAE Research Project RP-1043, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta. Also, Report HL 99-20 #4036-3, Ray W. Herrick Laboratories, West Lafayette, IN.

Comstock, M.C., B. Chen, and J.E. Braun. 1999. Literature review for applications of fault detection and diagnostic methods to vapor compression cooling equipment. Report HL 99-19 #4036-2, Ray W. Herrick Laboratories, West Lafayette, IN.

Davies, A. 1998. Handbook of Condition Monitoring. London: Chapman and Hall.

Grimmelius, H.T., J.K. Woud, and G. Been. 1995. On-line failure diagnosis for compression refrigeration plants. International Journal of Refrigeration 18(1):31-41.

Jia, Y., and T.A. Reddy. 2003. Characteristic physical parameter approach to modeling chillers suitable for fault detection, diagnosis and evaluation. ASME Journal of Solar Energy Engineering 125:258-65.

Katipamula, S., and M.R. Brambley. 2005a. Methods for fault detection, diagnostics and prognostics for building systems--A review, part I. HVAC&R Research 11(1):3-25.

Katipamula, S., and M.R. Brambley. 2005b. Methods for fault detection, diagnostics and prognostics for building systems--A review, part II. HVAC&R Research 12(2):169-87.

Katipamula, S., R.G. Pratt, and J.E.Braun. 2001. Building systems diagnostics and predictive maintenance, Chapter 7.2. CRC Handbook for HVAC&R Engineers, ed. J. Kreider. Boca Raton, FL: CRC Press.

Li, H., and J.E. Braun. 2003. An improved method for fault detection and diagnosis applied to packaged air conditioners. ASHRAE Transactions 109(2).

Manly, B.J.F. 2005. Multivariate Statistical Methods: A Primer, 3d ed. Boca Raton, FL: Chapman & Hall/CRC.

McIntosh, I.B.D., J.W. Mitchell, and W.A. Beckman. 2000. Fault detection and diagnosis in chillers--Part 1: Model development and application. ASHRAE Transactions 106(2).

Reddy, T.A. 2006. Evaluation and assessment of fault detection and diagnostic methods for centrifugal chillers--Phase II. Final Project Report of ASHRAE RP-1275, American Society of Heating, Refrigerating and Air-Conditioning Engineers, Inc., Atlanta.

Reddy, T.A. 2007. Formulation of a generic methodology for assessing FDD methods and its specific adoption to large chillers. ASHRAE Transactions 113(1).

Rossi, T.M., and J.E. Braun. 1997. A statistical, rule-based fault detection and diagnostic method for vapor compression air conditioners. HVAC&R Research 3(1):19-37.

Stylianou, M.P. 1997. Application of classification functions to chiller fault detection and diagnosis. ASHRAE Transactions 103(1):641-48.

Wang, S., and J. Cui. 2006. A robust fault detection and diagnosis strategy for centrifugal chillers. HVAC&R Research 12(3):407-28.

T. Agami Reddy, PhD, PE

Fellow ASHRAE

Received October 25, 2006; accepted April 20, 2007

This paper is based on findings resulting from ASHRAE Research Project RP-1275.

T. Agami Reddy is a professor in the Civil, Architectural and Environmental Engineering Department at Drexel University, Philadelphia, PA.