Printer Friendly

Characteristics and Causes of Outliers in Inverse Modeling of Residential Building Energy Use Data.


The energy use attributed to buildings has significantly increased in recently years. Buildings account for approximately 40% of worldwide energy consumption (UNEP 2016). In the U.S., residential and commercial buildings consumes approximately 40% of total energy use (U.S. EIA 2016). Therefore, the identification and implementation of methods to reduce the energy and electricity use of buildings is of primary importance. In a residential building, variability of energy use in end uses that are occupant-dependent is up to 10 times higher than end uses that are not occupant-dependent (Cetin et al. 2014). A recent study as well as several review studies of behavioral energy efficiency (Cetin et al. 2016, Mahone et al. 2011) have found that indirect feedback methods achieve an average of 2% for enhanced billing strategies such as providing insights and information on whole-home energy use, disaggregated end uses, and forecasted energy use. To achieve accurate predictions of monthly energy use, the use of inverse modeling techniques such as change-point models is necessary (ASHRAE 2013). There are many other inverse modeling methods (Zhang et al 2015), however the limited amount and frequency of monthly data per home, the highest frequency of data available to approximately half of U.S. residential buildings, statistically lends itself to the use of models with a small number of independent variables, such as change-point models.

Change-point models relate the outdoor temperature to the energy use of a building (ASHRAE 2013, Kissock et al. 2003, Paulus et al. 2015). Change-point models typically divide the energy use performance of a building into multiple seasons, using multiple different regression models to represent energy performance behavior of a building and its systems throughout the heating, cooling and/or transition seasons. However, one challenge that arises with these models, particularly for residential buildings, is that the prediction of energy use is strongly influenced by occupant behavior, which can be unpredictable. For some residential buildings, energy use patterns are not consistent, limiting inverse model's accuracy due to (a) the presence of energy use outlier months, in which a home consumed significantly more or less than the energy use trends and data-driven model would have otherwise predicted, or (b) due to homes with monthly energy use patterns that are inconsistent with what change-point modeling inputs would expect. For (a), in many cases, such as those mentioned by Kim et al. (2015), the energy use prediction using a change-point model might not align with the actual monthly use for most months due to the presence of outliers. This limits the accuracy of the models. In most cases, unlike Kim et al. (2015), the ability to directly contact the homeowner to understand what has caused the outlier in the energy use data is not possible. Similarly, for (b), it is challenging to develop the inverse change-point models for some residential buildings with highly variable energy use. In these buildings, the energy use does not have a strong relationship with monthly outdoor temperature data, nor does it follow the expected trends associated with change-point models. In this research, after developing an inverse change-point model with monthly energy use data for 140 homes, multiple different methods for detection of outliers are applied, and the causes of these outliers is determined using disaggregated end-use data. The understanding of characteristics of these outliers and why these have occurred will help in the treatment of those inconsistencies in the models of buildings energy use prediction. This helps lead to a solution for how to treat outliers in inverse models for better prediction of building energy use.


The methodology developed for identifying characteristics and causes of outliers in change-point models of residential building energy use is shown in Figure 1. This includes five main steps: (1) data quality control (QA/QC), (2) develop the inverse models, (3) identify outlier type as either (a) data point or (b) pattern outliers, (4) identify characteristics and causes of outliers or identify the source of model issues, and finally (5) the outlier(s) treatment.

Step 1--Data quality control

This step involves the collection, processing, quality control, and characterization of building energy use data and weather data. A minimum of monthly-level frequency whole-building data is required for inverse model development, outlier detection and treatment (Steps 1-3, 5). However for definitively determining the cause of identified outliers (Step 4), highly granular energy use data and associated weather data must be used. In this study, energy use data for residential buildings located in Austin, TX (Jan 1, 2012 - Dec 31, 2015) is utilized. To be included in the analysis, each home must have least one complete year of continuous data in order to cover the energy performance of the building throughout all seasons. Data includes one-minute level whole-home and circuit level electricity use, representing appliances, lighting, HVAC, and other plug loads, as described in Cetin et al. (2014). For the temperature data, hourly averages are grouped into bins, where the bin size is 2.8[degrees]C (5[degrees]F). The distribution of hourly average outdoor temperature in 2015 are demonstrated in Figure 2. Compared to the averaging method, this bin method helps limit the influence of significant fluctuation of hourly data that might occur over the time period of study (Erbs et al. 1983). This is one of several methods commonly used for inverse modeling (ASHRAE Handbook 2013) and was found to also work well when using monthly energy use data for residential buildings (Cetin and Kallus 2016).

Step 2--Development of inverse model

Change-point models use outdoor temperature data as the independent variable and energy consumption as dependent variable, to create two, three, four, and five-parameter change-point heating and/or cooling models (Kissock et al. 2003, Haberl et al. 2003). A base temperature is determined as the balance-point temperature. At this temperature or temperature range, a building requires minimal energy use for heating and/or cooling due to the comfortable ambient outdoor temperatures that also enable thermally comfortable indoor conditions (ASHRAE 2013) without the use of HVAC. The basis of the algorithm developed for choosing base temperature is discussed in Paulus et al. (2015). To choose the best change-point model type for each home, a shape test (positive or negative slopes for each part of the model), significance test (p-value below 0.05 for each regression line), data population test (at least three data points in each regression line) (Paulus et al. 2015), and [R.sup.2] test ([R.sup.2] value higher 0.5 for each regression line). A p-value of 0.05 is a commonly used threshold in statistics to test the significance of parameters of models (Paulus et al. 2015). [R.sup.2] value of 0.5 is acceptable threshold of the coefficient of determination that is used to check the fitness of data points in the models (Henseler et al. 2009, and Hair et al. 2013). The data population test is to check if there are enough of data points to form the regression line in heating or cooling season of change-point model (Paulus et al. 2015). The shape test is to make sure the appropriate shape of change-point model with appropriate positive or negative slopes for each part of the change-point model (Paulus et al. 2015). With all tests passed, the inverse change-point model that has the lowest value of root mean squared error (RMSE) is chosen.

Step 3--Identify and classify outliers

The types of outliers are classified into two categories, (a) data point outliers and (b) pattern outliers. For (a), data point outliers occur if the monthly electricity usage lies significantly outside the model prediction, thus having a significant influence on the resulting model parameters if the data point is included. For (b), the electricity use patterns of homes that are inconsistent with what inverse modeling techniques accept as inputs (i.e. if there is no inverse change-point model found for a specified residential building due to insufficient passing of requisite tests of inverse change-point development, as discussed in Step 2 and associated references).

To determine point outliers, outlier detection is conducted using the standard deviation method. The standard deviation is among the most popular method applied in energy performance contracting and in measurement and verification (M&V) activities (M & V Guidelines 2011). Two other methods for outlier detection are also applied, including the quartile method and Grubbs' test method. The common threshold in standard deviation method is 2 (M & V Guidelines 2011, Seo 2006, Rousseeuw et al. 1987). The minimum threshold value used in quartile method is 1.5 (Tukey 1977). The threshold in Grubbs's test is the G critical value determined based on t-distribution (NIST 2012). Buildings with pattern outliers are identified as those that have not passed one or more of the four tests, i.e. the model generation algorithm results in no model creation for that particular building(s).

Step 4--Determine causes of outliers

This step includes two parts. Note if disaggregated and/or highly-granular energy data is not available for a particular home of study, this step may be skipped, as the cause of the outliers cannot definitively be determined. In Step 4a, the data point outlier(s) is studied to determine the cause of outlier occurrence. 10-15 individual circuits per home are used which represent both non-HVAC and HVAC end uses. As monthly non-HVAC loads are often fairly consistent throughout the year (Cetin et al 2016), an average of each non-HVAC use circuit is determined, then if one or more months of energy use (points) occur outside a threshold standard deviation, this is considered a contributing factor of the whole-home energy use outlier. For the HVAC system, a change-point model is developed specifically for the HVAC system energy use and the same outlier detection method is used from Step 3a. For Step 4b, the requisite tests (shape, significance, [R.sup.2] , and data population) that that failed when developing models was determined; the cause is assess through evaluation of each failed tests from the 5-parameter (5-P) to 2-parameter (2-P) models. The term of "pattern outlier" is referenced from several studied (Joon and Bea 2010, Pai et al 2014, Zengyou et al. 2005, Lin et al. 2010, Said et al. 2012, Gagare and Natikar 2015). For example, Gagare and Natikar (2015) suggested that "Pattern those which occur unusually or surprisingly called as Outlier Pattern".

Step 5--Outlier Treatment

For data point outlier(s), the comparison of the ability of the developed model to predict electricity consumption from in-sample data and out-of-sample data is conducted, using the change-point models with and without the outlier(s). The values used to evaluate model performance are the Residual Sum of Squares (RSS) and Root Mean Squared Error (RMSE). These values are commonly used statistical measures to assess the fit of a model to a dataset (Grace-Martin 2012). If the models without outliers have lower values of RMSE and RSS for in-sample and out-of-sample data, the outlier(s) are removed for better for the accuracy of electricity prediction. Otherwise, outliers are kept. For the case of pattern outliers, other types of inverse models are applied, including (a) a modified version of a change-point model, termed a "segmented" change-point model. The "segmented" model does not requires the different segments of the change-point model to intersect. The second method (b) is relaxing the criteria of the four tests used when developing change-point models, including increasing the p-value or threshold of [R.sup.2]. Then, similarly, the accuracy of models are checked with the values of RSS and RMSE using in-sample data and out-of-sample data.


After quality control, a total of 140 single family homes in the dataset have 100% data of whole-home and up to 10-15 individual circuits of electricity use data at one-minute intervals; data was combined into monthly intervals for this study. Using monthly electricity use data in 2015 as dependent variable and binned outdoor temperature data as independent variable, a coded algorithm was created to develop the change-point models for each home. Models were created for 128 homes; 12 homes did not fit the minimum set criteria for the development of a change-point model and thus do not have models using the original algorithm (Table 1). Examples of model results are included in Figure 3. In homes that have models (Figure 3a-3d), the base temperature is automatically determined, ranging from 10[degrees]C (50[degrees]F) to 23[degrees]C (73[degrees]F). This Figure also shows several point outliers (2b-d). Due to the occurrence of point outliers, the change-point models fit to the data in Figure 3(b) and (d) can be improved. Based on the improvement in model fit and accuracy from removing the outlier in the in-sample and out-of-sample data in Steps 4a and 5a, the outlier is removed and a new, 3P change-point cooling model is used for both datasets. Figure 3e-3f are two examples of pattern outliers. Although the data in these houses satisfy the shape and data population test of 5P change-point model, they still do not pass the threshold criteria for the significance or [R.sup.2] tests. These data also do not satisfy the four tests' criteria for the 4P, 3P cooling, 3P heating, and 2P change-point models. Therefore, no model is initially assigned to these houses' data. If the relaxed criteria for the significance and [R.sup.2] test are utilized, or the "segmented" model is applied, the 5P model can be used for these houses.

For the point outliers, results from outlier method #1 (Standard Deviation), #2 (Quartile), and # 3 (Grubbs' Test) are 7, 17, 12 outliers respectively. The number of outliers detected from each method is different, since the properties of each outlier detection method are different. The standard deviation (SD) method is a more commonly used method among three methods for outlier detection in energy data since the SD method measures the dispersion of all individual data points around the mean (Lal 2012); the quartile and Grubbs' test do not reflect the properties of all observation values of dataset. The quartile method only looks at the variation of middle of dataset and excludes the extreme values, while Grubbs' test just considers the maximum/minimum value and do not evaluate remaining values.

For each house in the dataset (n=140), 17% of houses were found to have point outliers. A larger number of point outliers occurred in December, January, and February in the heating season, around the holiday and winter vacation times, and in June and September for cooling season (Table 1). The causes of the point outliers include both the HVAC system and non-HVAC appliances such as cooking appliances (Figure 4), the water heater, clothes washer and dryer, plug loads and lighting. To determine whether or not to keep or remove a point outlier, the RSS and RMSE are compared between the in-sample data (2015) and out-of-sample data (2016) with and without the outliers (Table 2). If the RMSE and RSS values are lower if the outlier(s) is removed using the out-of-sample data, removing the outlier(s) is recommended. Otherwise, keeping outlier(s) in the models is suggested. A total of 11 homes (46%) are recommended to remove the outliers, while 13 homes (54%) are recommended to keep the outliers in the model (Table 2). There are more houses that keep outlier(s) than remove outlier(s) because the energy uses in these residential houses are less predictable. Therefore, it is beneficial to check accuracy of models with or without outlier(s) by using in-sample and out-of-sample data before making a decision to keep or remove outlier(s) in the models.

For pattern outliers, there are 12 houses (9%) that do not fit the criteria of any of the five types of inverse change-point models. The model developed based on the data from these houses failed one or more of the following criteria: (a) it did not have appropriate signs associated with the slope of the developed model (negative or flat in heating season, positive or flat in cooling season); (b) the p-value of the independent variable was not considered significance (p-value < 0.05); (c) the [R.sup.2] value was less than the threshold level ([R.sup.2]<0.5); (d) there were not at least three data points available for use in creation of each regression line portion of the model. Table 3 includes a summary of the number of houses that did not satisfy all requirements when testing the fit of the 5-P change-point model to the 2-P change-point model. Each of the homes was tested to determine if the four criteria (a-d) were satisfied; if a home did not fit the criteria of a 5-P change-point model, it was then checked for the 4-P change-point model, then 3-P then 2-P, thus the values in Table 3 include the results of testing of all homes with outliers for each of the different change-point models.

In Table 3, nearly 70% of houses failed the (b) significance test and/or the (c) [R.sup.2] test, while only two houses failed the (a) shape test, and 1 house did not satisfy the (d) data population test. Thus the (b) significance and (c) [R.sup.2] test are the most influential in determining the fit of a home's monthly energy use data to a model, even if the proposed model meets criteria (a) and (d). For these homes with pattern outliers, developing a model to better fit the data is attempted through either using relaxed criteria applied by increasing the acceptable p-value in the significance test (p-value < 0.10), decreasing the required [R.sup.2] value in [R.sup.2] test ([R.sup.2] > 0.4), or using a "segmented" change-point model, where the heating and cooling season portions of the model are not required to intersect at a common point. Allowing for a model that has segments that do not intersect can make logical sense from a system operational standpoint. For example at the outdoor temperature where the HVAC system operational settings are changed from single mode cooling, to single mode heating operation, which is common in residential buildings, the set points used in the two seasons have a significant influence on the characteristics of each of the model segments, and may therefore make the operational characteristics significantly different and not necessarily compatible in the cooling season as in the heating season. Examples of both options are demonstrated in Figure 5. Two inverse change-point cooling models with 2-P and 3-P (17% of houses, Figure 5a-b) are created by using relaxed criteria. The use of a proposed "segmented" change-point model enables six houses (50%) to have 4-P (Figure 5c) and 3-P cooling "segmented" change-point models. In Figure 5c, the 4P change-point "segmented" model was chosen with four tests satisfied. In this type of model, the regression lines in the heating and cooling season did not intersect at a common point to provide a strong fit to the data. The remaining homes still did not have a model due to highly unpredictable energy use patterns. The RMSE and RSS values are also calculated to check the accuracy of these type of models with in-sample and out-of-sample data (Table 4). Both median values of RMSE and RSS from the in-sample data are smaller than values from the out-of-sample data. As compared to other houses with developed models, the fit of the models of this house is not as good, and this is generally the case for homes that cannot fit anything but a 2P change-point model, particularly with relaxed criteria. However, the use of the relaxed criteria still allows for a model to be created for this house. Therefore, using the inverse change-point models with relaxed criteria or using "segmented" change-point model can be beneficial in some houses that do not have any suitable inverse models initially, to enable a model to be developed.


When developing inverse change-point models to predict the energy use of residential buildings, outliers occur often and affect the accuracy of models. For the monthly data point outliers, their causes and characteristics are investigated by studying one-minute interval circuit level data. Understanding of characteristics of these outliers and why these have occurred helps to evaluate the impact of outliers on inverse change-point models effectively before determining whether to remove or keep these outliers. For the patterns outliers, the evaluation of these outliers are based on the examination of four requisite tests, and treatment for these houses includes relaxed regression criteria or using another type of proposed model, called a "segmented" change-point models. This effort is a strong starting point that, through further study, will help to realistically predict the cause of such outliers and the likelihood that they will occur for homes without highly detailed data. Additionally, further study is needed to detect the occurrence of outliers in more complex models using independent variables including weather data (e.g. temperature, humidity, solar radiation, wind speed), and other smart technology data. This type of inverse model will enhance the prediction of energy use data in residential buildings.


The dataset of residential building energy use data used in this work are from the Pecan Street Research Institute, a nonprofit research institute headquartered in Austin, TX.


ASHRAE. 2013. ASHRAE Handbook--Fundamentals. Atlanta: ASHRAE.

Cetin, K.S., Tabares-Valasco, P.C., and Novoselac, A. 2014. Appliance daily energy use in new residential buildings: Use profiles and variation in time-of-use. Energy and Buildings 84(2014): 716-26.

Cetin, K.S. and Kallus, C. (2016). Data-Driven Methodology for Energy and Peak Load Reduction of Residential HVAC Systems. International Conference on Sustainable Design, Engineering and Construction. Tempe, AZ. May 18-20, 2016

Ehrhardt-Martinez, K., Donnelly, K.A., and Laitner, J.A. 2010. Advanced metering initiatives and residential feedback programs: a meta-review for household electricity-saving opportunities. American Council for an Energy-Efficient Economy, Washington, D.C, U.S.A. E105.

Erbs, D.G., Klein, S.A., and Beckman, W.A. 1983. Estimation of Degree-Days and Ambient Temperature Bin Data from Monthly-Average Temperatures. ASHRAE Journal, June 1983.

Gagare and Natikar (2015). An Improved Framework for Outlier Periodic Pattern Detection in Time Series Using Walmat Transaction Data. International Journal of Innovation in Engineering, Research and Technology [IJIERT]. ICITDCEME'15 Conference Proceedings. ISSN No - 2394-3696.

Grace-Martin, K. 2012. Assessing the Fit of Regression Models. Cornell University. Cornell Statistical Consulting Unit. <>.

Haberl, J.S., Claridge, D.E., Sreshthaputra, A., Kissock, J.K. 2003. Inverse modeling toolkit: Application and Testing. ASHRAE Transactions 109(2) KC-03-2-2 (RP-1050)

Hair, J.F., Ringle, C.M., Sarstedt, M. 2013. Partial Least Squares Structural Equation Modeling: Rigorous Applications, Better Results and Higher Acceptance. Long Range Planning 46(2013):1-12.

Henseler, J., Ringle, C., and Sinkovics, R. 2009. The use of partial least squares path modeling in international marketing. Advances in International Marketing (AIM), 20, 277-320

Joon and Bea (2010). A pattern-based outlier detection method identifying abnormal attributes in software project data. Information and Software Technology .52 (2010) 137-151.

Kim, K.H., and Haberl, J.S. 2015. Development of methodology for calibrated simulation in single-family residential buildings using three-parameter change-point regression model. Energy and Buildings 99(2015): 140-52.

Kissock, J.K., Haberl, J.S., and Claridge, D.E. 2003. Inverse modeling toolkit: Numerical algorithms. ASHRAE Transactions 109(2): 8.

Lal K. 2012. Measures of Dispersions: Range, Quartile Deviation, Mean Deviation, Standard Deviation and Lorenz Curve-Their Merits and Limitations Geography Class: Previous and Specific Uses. Kurukshetra Univeristy. <>

Lin et al. (2010). Research on Maximal Frequent Pattern Outlier Factor for Online High Dimensional Time-Series Outlier Detection. Journal of Convergence Information Technology. 5 (2010) 66-71.

Mahone, A., and Haley, B. 2011. Overview of residential energy feedback and behavior-based energy efficiency. Energy and Environmental Economics, Inc., California, U.S.A. 415.391.5100.

M & V Guidelines. 2011. M & V Guidelines for HUD Energy Performance Contracts. Guidance for ESCo-Developed Projects. <>

NIST 2012. Engineering Statistics Handbook. Section 3. Grubbs' test for outliers. E-Handbook of Statistical Methods. U.S. Department of Commerce. <>.

Paulus, M., Claridge, D., Culp, C. 2015. Algorithm for automating the selection of a temperature dependent change-point model. Energy and Buildings. 87 (2015) 95-104.

Pai et al (2014). A relative patterns discovery for enhancing outlier detection in categorical data. Decision Support Systems. 67 (2014) 90-90.

Rousseeuw, P.J., and Leroy, A.M. 1987. Robust regression and outlier detection. Wiley Series in Probability and Statistics 10.1002/0471725382.

Said et al (2012). Outlier Detection Scoring Measurements Based on Frequent Pattern Technique. Research Journal of Applied Sciences, Engineering and Technology 6(2013): 1340-1347.

Seo, S.W. 2006. A review and comparison of methods for detecting outliers in univeriate data sets. Gradute School of Public Heath, University of Pittsburgh.

Tukey, J.W., 1977. Exploratory Data Analysis. Princeton University. Pearson 1977.

United Nations Environment Programme (UNEP). 2016. Sustainable building and climate initiative. United Nations Avenue, Nairobi, Kenya. <>.

United States Energy Information Administration (US. EIA). 2016. How much energy is consumed in residential and commercial buildings in the United States? U.S. Department of Energy, Washington, DC, U.S.A.

Zengyou et al. (2005) FP-outlier: Frequent pattern based outlier detection. Computer Science and Information Systems. 2 (2005) 103-118.

Zhang, Y., O'Neill, Z., Dong, B., and Augenbroe, G. 2015. Comparisons of inverse modeling approaches for predicting building energy performance. Building and Environment 86(2015): 177-90.

Huyen Do, M.Sc.

Student Member ASHRAE

Kristen Cetin, PhD, PE

Associate Member ASHRAE

Trevor Andersen

Student Member ASHRAE
Table 1. Inverse change-point (CP) models and number of point outliers

                # of     Outliers  Number of point outliers in each
Type of models  houses   (#)       Jan  Feb  Mar  Apr  May  Jun  Jul
5-P CP            3      --        --   --   --   --   --   --   --
4-P CP           18       2        --   --   --   --   --   --   2
3-P CP cooling  102      20        3    3    1    --   --   3    --
2-P CP cooling    5       2        --   --   1    --   --   --   1
No model         12      --        --   --   --   --   --   --   --

                Number of point outliers in each
Type of models  Aug  Sep  Oct  Nov  Dec

5-P CP          --   --   --   --   --
4-P CP          --   --   --   --   --
3-P CP cooling  --   3    1    --   6
2-P CP cooling  --   --   --   --   --
No model        --   --   --   --   --

Table 2. Accuracy of inverse change-point models with/without point

Median values        In-sample data
                     with outlier(s)  without outlier(s)

For houses with recommendation of removing outlier(s) (N=11)
RMSE (kWh)           104              85
RSS ([(kWh).sup.2])  108,733          65,270
For houses with recommendation of keeping outlier(s) (N=13)
RMSE (kWh)           127              86
RSS ([(kWh).sup.2])  161,569          65,890

Median values        Out-of-sample data
                     with outlier(s)     without outlier(s)

For houses with recommendation of removing outlier(s) (N=11)
RMSE (kWh)           118                 106
RSS ([(kWh).sup.2])  140,788             113,073
For houses with recommendation of keeping outlier(s) (N=13)
RMSE (kWh)           127                 142
RSS ([(kWh).sup.2])  162,022             201,657

Table 3. Causes of pattern outliers of homes, by type of model

Types of models  Number of houses that did not satisfy the requirements
                 (a) Shape test  (b) Significance test  (c) [R.sup.2]

5-P CP           2                8                      8
4-P CP           0                9                      8
3-P CP cooling   0               11                      6
3-P CP heating   1                4                      4
2-P CP           0                9                     12

Types of models  Number of houses that did not satisfy the requirements

                 (d) Data population test

5-P CP           0
4-P CP           0
3-P CP cooling   1
3-P CP heating   0
2-P CP           0

Table 4. Accuracy of relaxed criteria and "segmented" change-point

Median values        In-sample data  Out-of-sample data

RMSE (kWh)           159             192
RSS ([(kWh).sup.2])  478,337         578,274
COPYRIGHT 2018 American Society of Heating, Refrigerating, and Air-Conditioning Engineers, Inc. (ASHRAE)
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2018 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Author:Do, Huyen; Cetin, Kristen; Andersen, Trevor
Publication:ASHRAE Conference Papers
Article Type:Report
Date:Jan 1, 2018
Previous Article:CFD Analysis of Dispersion of Particles and Gasses in Buildings for Optimal IAQ Sensor Positioning.
Next Article:Control-Oriented Modeling of an Air-Based Electric Thermal Energy Storage Device.

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters