Adding value to valuation with confidence.
In the appraisal process, reporting only the most probable point on a value distribution range does not fully reflect the probabilistic nature of a value. The degree of confidence an appraiser has in a value range can be expressed in probabilities. This article reviews a traditional statistical technique of estimating confidence levels from a normal distribution and proposes a method for appraisers to measure and report confidence levels.
How Important Is Variation?
A set of twins come home from school with their report cards. They both have a C average. So are they identical? The first one has all C's. The second has A's in half his classes and F's in the other half. Same average but dramatically different variation.
Variation can also describe the degree or level of confidence in an estimate. Small variation around a value suggests higher confidence, while large variation would indicate lower confidence.
For many clients, the confidence level the appraiser has in the value estimate is as important as, if not more than, the value estimate itself. Appraisers generally do not offer much discussion on the variation or confidence of their value estimates. Nassim Taleb (1) argues that since estimating is based on probability, it is insufficient to estimate anything without also including an estimation error in the analysis. In other words, if value is estimated, the confidence of the estimate needs to be considered and reported as well.
Since variation measures risk, it should be estimated and reported. Rummerow notes, "Most investors, lenders, and other users of valuations are deeply concerned with risk." He also states, "No doubt appraisers must report specific value estimates. But value is added for most clients if the appraiser also reports on the possible variation of prices from this point estimate." (2) Wolverton writes, "I can't imagine why an analyst or client would not value an understanding of the relative precision of an inference." (3) Perhaps the most astute observation was made by Trimble, who notes "estimates always have uncertainty. Estimates free of uncertainty are called facts." (4)
Appraising and Statistics
If economists suffer from "physics envy" because they cannot reduce human behavior to a few simple equations, then appraisers must be afflicted with statistics envy. The statistics discipline has developed powerful tools to analyze data. Unfortunately, these tools make assumptions and requirements that appraising rarely provides.
Much has been written about statistics and appraising. The Appraisal of Real Estate (5) devotes Chapter 14 and Appendix B to statistical analysis in real estate. The Appraisal Institute's educational criteria require demonstrated proficiency in basic statistics. To support this requirement the Appraisal Institute has created courses and published the text An Introduction to Statistics for Appraisers. (6)
Appraising has always had a certain "statistickiness" about it. However, The Appraisal of Real Estate and many articles in The Appraisal Journal acknowledge the weakness of applying traditional statistical techniques to the appraisal process.
The Philosophy of Confidence
In statistical analysis, confidence is expressed in probabilities on a range of possible values. Probability is first and foremost a philosophical problem. (7) Once the basic logic principles are established, then the math can be applied. Traditional statistics has developed a variety of philosophical approaches and accompanying mathematics to support them. Unfortunately, classical statistics philosophy makes a number of assumptions that most appraisals cannot meet. Therefore, a new philosophy must be developed before confidence can be estimated in valuation. We may be able to use classical methods, however, to build a new philosophy that does not require strict conformance to the assumptions. An expanded philosophy may also provide for judgment narratives instead of formal mathematical methods.
Give Yourself a Margin of Error
A note on confidence: under traditional inferential statistics, precision and confidence are inversely related. More precision means less confidence and more confidence means less precision. Statistically speaking, point estimates of the mean in a normal distribution actually have zero confidence. Under a bell-shaped curve, there is a 50% chance the value could be higher and a 50% chance it could be lower.
In the process of estimating value, appraisers develop a sense of the variation of their estimate. Providing variation estimates along with point estimates of value may significantly help users and appraisers better understand and manage their risk.
Learning from Las Vegas
Perhaps appraisers should consider thinking more like oddsmakers and betting handicappers. In other words, appraisers should think about the probability distribution of values.
Appraisers hold that one of the meanings of market value is the "most probable" price. "Most probable" implies a range of values with different probabilities and potential for errors. Statisticians call this a probability density function. If we can estimate the most probable price, we should also have some idea of the shape of the probability density function. While this is generally not derivable from the data collected in most appraisals, a shape should come to the appraiser's mind based on judgment and experience.
To simplify the concept let's consider values to be a normal distribution or bell curve. Values are plotted on the horizontal axis and the probability of each value is plotted on the vertical axis. The area under the curve between two values is the cumulative probability.
While the normal distribution may not be realistic for all valuation situations, it does provide a beginning point in thinking about the probability distribution of value estimates. Note that other shapes that are skewed and asymmetrical would be common in real estate markets.
The shape of the curve looks like a bell. The top or high point of the bell is the most probable point. The width of the shape measures the variation or deviation from the center of the bell. Figure 1 illustrates a standard normal distribution with a mean of zero and a standard deviation of one.
Variation can be expressed in standard deviations. The greater the variation the larger the standard deviation. This can be observed in Figure 2. One can find the area between two points under a normal distribution curve with Excel or tables published in introductory statistics books.
With a normal bell curve the highest point is the mean, the mode, and the median. In this distribution the most probable price is the highest point on the curve. Thinking about a bell curve allows an appraiser to consider both the estimate of value and the estimate of variation.
Consider two appraisals that both report a value of $1,000,000. Are the subject properties identical in terms of risk? In valuing the first property, the appraiser estimated a standard deviation of $100,000 and assumed a normal bell-shaped curve distribution. The value of a standard deviation estimate is that it allows the analyst to estimate probabilities of a different value. Note that the downside risk is of primary interest to lenders that rely on appraisal reports in financing properties.
Consider a drop in value of 10% or a value of $900,000 or less. What is the probability that this would occur? One approach would be to measure the area under the curve to the left of $900,000. This area is 15.9% of the total area under the curve. One could interpret this as a 15.9% probability. How about a value of less than $800,000? This is two standard deviations below the value estimate. The area to the left of $800,000 is only around 2.3% of the curve. The distribution of these probabilities would look like the shaded areas of Figure 3.
Consider a second appraisal with a market value estimate of $ 1,000,000 and a standard deviation of $200,000 as illustrated in Figure 4. You will note that both distributions have the same mode, median, and mean. The appraiser would report the same market value for both distributions. But is there something different about the confidence of the estimate?
The appraiser should have much less confidence in the second distribution. To see this, consider the probability once again of a value dropping below $900,000. In the case of the first distribution (Example A, Figure 3) the probability was 15.9%. In the second case (Example B, Figure 4), the probability increases to 30.9%, or almost double the risk. What about a value drop to $800,000? The first distribution indicated 2.3%, but the second distribution indicated 15.9%, or almost seven times greater. This is seen in Figure 5 when we overlay of the two distributions.
Calculating a Confidence Interval
In some cases appraisers are asked to estimate a confidence interval. This is simply extending the range on either side of the mean. Assume an appraiser is asked to estimate value with a 90% confidence. This can be accomplished by measuring 45% on either side of the mean. To find the range, an appraiser could refer to a z-score table from a basic statistics book or use Excel. Based on these sources, 45% of the curve is [+ or -]1.645 standard deviations. Figure 6 illustrates this concept for the two distributions, and Table 1 summarizes the analysis of these two distributions.
How Can Appraisers Estimate and Report Confidence?
Normal curve analysis generally requires a random sample of unadjusted data. Appraisers don't usually do this. Nonetheless, it should not prevent us from using the concept to more deeply understand and explain the confidence we have in our estimates.
One Price, Two Values
Remember that in each sale there is one price but two values. If the seller's lowest value and the buyer's highest value overlap, a value transaction zone is created and a sale is possible. The price moves within this value range. Value is created in the transaction for both the buyer and the seller and allows them to walk away after the transaction better off. If this were not true, a sale would never occur.
The problem is that buyers and sellers do not typically disclose their values. We know prices, not values. All we know is that the price fell within the transaction zone, or the zone where buyer and seller values overlapped. (8) Once again, what most clients are interested in is the transaction value range of market participants. A point estimate is interesting, but the value range provides more valuable decision-making information.
A somewhat common request is that an appraiser estimate confidence on a range of plus or minus 10%. This can be accomplished in three steps. The first step is to simply multiply the point estimate of value by 10%. You will add and subtract this factor to and from the point estimate of value to calculate the range.
The second step would be to approximate the standard deviation. A quick approximation for the standard deviation can be made using the range rule. (9) The range rule tells us that the standard deviation of a sample is approximately equal to one fourth of the range of the data. In other words,
Standard Deviation = (Maximum - Minimum)/4.
In this case the minimum and maximum value indications would be used in the formula. This rule works well when the data comes from a normal distribution and the sample size is at least 30. These two assumptions are generally not met under typical appraisals. To compensate, we would lower the denominator to two, so the formula would be RANGE/2.
This also happens to be close to what you would get if you calculated the standard deviation on three values indications. Use the simple range rule by dividing the range by two, or use the STDEV.S function in Excel to calculate an approximation of the standard deviation.
The third step is to calculate the confidence level of the area under the curve between the value range. This is easily accomplished using a z-score table or the NORM.DIST function in Excel.
A z-score table shows the percentage of the total area under the curve indexed by the number of standard deviations from mean. It is very useful for your analysis. It allows you to know the area under the curve if you know the standard deviations, or if you know the area under the curve you can find the number of standard deviations.
The NORM.DIST function in Excel calculates the area under the probability distribution curve and requires four arguments: x, mean, st_dev, and cumulative. The x in this case is the point estimate of value plus 10%. The mean is the point estimate of value. The st_dev is the approximate standard deviation calculated in the second step. Cumulative requires a logical operator of true.
This function will return the total area under the curve up to the x value. There are two additional adjustments that need to be made to calculate the area under the plus or minus 10% range of the curve. The first is to subtract the whole area to the left of the point estimate. Under a normal distribution this is 50% or 0.5. Now we have the area from the point estimate up to the plus 10% value. To include the minus 10% area we can simply multiply by two. The complete equation would be as follows:
=(NORM.DIST(x, mean, st_dev, cumulative)-0.5)*2
The appraiser could enter the values directly into this formula or refer to cells. Referring to cells is more powerful because it allows you to observe sensitivity. Cell descriptions for point estimates are shown in Table 2.
Look back at the first example in the normal distribution where the value was $1,000,000 and the standard deviation estimate was $100,000. In that example the standard deviation happens to also be 10% of point estimate of value. One standard deviation from the mean or center of a normal distribution includes around 34% of the area under the curve. One standard deviation on both sides of the mean would contain around 68% of the area under curve. Based on this analysis, the appraiser could report a confidence level of 68% that the value range was between $900,000 and $1,100,000.
The second example also had a value of $1,000,000, but the standard deviation was $200,000. Dividing the 10% factor of $100,000 by the standard deviation of $200,000 suggests that the range is plus or minus one-half a standard deviation. The z-score table indicates that this would contain 19% on each side or 38% of the area under the curve. The appraiser would report a confidence level of 38% that the value range was between $900,000 and $1,100,000.
The third example, with a value of $1,000,000 but the standard deviation of $50,000, would suggest a range of two standard deviations on either side of the mean. The z-score table indicates that this would contain 48% on each side or 96% of total area under the curve. The appraiser would report a confidence level of 96% that the value range was between $900,000 and $1,100,000.
We can see the problem if an appraiser reports only the most probable price without also discussing confidence. There really is a big difference between 38% and 95% confidence.
Although this method lacks the mathematical rigor of traditional statistical analysis, it does serve the purpose of applying the logic of recognizing and reporting variation. Appraisers could also provide more estimates if required by the client. For example, a client might need to know the confidence level for a tighter range or the range from a higher confidence level.
It is perfectly reasonable for a client to ask two questions: what is the confidence level of the value range plus or minus 10% from the point estimate, and what is the value range at a 90% confidence level? Appraisers should be able to answer both of these questions using this simple method. In one case, the range is used to estimate confidence; in the other case, confidence is used to estimate the range.
Appraisers develop a level of confidence in their opinions as they move through the appraisal process. Kummerow notes, "Making clients aware of the inherent variability of prices could reduce professional indemnity exposures." (10) How to measure and report confidence is still subjective.
This article suggests a possible method: assume a normal distribution and the indication range divided by two is one standard deviation. This assumption should be stated before confidence intervals are presented. Expect the development of new methods of measuring confidence as the probabilistic nature of valuation is more deeply understood. Appraisers need to evolve a new species of statistics. This process will better serve the needs of our clients while improving communication and reducing liability.
Dell, George. "Common Statistical Errors and Mistakes: Valuation and Reliability." The Appraisal Journal (Fall 2013): 332-347.
Rummerow, Max. "Thinking Statistically About Valuations." The Appraisal Journal (July 2000): 318-326.
Rummerow, Max. "A Statistical Definition of Value." The Appraisal Journal (October 2002): 407-416.
Rummerow, Max. "Protocols for Valuations." The Appraisal Journal (Fall 2006): 358-366.
Internet resources suggested by the Y T. and Louise Lee Lum Library
Dell Software--Standard Distribution Tables
ExcelFunctions.net--Excel NORMDIST Function
ExcelUser, Inc.--Introduction to Excel's Normal Distribution Functions
(1.) Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable (New York: Random House, 2007).
(2.) Max Kummerow, "Thinking Statistically About Valuations," The Appraisal Journal (July 2000): 320.
(3.) Marvin L. Wolverton, letter to the editor, The Appraisal Journal (Spring 2014): 176.
(4.) Matthew C. Trimble, letter to the editor, The Appraisal Journal (Spring 2014): 173.
(5.) Appraisal Institute, The Appraisal of Real Estate, 14th ed. (Chicago: Appraisal Institute, 2013).
(6.) Marvin L. Wolverton, An Introduction to Statistics for Appraisers (Chicago: Appraisal Institute. 2009).
(7.) Teddy Seidenfeld, Philosophical Problems of Statistical Inference: Learning from R.A. Fisher (London: D. Reidel Publishing Co., 1979).
(8.) Richard U. Ratcliff, Valuation for Real Estate Decisions (Santa Cruz, CA: Democrat Press, 1972).
(9.) David F. Groebner and Patrick W. Shannon, Business Statistics: A Decision Making Approach, 2nd ed. (Columbus, OH: Charles E. Merrill, 1985), 97.
(10.) Kummerow, "Thinking Statistically About Valuations," 318.
by Gale L. Pooley, PhD, MAI, SRA
Gale L. Pooley, PhD, MAI, SRA, is president of Analytix Appraisal Group and has more than twenty-five years of professional real estate valuation and research experience. He has served on the faculty of several universities, and he currently teaches at Alfaisal University in Riyadh, Saudi Arabia. Pooley holds a BBA in economics from Boise State University and a PhD in economics and education from the University of Idaho.
Table 1 Scenario Indications with Market Value of $1,000,000 Confidence Standard 10% Decline 10% Decline Deviation Probabilty Probabilty [less than or equal [less than or to] $900,000 equal to] $800,000 High $100,000 15.9% 2.3% Moderate $200,000 30.9% 15.9% Ratios 2.00 1.94 6.91 90% Confidence Interval Confidence Lower Upper Range High $835,500 $1,164,500 $329,000 Moderate $671,000 $1,329,000 $658,000 Ratios Table 2 Cell Descriptions for Point Estimates Description Type Calculation Short Name Point Estimate Value PE Point Estimate +10% Calculated PE x 1.1 PEP Point Estimate -10% Calculated PE x 0.9 PEM Highest Indication Value HI Lowest Indication Value LI Range Calculated HI-LI RG Standard Deviation Calculated RG/2 SD Confidence Calculated See above CF
|Printer friendly Cite/link Email Feedback|
|Author:||Pooley, Gale L.|
|Date:||Jun 22, 2015|
|Previous Article:||Asset allocations: are you reconciling?|
|Next Article:||What's so special about special-purpose property?|