Managing value creation within the firm: an examination of multiple performance measures.
Keywords: balanced scorecard; performance measures; nonfinancial measures.
This paper investigates the relations that exist among multiple performance measures to determine how they provide information about the creation of firm value. We address this issue using the framework of one common performance measurement system, the balanced scorecard (BSC). We ask two related research questions. First, are the performance measures directly related to each other or are the relations indirect and mediated by intervening measures? Second, do the relations among measures change depending on whether the performance measurement system uses both financial and nonfinancial measures versus only financial measures?
Understanding how value is created within the firm through the identification, measurement, and management of the drivers of long-term shareholder value has been a recent focus of accounting research (Ittner and Larcker 2001). Positive cash flows and operating profits are the result of managerial actions, such as upgrading employee skills or implementing programs to improve customer satisfaction. Managerial actions and decisions can directly lead to higher operating profits or they can indirectly lead to higher operating profits through their impact on other areas of operations. For example, if a manager purchases new equipment, which increases employee productivity, profit may be affected in a number of ways. Profits could increase indirectly through increased production, which could lead to increased market share and therefore higher revenues. Profits could also increase directly through lower operating costs. In discussing value drivers, McKinsey & Company consultants Copeland et al. (2000, 71) state:
The important point is that companies should be as concerned about how a business achieves its financial results as about whether it meets its financial targets. Value drivers help companies to understand the reasons for their current performance and how their future performance will likely develop.
To understand value drivers, managers must have in place performance measurement systems designed to capture information on all aspects of the business, not just the financial results. A performance measurement system comprised of multiple measures allows managers to better monitor employees' actions and guide firm behavior (Antic and Demski 1988). Multiple measures also provide better information on changes in the economy and competition (Lev 2001; Banker et al. 2000; Balkcom et al. 1997; Chow et al. 1997). For example, firms are increasingly competing with intangible assets that are not captured by traditional, financial indicators (Wallman 1995) and, therefore, are including multiple performance indicators in their management control system.
The BSC is a popular performance measurement system that uses multiple measures (Kaplan and Norton 2001 a, 2001b). Firms implement a BSC by selecting both financial and nonfinancial measures across four hierarchical perspectives: learning and growth, internal business processes, customer, and financial (see illustration in Figure 1). The lowest level in the hierarchy is the learning and growth perspective because actions taken there, such as training employees, affect outcomes of the other perspectives at a future date. (1) The financial perspective is considered the highest-level perspective.
[FIGURE 1 OMITTED]
Within each perspective, firms select both generic outcome measures that gauge the results of actions taken and unique measures that provide information about the cause or the driver of the outcome. The unique measures are tailored to the firm's competitive strategy and assist the manager in guiding the firm in accordance with its overall strategy and mission. In contrast, generic outcome measures are common across organizations. Kaplan and Norton (1996, 43) state that generic measures are those that "show up in most organizations' scorecards" and include measures such as return on investment, customer satisfaction, market share, and new product or service introductions. Specifically, Kaplan and Norton (1996, 149) state that:
all Balanced Scorecards use certain generic measures. These generic measures tend to be core outcome measures, which reflect the common goals of many strategies, as well as similar structures across industries and companies.
We collect data for 125 firms over five years on proxies for seven generic outcome measures within the four BSC perspectives: employee skills, new product and service introductions, customer satisfaction, market share, revenue, operating costs, and profitability. Our first research question focuses on understanding how performance measures capture value creation within firms. The BSC describes a series of causal relations both within and between four hierarchical perspectives that culminate in the achievement of financial objectives. This can be described as a relatively simple value-creation process where each perspective contains outcome measures that are drivers of only the next perspective in the hierarchy (Hilton et al. 2003, 486; Kaplan and Norton 1996, 31). Alternatively, the BSC has also been described as a complex value-creation process where outcome measures in lower-level perspectives influence outcome measures in all higher-level perspectives (Kaplan and Norton 2001b, 61).
Using structural equation models, we investigate whether value creation appears to be more consistent with a simpler process beginning with the employee and culminating with financial outcomes versus a more complex process in which value results from the simultaneous interplay of employees, internal business processes, customers, and financial outcomes. We find that a model that allows associations between outcome measures of each BSC perspective and outcome measures of all higher-level perspectives best fits the data. That is, the results are consistent with the more complex process described above. In turn, the results also suggest that the relatively simple process that begins with the learning and growth perspective and culminates with financial outcomes is too simplistic. A manager should simultaneously consider outcome measures for all BSC perspectives and should not assume that the higher-level perspectives subsume the outcomes of lower-level perspectives.
Our second research question investigates whether the design of firms' performance measurement systems influences the relations among the generic outcome measures. Using disclosures in the sample firms' proxy statements, we classify firms' performance measurement systems based on whether they use financial performance measures alone versus using a combination of financial and nonfinancial measures (hereafter, mixed measures). We expect that linking mixed measures to compensation focuses managers' attention on those measures, which results in a positive relation between the nonfinancial outcome measures of the lower-level BSC perspectives and the outcome measures of the financial perspective. We find that whether firms use solely financial measures versus mixed measures influences many of the relations among the outcome measures. Most importantly, as expected, we find a positive relation between both customer satisfaction and new product or service introductions and financial outcome measures only when the firm uses a performance measurement system comprised of mixed measures.
This study makes several contributions to the existing literature. First, accounting studies often focus on the association between a single performance measure and a financial outcome measure. For example, Ittner and Larcker (1998) and Foster and Gupta (1999) investigate the relation between customer satisfaction and firm performance. In addition, as we discuss in a subsequent section, numerous studies from other fields have examined contemporaneous relations between two individual performance measures, but have not considered multiple measures simultaneously. As discussed by Ittner and Larcker (2001), prior studies, both in accounting and other disciplines, that focus on single measures ignore trade-offs among multiple measures, which can result in misleading inferences. We overcome this limitation by using cross-sectional data, which provides a large enough sample to allow us to investigate the relations among multiple measures simultaneously. We are able to provide evidence about how the measures work together to capture value creation and about the incremental information content of specific financial and nonfinancial measures, after controlling for size and industry.
Second, prior accounting studies have investigated the BSC framework using data for a single organization or industry. For example, Malina and Selto (2002) investigate one manufacturing firm's actual implementation of its performance measurement model (similar to a B SC), Banker et al. (2001) examine the relations among employee satisfaction, customer satisfaction, and future performance for one retail chain, and Smith and Wright (2004) investigate product value attributes, customer loyalty, and financial performance in the personal computer industry. Our use of cross-sectional, readily available data (as opposed to data from firms' actual BSCs) allows for increased generalizability of the results across similar firms. (2) In addition, because we do not focus on a single firm that has implemented a BSC, the results are generalizable regardless of the type of performance measurement system firms have in place. Since the measures evaluated in the study are common outcome measures that are fundamental to the success of many firms, the study provides information to managers who are contemplating implementing any performance measurement system based on multiple measures, such as the BSC (Kaplan and Norton 2001 a), the service profit chain (Heskett et al. 1994; Heskett et al. 1997), or the action-profit linkage model (Westbrook et al. 2000).
Third, this study contributes to the literature investigating the benefits of various performance measurement systems. Ittner and Larcker (2001, 375) note that the "performance effects of the balanced scorecard and other value driver techniques remain open issues." Use of cross-sectional data allows us to divide the sample based on the design of firms' performance measurement systems and to examine differences in the relations among performance measures for firms that focus solely on financial measures versus firms that focus on mixed measures. We find positive relations between two important nonfinancial measures, product introductions and customer satisfaction, and financial outcomes only when the firm has a mixed performance measurement system in place.
The remainder of this paper is organized as follows. We first discuss the BSC framework and the performance measures analyzed in this study. Second, we describe the empirical models used in this study and discuss our expectations based on prior literature. Third, we describe our sample and variable measurement. We then present the results related to our research questions, as well as sensitivity analyses. Finally, we provide concluding comments.
BALANCED SCORECARD FRAMEWORK
Although the BSC is designed to translate the firm's strategy and mission into measures that managers can use to manage the organization, BSCs contain both generic measures that are common across organizations and unique measures that are tailored to the firm's competitive strategy. Kaplan and Norton (1996, 43) describe generic measures as those that "show up in most organizations' scorecards" and include return on investment, customer satisfaction, market share, and new product introductions. As depicted in Figure 1, these measures typically fall under four hierarchical perspectives: learning and growth, internal business processes, customer, and financial. When building strategy maps, Kaplan and Norton (2001b) suggest that cause-and-effect relations exist among perspectives. That is, an outcome measure of a lower-level perspective may be an indicator or predictor of an outcome measure of a higher-level perspective. For example, introducing new products improves customer satisfaction, which leads to increased revenue.
A growing stream of literature provides evidence that even when managers collect and track unique measures, they still place primary reliance on traditional generic (often financial) measures (e.g., Lipe and Salterio 2000; Stivers et al. 1998; McNair et al. 1990). This reliance on generic measures increases the relevance of a cross-sectional study that examines the relations underlying common, generic outcome measures in a BSC framework. Accordingly, we focus our analysis on a set of outcome measures that are employed by a wide range of firms and measured consistently by many of these firms. While these generic outcome measures are common to most firms, they are not the only possible outcome measures a firm could use. In the "Results" section, we discuss sensitivity analyses using alternative measures.
Learning and Growth Perspective
As illustrated in Figure 1, the BSC describes a series of causal relations both within and between four hierarchical perspectives that culminate in the achievement of financial objectives. The first level in the hierarchy of perspectives is the learning and growth perspective. Outcome measures of the learning and growth perspective become indicators of the outcomes of each of the three perspectives above it in the hierarchy. In the learning and growth perspective "managers define the employee capabilities and skills, technology and corporate climate needed to support a strategy" (Kaplan and Norton 2001a, 94). On average, employees with higher skills and knowledge are compensated with higher salaries and employee benefits (Milkovich and Newman 2002). Companies typically offer benefits packages designed to foster long-term relationships with skilled employees, such as retirement benefits. Therefore, we use retirement cost per current employee (3) to proxy for the level of employee skills (SKILL). While this measure is not an actual measure that a company would include in a performance measurement system, we consider it to be a reasonable proxy for the strategic skills coverage ratio, a common performance measure defined as the number of employees with the skills for specific strategic jobs relative to anticipated organizational needs (Kaplan and Norton 1996, 133). Other common outcome measures include employee satisfaction, employee retention, and employee productivity (Niven 2002, 140; AICPA and Maisel 2001; Ittner et al. 1997; Kaplan and Norton 1996, 129). (4)
Internal Business Process Perspective
The second level in the BSC hierarchy is the internal business process perspective. A generic view of the internal business process perspective encompasses the entire internal value chain, which Kaplan and Norton (2001a) decompose into four processes common to all firms: innovation, customer management, operational, and regulatory and environmental. For the innovation process, managers define measures that capture development of new products and services and market penetration. In the customer management process, managers define measures that capture creation of customer value. For operations, managers define measures that show whether the company has achieved operational excellence through improving supply chain management, resource management, asset utilization, and other internal processes. Finally, measures are used to determine whether the firm is a good corporate citizen in the regulatory and environmental process. The outcomes of the internal business process perspective facilitate achievement of customer objectives.
Kaplan and Norton (1996) suggest that a generic outcome measure found across various organizational scorecards is the number of new products and services introduced (INTRO), which we use as a proxy for the innovation component of the internal business process. This is a measure commonly used in performance measurement systems. A recent AICPA survey of performance measurement practices (AICPA and Maisel 2001) showed that 22 percent of the firms surveyed use a measure of new product development. Other potential generic outcome measures are operating process quality and cycle time (Niven 2002, 131; AICPA and Maisel 2001; Ittner et al. 1997; Kaplan and Norton 1996, 100-107); however, these measures are difficult to apply consistently across a large cross-section of firms. (5)
The third level in the BSC hierarchy is the customer perspective, which focuses organizations on the external environment and allows firms to understand, discover, and emphasize customer needs. The customer perspective identifies outcome measures that will facilitate the achievement of the organization's financial objectives (Kaplan and Norton 1996).
We use customer satisfaction (CUST) and market share (MKTSH) to proxy for outcome measures of the customer perspective. We chose these measures because firms commonly use them to monitor performance and determine compensation. For example, Ittner et al. (1997) find that 37 percent of the firms using nonfinancial measures in their executive bonus contracts include customer satisfaction measures and 11 percent use market share measures. In addition, the AICPA and Maisel (2001) survey of performance measurement systems found that 70 percent of the firms use a customer satisfaction measure and 42 percent of the firms use a market share measure. Other potential generic measures of the customer perspective include customer acquisition, customer retention, and customer profitability (Niven 2002, 127; Kaplan and Norton 1996, 67). We focus our analysis on customer satisfaction since it is necessary to retain customers and eventually achieve customer profitability. Kaplan and Norton (1996, 68) suggest that customer satisfaction is both an outcome of underlying drivers and, in turn, a driver of another generic outcome measure of the customer perspective, market share. Thus, we include both measures in our study.
The highest level in the hierarchy is the financial perspective. The financial perspective contains outcome measures that result from achievement of objectives in the lower perspectives. Companies improve shareholder value through a revenue growth strategy, a productivity strategy or a mix of the two (Kaplan and Norton 2001a). A productivity strategy is implemented through cost structure improvement or more efficient asset utilization. We use revenues (REV), operating costs (COST), and profitability (PROFIT) to capture these common financial themes. (6)
RESEARCH QUESTIONS AND MODELS
Question 1: Simple or Complex Value-Creation Process
Our first research question asks whether relatively simple relations among BSC performance measures or more complex relations appear to be more consistent with the value-creation process of most firms. This question seeks to understand how performance measurement systems capture value creation within the firm in the current year. (7) We first investigate a relatively simple, fully mediated model. A mediator variable transmits effects from one variable to another. In a fully mediated model, all associations between outcome measures in lower-level perspectives and outcomes in the financial perspective are mediated by the outcome measures in the intervening perspectives in the hierarchy. Kaplan and Norton (2001b, 76) describe the BSC as forming a series of causal relations, both within and between perspectives, which culminate in achievement of financial objectives. This relatively simple representation describes each perspective as containing outcome measures that are drivers of only the next perspective in the hierarchy. We then investigate a more complex, partially mediated model. In a partially mediated model, we relax the assumption that each outcome measure is only a driver of the next perspective and instead allow each outcome measure to be associated with outcome measures in all of the higher-order perspectives.
Fully Mediated Model
Kaplan and Norton (1996, 31) describe the fully mediated model by saying "We can now see how an entire chain of cause-and-effect relationships can be established as a vertical vector through the four BSC perspectives." Management accounting textbooks also use this representation of the BSC (e.g., see, Hilton et al. 2003, 486). In addition, other popular representations of performance measurement systems depict similar frameworks. For example, the service profit chain asserts that employee satisfaction influences customer satisfaction, which influences financial outcomes (Heskett et al. 1997), and the action-profit linkage chain asserts that managerial actions influence outcomes, which influence profits (Westbrook et al. 2000). Thus, we begin our analysis by examining the extent to which a relatively simple, fully mediated model captures the value-creation process within the firm. We use the following structural equation model:
(1) INTR[O.sub.t] = [a.sub.0] + [a.sub.1]SKIL[L.sub.t] + [[epsilon].sub.1]; CUS[T.sub.t] = [b.sub.0] + [b.sub.1]INTR[O.sub.t] + [[epsilon].sub.2]; MKTS[H.sub.t] = [c.sub.0] + [c.sub.1]CUS[T.sub.t] + [[epsilon].sub.3]; FI[N.sub.t] = [d.sub.0] + [d.sub.1]MKTS[H.sub.t] + [[epsilon].sub.4];
Partially Mediated Model
The relatively simple, fully mediated model may not be robust enough to reflect the actual value-producing activities of some firms to the extent that outcome measures in lower-level perspectives influence outcome measures in all higher-level perspectives. For example, in Mobil's BSC, outcome measures of the internal business process perspective directly influence not only the customer perspective, but also the financial perspective (Kaplan and Norton 2001b, 61). Empirical evidence suggests that, consistent with a partially mediated model, there are multiple relations among the outcome measures of the BSC perspectives. For each perspective, we first discuss this evidence and then discuss how we expect the structural equation model to change.
Previous research suggests the existence of a relation between employee skills and outcome measures of higher-level BSC perspectives. Theoretical papers in the economics literature find that the supply of skilled labor (Acemoglu 2002) and high and growing wages (Bester and Petrakis 2003) lead to innovative activity. Doms et al. (1997) support this link empirically by showing that companies that adopt new factory technologies have highly skilled work forces and higher employee productivity before the adoption, consistent with highly skilled, productive employees leading to more innovation. In addition, the service profit chain links employee skills to customer outcome measures (Heskett et al. 1994). Finally, the management literature has documented a positive relation between financial outcomes and organizational work structures related to employee involvement, recruiting, and retention of highly skilled employees (Huselid 1995). In addition, employee training effectiveness is positively related to both perceived firm performance (Delaney and Huselid 1996) and retail store sales volume (Russell et al. 1985). Based on these findings, we expect that positive relations exist between employee skills and each of the outcome measures of the higher-level perspectives.
Previous research also suggests the existence of a relation between product introductions and outcome measures of higher-level BSC perspectives. Rust et al. (2000) suggest that there is a "product death spiral," where a lack of innovation leads to the death of a firm because customers are not satisfied and will cease making purchases unless there is a broad line of products or services available. In contrast, new products and services fill consumers' unfulfilled desires and needs, thus enticing consumers to make more purchases and increasing firms' market share. Empirical studies investigating new product introductions, product line breadth, and product proliferation find a direct and positive relation with market share, thus supporting the theoretical links between new product introductions and customer outcome measures (Kekre and Srinivasan 2002; Bayus and Putsis 1999; Banbury and Mitchell 1995). Finally, Kekre and Srinivasan (2002) find that a broader product line is associated with increased selling prices and thus increased revenues, while Bayus and Putsis (1999) find that broadening a product line also increases operating costs due to increased complexity and loss of production benefits from diluting economies of scale. However, new products need less marketing support than older products, thereby reducing selling, general and administrative costs and increasing firm profits (Bayus et al. 2003). Overall, studies have found that new product introductions affect both revenues and costs and, therefore, it is not surprising that evidence on the relation with profitability is mixed (Kekre and Srinivasan 2002; Bayus and Putsis 1999). Based on these findings, we expect that a positive relation exists between new product introductions and the outcome measures of the customer perspective, along with revenues and costs. We are uncertain as to whether a relation with profitability exists.
Finally, studies have found a positive, contemporaneous relationship between customer satisfaction and both revenues and costs (Behn and Riley 1999; Anderson et al. 1994; Rust and Zahorik 1993). However, the evidence is mixed as to whether customer satisfaction is a leading indicator of profitability, primarily due to increased costs required to achieve a high level of customer satisfaction (Banker et al. 2000; Foster and Gupta 1999; Ittner and Larcker 1998). Prior literature (see, e.g., Behn and Riley 1999; Szymanski et al. 1993; Capon et al. 1990) has also found a positive relation between market share and financial outcomes. Therefore, we expect to find a positive relation between the customer perspective outcome measures and revenue and cost measures. We are uncertain as to whether a relation with profitability exists.
We expand the earlier fully mediated model to include the relations identified in previous studies. This results in the following partially mediated structural equation model:
(2) INTR[O.sub.t] = [a.sub.0] + [a.sub.1]SKIL[L.sub.t] + [[epsilon].sub.1]; CUS[T.sub.t] = [b.sub.0] + [b.sub.1]SKIL[L.sub.t] + [b.sub.2]INTR[O.sub.t] + [[epsilon].sub.2]; MKTS[H.sub.t] = [c.sub.0] + [c.sub.1]SKIL[L.sub.t] + [c.sub.2]INTR[O.sub.t] + [c.sub.3]CUS[T.sub.t] + [[epsilon].sub.3]; FI[N.sub.t] = [d.sub.0] + [d.sub.1]SKIL[L.sub.t] + [d.sub.2]INTR[O.sub.t] + [d.sub.3]CUS[T.sub.t] + [d.sub.4][KTS[H.sub.t] + [[epsilon].sub.3];
Research Question 2: Moderating Effects of the Performance Measurement System
Our second research question asks if relations among the performance measures depend on whether the firm's performance measurement system includes only financial measures or both financial and nonfinancial measures. Prior research on performance measurement systems has focused on whether the type of performance measurement system impacts firm performance. This study focuses on how performance measurement systems impact firm performance. Specifically, we investigate whether there is a positive relation between the nonfinancial outcome measures of the lower-level BSC perspectives and the outcome measures of the financial perspective when performance measurement systems contain nonfinancial measures.
While no studies have specifically examined how relations among performance measures vary for firms with different performance measurement systems, several studies have found mixed results concerning the performance implications of alternative performance measurement systems. Ittner et al. (2003) find that financial services firms that use a broad set of financial and nonfinancial measures more extensively than other financial services firms earn higher stock returns, but do not achieve a higher return on assets or greater sales growth. They also find no evidence of higher accounting or stock market performance for firms using the BSC. In contrast, Hoque and James (2000) find that for a sample of 66 Australian firms, greater BSC use is associated with improved performance. Likewise, Said et al. (2003) find that the use of nonfinancial measures is associated with future accounting and market based returns. In addition, Davis and Albright (2003) find an improvement in internal performance measures alter implementation of a BSC program at certain branches of one bank. These studies suggest that firm performance may be influenced by the type of performance measurement system a firm implements, but they do not identify how performance is affected.
Extant literature suggests that a focus on mixed measures affects performance through a shift in managers' behavior. A few recent performance measurement systems are based on this concept. For example, the service profit chain posits that profits are improved when managers actively monitor multiple aspects of the business, including customer loyalty, satisfaction, and value (Heskett et al. 1997). In addition, one perceived benefit of the BSC is that it focuses managers' attention on a mix of measures, thus discouraging managers from improving one area of operations at the expense of another.
Agency theory suggests that a compensation contract based on noncongruent measures results in suboptimal effort allocation across tasks (Hemmer 1996; Feltham and Xie 1994; Bushman and Indjejikian 1993). Gersbach (1998) demonstrates analytically that using a "general control" system with one aggregate measure such as profitability induces low effort across tasks unless the tasks are perfectly equivalent. He finds that the use of "specific control" (e.g., mixed measures) will improve results because managers will focus attention on all tasks. Smith (2002) shows analytically that managers shift their behavior when additional measures are introduced to the performance measurement system. He also shows that managers' behavior is driven by the weights placed on the underlying measures, and demonstrates that suboptimal effort allocation may result if the mixed measures are improperly weighted.
Overall, the literature suggests that when managers are faced with multiple tasks, their behavior will differ depending on whether the performance measurement system consists solely of an aggregated financial outcome measure or includes mixed measures. Managers' effort can be focused across the various tasks by linking the tasks with a mix of appropriate performance measures. Otherwise, managers may choose to pay too little attention to tasks that are not appropriately captured by the performance measurement system of the firm. For example, if the goal of the company is to increase shareholder wealth but managers' compensation is based on earnings, then managers may make short-run decisions, such as expending too little effort on introducing new products or focusing on new products with short-term profit potential without regard for customer satisfaction. However, if the goal is to increase shareholder wealth and compensation is based on multiple, mixed measures, such as new product introductions, customer satisfaction, and earnings, then managers' effort should shift toward the additional measures. Focusing managers' effort on introducing new products that are appealing to customers should lead to positive relations among new product introductions, customer satisfaction, and financial outcomes.
Based on the above discussion, we expect that nonfinancial outcomes in lower-level perspectives will have a positive relation with financial outcomes when a firm implements a performance measurement system that includes a mix of financial and nonfinancial measures. Each perspective can contain financial and/or nonfinancial outcome measures. For example, in the learning and growth perspective, employee satisfaction is a nonfinancial measure, while training dollars per employee is a financial measure. Thus, in our setting, we expect that new product introductions, customer satisfaction, and market share will have a positive relation with the financial outcome measures when firms include nonfinancial measures in their performance measurement system. We have no expectations regarding the learning and growth perspective because our measure of employee skills is based on financial numbers.
To test our second research question, we split the sample into firms that use solely financial measures in top executives' incentive compensation contracts versus firms that use mixed measures. We define a firm as using mixed measures when they base executive compensation on both financial and some type of nonfinancial measure such as innovation or customer satisfaction.
We base our sample on firms included in the American Society for Quality (ASQ) customer satisfaction index (ACSI). In 1994 the ASQ began disclosing an annual indicator of customer satisfaction for companies that sell products or services nationally in more than 30 different industries. The ASQ has not set a minimum sales threshold for inclusion in the ACSI survey; however, selected companies generally represent a major proportion of sales within their two-digit SIC code industry. Although the ACSI survey is limited to larger firms, it employs a uniform methodology. In addition, since the ASQ selects the firms that it includes, the sample is not biased toward firms that self-select to disclose customer satisfaction. We discard insurance firms (8) and require firms to be included in the Compustat database. Our sample consists of 589 firm-years (127 firms) that have the necessary Compustat and ACSI data from 1994 to 1998. We pool our data cross-sectionally and over time. (9) To ensure that outliers do not drive our results, we review the Mahalanobis [D.sup.2] scores and remove 26 firm-years for 11 firms (two firms are completely deleted) from the sample. (10) This results in a final sample size of 563 firm-years (125 firms). Table 1 provides a description of the sample firms.
Overall, our sample consists of large firms. Sample observations have a mean (median) asset base of $33,457 ($13,592) million and mean (median) revenues of $21,294 ($11,862) million. On average, sample firms incur annual retirement costs of $2,298 per employee with a maximum of $9,655 per employee. They introduce 2.01 new products or services per year on average, although the median firm introduces only 1.00 new product or service per year. Sample firms have an average customer satisfaction index of 77.05 on a scale of 0 to 100, with 100 being the maximum possible score. This is similar to the average customer satisfaction of all firms included in the ACSI from 1994 to 1998, which is 76.8.
Population differences exist across our subsamples. As shown in Panels B and C of Table 1, (11) firms that only use financial measures as a basis for executive compensation have a smaller asset base and are more heavily concentrated in wholesale and retail trade, and financial services. Our sample is also more heavily concentrated in food, textile, and chemicals, transportation and utilities, and wholesale and retail trade than is the Compustat population. Panel C indicates that at the one-digit level, there are 39 firms (31.2 percent) in transportation and utilities and 32 firms (25.6 percent) in food, textile, and chemicals. We control for both size and industry in all of our statistical tests.
We test our research questions using the seven generic outcome measures described earlier. These measures proxy for outcomes of the BSC perspectives: employee skills, new product or service introductions, customer satisfaction, market share, revenue, operating expenses, and profitability. These variables are obtained from a variety of sources including Compustat, Lexis/Nexis, and the Internet.
We measure employee skills (SKILL) as pension and retirement costs adjusted for nonservice cost items (12) (Compustat items #43-#295 + #331) divided by the number of employees (Compustat item #29). We collected from Lexis/Nexis (see Chaney et al. 1991) the total number of new and derivative product or service introductions (INTRO) during each sample firm's fiscal year (hereafter product introductions). We reviewed all Wall Street Journal articles between 1994 and 1998 that discussed the sample firms (13) and identified those describing possible new product introductions. Two independent coders determined the number of new products introduced. Intercoder agreement was 90 percent. Customer satisfaction (CUST) is the ACSI score reported annually on the ASQ website (Ittner and Larcker 1998; Fornell et al. 1996). The University of Michigan, CFI Group, and the American Society for Quality began producing the ACSI in 1994, based on information collected from a survey of at least 250 customers of each firm. Market share (MKTSH) is the firm's market share divided by market share for the top three firms in the two-digit SIC code. (14) The three financial outcome measures are: revenue (REV) (Compustat item #12); operating costs (COST), defined as revenue less operating income before depreciation (Compustat items #12-#13); (15) and profitability (PROFIT), defined as net income before extraordinary items (Compustat item #18).
We control for two exogenous factors that could impact the relation between the variables and may influence financial performance: industry-specific trends and size. We control for industry effects by adjusting all variables, except market share, by the industry mean for each year within our sample of firms. (16) By definition, relative market share is already relative to the industry. We control for size effects by deflating all industry-adjusted variables by total assets. (17) Finally, we address our research questions using structural equation models estimated using maximum likelihood. (18) This statistical technique relies on multivariate normality. Because our data is skewed and has higher than desired kurtosis, we transform the variables to achieve a more normal distribution (Kline 1998). (19)
Our sample is comprised of a cross-section of firms that may or may not use the BSC or some similar system of multiple performance measures. To test our second research question, we classify firms as having a performance measurement system that is based on either financial performance measures or a mix of financial and nonfinancial performance measures. For each firm, we read the compensation committee report in the annual proxy statement. We code the firm as 0 if the firm stated that they rely on financial measures in the compensation of top executives and as 1 if they indicate that they use some type of nonfinancial measure such as innovation or customer satisfaction along with traditional financial measures.
Simple or Complex Value-Creation Process
Our first research question asks whether there is a relatively simple relation among BSC performance measures or whether the relations are more complex. We investigate this research question by evaluating the two alternative structural equation models (SEM) described earlier. (20) In addition to testing whether specific relations between variables are significant, SEM provides an evaluation of the entire model. Thus, we can focus the analysis on the overall performance measurement system structure and address the question of which type of system (fully mediated or partially mediated) is more consistent with the data (Kline 1998, 13). A relatively simple relation among BSC performance measures is represented by a fully mediated model, which specifies that the outcome measure of each BSC perspective is associated with the outcome measure of the next perspective in the hierarchy, but not with the outcomes beyond the next perspective. We estimate this model separately for each outcome measure of the financial perspective: REV, COST, and PROFIT. Using all three of the financial outcome variables allows us to determine whether the relation is with revenue, cost, profit, or some combination of these measures. For example, an increase in customer satisfaction could be associated with both higher revenues and costs, but, if the revenues increase more than costs, this would also yield a positive association with profits. The coefficients on all paths between SKILL, INTRO, CUST, and MKTSH are the same whether REV, COST, or PROFIT is the financial outcome measure. Therefore, we present these coefficients only once. This methodology is consistent for all results presented in all of the tables. The results of estimating the fully mediated model are reported in Table 2. Although all of the paths are positive and significant, the overall model fit is poor across all three of the financial outcome models ([chi square] > 400, p < 0.01, CFI < 0.71, GFI < 0.83, RMSEA > 0.34). This conclusion is based on the fact that indicators of a good model fit (Kline 1998) would include an insignificant [chi square] statistic, a RMSEA value less than 0.10, and CFI and GFI values greater than 0.90.
In contrast to the fully mediated model, the more complex, partially mediated model specifies that the outcome measure of each BSC perspective is associated with the outcome measures of all higher-level perspectives. A model that includes direct links between all outcome measures is fully saturated since it leaves no degrees of freedom. This type of model is theoretically uninteresting because, by definition, a saturated model has a perfect fit (Kline 1998). We therefore restrict the path from product introductions to market share to have zero value, which provides a degree of freedom and allows us to statistically test the two competing models. (21) We present our results in Table 3. We estimate the model separately for each outcome measure of the financial perspective and present the coefficients on all paths between SKILL, INTRO, CUST, and MKTSH only once. The fit of all three partially mediated models is much improved over the fully mediated models, as evidenced by a [chi square] less than one, RMSEA equal to 0, and a CFI equal to 1. (22) In addition, the [chi square] difference statistic is significant, suggesting that there is a significant difference between the models ([chi square] difference > 400, df= 5, p < 0.01) (Kline 1998). This indicates that a relatively simple representation of the BSC, whereby the outcome measure of each BSC perspective is associated with only the outcome measure of the next perspective in the hierarchy, is over-simplified. Instead, the data are more consistent with value being created through a more complex process where managerial actions have both direct and indirect affects on multiple areas of the business.
The results in Table 3 also indicate that, after controlling for industry, size, and outcomes of the other perspectives, the level of employee skills is positively associated with product introductions (p < 0.01) and customer satisfaction (p < 0.01), but not with market share or financial outcomes. We also find that product introductions are significantly associated with customer satisfaction (p < 0.01) and profit (p < 0.05), but not with market share, revenues, or costs. Looking at the customer perspective, we find that firms with higher customer satisfaction have a higher market share (p < 0.01) and profit-ability (p < 0.05). We also find that firms with a higher market share have higher revenues (p < 0.01) and operating costs (p < 0.01). Figure 2 summarizes the significant paths.
[FIGURE 2 OMITTED]
Table 4 summarizes the direct and indirect effects of the partially mediated model shown in Table 3. These results suggest that managers may want to consider not only indirect, mediated effects, but also direct effects of lower-order outcome measures on higher-order outcome measures. (23) The results suggest that information may be lost when relations are modeled as in the relatively simple, fully mediated model, which ignores many of the direct effects. For example, our proxy for the level of employee skills (SKILL) has a total effect of 0.755 (p < 0.01) on customer satisfaction (CUST), which indicates that increasing the level of employee skills by one standard deviation would increase customer satisfaction by 0.755. However, Table 4 shows that most of the effect of employee skills on customer satisfaction is the result of the direct effect of employee skills on customer satisfaction (0.737, p < 0.01), while the indirect effect of employee skills on customer satisfaction through product introductions is small (0.018, p < 0.05). In addition, product introductions and customer satisfaction both have significant direct effects on profits (p < 0.05 for both). These direct effects are ignored in the relatively simple, fully mediated model. Table 3 shows that our proxy for employee skills does not have a significant effect on market share, revenue, and costs. Likewise, product introductions are not significantly associated with market share, and customer satisfaction is not significantly associated with revenue and costs. However, Table 4 shows that each of these relations has a significant indirect effect. For example, the level of employee skills has an indirect effect on market share, revenue, and costs. These results indicate the importance of incorporating both direct and indirect associations.
Overall, the results indicate that the data is better represented by a partially mediated model that allows lower-level outcome measures to affect higher-level outcome measures both directly and indirectly than by a fully mediated model. The fully mediated model is apparently too simplistic, and to achieve a more complete appreciation of the cause-and-effect relations at work in their firm's value-creation process, managers need to consider the outcome of each BSC perspective simultaneously.
Moderating Effects of the Performance Measurement System
Our second research question asks whether the relations among the outcome measures differ between firms using only financial measures in their performance measurement system versus firms using a mix of financial and nonfinancial measures. Table 5 shows that, after controlling for industry and size, many of the results are similar across the two subsamples of firms. For example, the paths from employee skills to product introductions and customer satisfaction, customer satisfaction to market share, and market share to revenue and cost are significant and positive (p < 0.01) in both subsamples.
However, other results differ across the two subsamples. For firms that focus solely on financial measures, the path from customer satisfaction to revenue and cost is significant and negative (p < 0.05), and the paths from product introductions to all financial outcome measures are insignificant. In contrast, for firms that use both financial and nonfinancial measures, the paths from customer satisfaction to all financial outcome measures and the path from product introductions to profit are positive and significant (p < 0.01). These results suggest that the design of the performance measurement system moderates the relations between financial outcomes and both customer satisfaction and product introductions. As expected, the relations among performance measures differ in firms that focus their managers' attention on multiple aspects of the firm versus those that focus attention solely on financial outcomes. When firms introduce new products or improve customer satisfaction and emphasize both financial and nonfinancial outcomes in their performance measurement systems, they have higher profits. In contrast, firms that compensate managers based on financial measures alone do not realize the full benefits from innovative activities or improved customer relations.
Panel B of Table 5 reports the total standardized effect that employee skills, product introductions, customer satisfaction, and market share have on profit. For firms that use only financial measures, employee skills (0.268, p < 0.01) and customer satisfaction (0.179, p < 0.10) have a significant effect on profit. For firms that use mixed measures, employee skills (0.170, p < 0.01), product introductions (0.249, p < 0.01), and customer satisfaction (0.257, p < 0.01) each have a significant effect on profit. Although customer satisfaction and employee skills have a significant effect in both samples, customer satisfaction has a larger effect for firms using mixed measures, while employee skills has a larger effect for firms using only financial measures. The standardized effects provide evidence that nonfinancial outcome measures have a greater effect on profits when the firm emphasizes both financial and nonfinancial measures versus solely relying on financial measures. This is consistent with the earlier discussion that argued that managers will shift their behavior if the firm focuses on mixed measures. The standardized total effect of market share on profits is insignificant for both sets of firms, suggesting that regardless of whether firms focus on mixed measures or solely on financial measures, the effect of market share on costs offsets the effect of market share on revenues.
Overall, the results indicate that the design of the performance measurement system (as proxied by measures used in executive compensation) moderates the relation among outcome measures. The implication is that if the firm competes through customer satisfaction or product introductions, it appears that the firm will benefit more if the performance measurement system uses both financial and nonfinancial measures.
For the learning and growth and internal business process perspective we use measures with limitations. Therefore, we reestimate the partially mediated model after replacing employee skills (SKILL) with employee productivity (PROD) or compensation per employee (COMP), and replacing product introductions (INTRO) with inventory turnover (INV). When we replace SKILL with PROD, we find that the paths for the learning and growth outcome measure are consistent with those shown in Table 3. When we replace SKILL with COMP, the sample is reduced to 191 observations that report compensation expense. Again, we find that the results using compensation expense are consistent with the base model. Overall, the results shown in Table 3 appear robust to the choice of the learning and growth measure. (24) When we replace INTRO with INV, we find results that are only partially consistent with those shown in Table 3. We find that the paths from the internal business process outcome measure to customer satisfaction are robust whether we use inventory turnover or product introductions. However, we find that the paths from the internal business process outcome measure to market share and the financial outcome measures are sensitive to the choice of variable. The results are not surprising since inventory turnover and product introductions proxy for different components of the internal business process. However, the results do demonstrate that managers should separately consider the components of the internal business process.
SUMMARY AND CONCLUSIONS
Multiple performance measures, value drivers, and the BSC are topics of much interest among accounting researchers and practitioners. Although researchers have argued for the potential of nonfinancial measures to be informative about firm performance, there is limited empirical evidence examining a mix of financial and nonfinancial measures. This is one of the first studies to empirically examine elements of the framework of a performance measurement system designed for use with a mix of financial and nonfinancial measures--the BSC.
Successful implementation of the BSC requires the best possible understanding of the links between the nonfinancial and financial measures. Our first research question compares a relatively simple model versus a more complex representation of the BSC in terms of the competing models' ability to capture how actions that affect employees, internal business processes, and customers create value within the firm. A relatively simple representation of the BSC recognizes that the outcome of each perspective in the hierarchy influences the outcome of the perspective that directly follows it. A more complex representation of the BSC permits the outcome of each perspective to influence the outcomes of all higher-level perspectives. After controlling for industry and size, we find that the more complex representation of the BSC interrelationships fits the data better. For example, higher levels of employee skills are directly associated with both higher levels of product introductions and customer satisfaction.
We also split the sample into firms that use only financial performance measures versus firms that use a mix of financial and nonfinancial measures in their executive compensation contracts. After controlling for industry and size, we find that the performance measurement system moderates many relations within the partially mediated model. Specifically, we find that there is a positive relation between financial outcomes and both customer satisfaction and product introductions. However, this relation holds only for firms that use both financial and nonfinancial measures in their performance measurement system. Firms that focus their managers' attention on multiple aspects of the firm and either introduce new products or improve customer satisfaction have higher profits. These results suggest that firms that compensate managers based on financial measures alone do not realize the full benefits from innovative activities or improved customer relations.
This study has several limitations. We use archival performance measures available beginning in 1994 to proxy for generic outcome measures. Although there is support for the measurement of these variables in the literature, they are only proxies for the outcome of each perspective and obviously contain unwanted noise. For example, we use retirement expense per employee to capture the construct of employee skills. Although we discuss sensitivity analyses that use two alternative measures, salary expense and employee productivity, a better generic outcome measure of learning and growth would be a measure of strategic skills coverage. Furthermore, we use product introductions because of our ability to develop archival measures of this component. Ideally, we would also have outcome measures across all four components of the internal business process.
Two interesting questions that future studies could investigate with different data are, first, whether the relations among measures differ for firms that have implemented (an effective) BSC, and, second, whether findings depend on whether the firm has linked the BSC measures to compensation.
TABLE 1 Descriptive Statistics for the Sample Firms Panel A: Descriptive Statistics Std. Variable Mean Median Dev. Minimum Maximum Employee Skills 2,298 2,133 1,591 0 9,655 (SKILL) (in $) New Product/Service 2.01 1.00 2.88 0 21 Introductions (INTRO) Customer Satisfaction 77.05 77.75 5.74 60 90 (CUST) Relative Market Share 0.17 0.13 0.12 0.01 0.65 (MKTSH) Revenue (REV) 21,294 11,862 27,089 481 160,866 (in $ millions) Operating Cost 17,822 8,930 23,576 208 136,484 (COST) (in $ millions) Net Income 1,148 593 1,691 -3,296 20,065 (PROFIT) (in $millions) Total Assets 33,457 13,592 56,792 816 337,508 (in $ millions) Panel B: Comparison of Sample Medians to the Mixed Measures Subsample Firms with Full Financial Firms with Variable Sample Measures Mixed Measures Employee Skills 2,133 2,902 2,108 (SKILL) (in $) New Product/Service 1.00 1.00 1.00 Introductions (INTRO) Customer Satisfaction 77.75 75.50 78.00 (CUST) Relative Market Share 0.13 0.13 0.14 (MKTSH) Revenue (REV) 11,862 10,295 11,522 (in $ millions) Operating Cost (COST) 8,930 8,700 7,814 (in $ millions) Net Income (PROFIT) 593 502 685 (in $ millions) Total Assets 13,592 11,045 15,178 (in $ millions) Panel C: Industry Classification 1998 SIC SIC Compustat Firms in Sample Population Code Description Number Percent Percent 0 Agricultural 1 0.8 0.37 and forestry 1 Metal and 1 0.8 4.98 construction 2 Food, textile, 32 25.6 12.87 and chemicals 3 Rubber, metal, and 19 15.2 24.51 machine products 4 Transportation 39 31.2 10.43 and utilities 5 Wholesale and 21 16.8 9.29 retail trade 6 Financial services 9 7.2 15.98 7 Hotel and other 3 2.4 16.89 services SIC SIC Firms with Firms with Financial Measures Mixed Measures Code Description Number Percent Number Percent 0 Agricultural 1 2 0 0 and forestry 1 Metal and 0 0 0 0 construction 2 Food, textile, 14 23 13 27 and chemicals 3 Rubber, metal, and 4 6 7 14 machine products 4 Transportation 18 29 21 43 and utilities 5 Wholesale and 16 26 5 10 retail trade 6 Financial services 8 13 1 2 7 Hotel and other 1 2 2 4 services The sample is 563 firm-years with the required hand-collected data over the period 1994-1998. The column headed "Firms with Financial Measures" shows the results for a subset of the sample that use only financial measures in executive compensation (281 firm-years). The column headed "Firms with Mixed Measures" shows the results for a subset of the firms that use a combination of financial and non-financial measures in executive compensation (228 firm-years). SKILL = pension and retirement cost adjusted for non-service cost items (Compustat data items #43--#295 + #331)/number of employees (Compustat data item #29); INTRO = number of new product or service introductions; CUST = customer satisfaction score based on ACSI; MKTSH = relative market share = (revenue/total industry revenue)/market share of the top three firms in the industry; REV = revenue (Compustat data item #12) (in millions); COST = revenue (Compustat data item #12) less operating income before depreciation (Compustat data item #13) (in millions); PROFIT = net income before extraordinary items (Compustat data item #18) (in millions); and Total Assets = book value of total assets (Compustat data item #6) (in millions) TABLE 2 Structural Equation Model of Balanced Scorecard Framework: Fully Mediated Model Panel A: Standardized Coefficients Dependent Independent Standardized Variables Variables Coefficients INTR[O.sub.t] SKIL[L.sub.t] 0.175 *** CUS[T.sub.t] INTR[O.sub.t] 0.224 *** MKTS[H.sub.t] CUS[T.sub.t] 0.786 *** FI[N.sub.t] RE[V.sub.t] MKTS[H.sub.t] 0.711 *** COS[T.sub.t] MKTS[H.sub.t] 0.704 *** PROFI[T.sub.t] MKTS[H.sub.t] 0.265 *** Panel B: Model Fit Statistics REV Model COST Model PROFIT Model [chi square] 411.48 408.91 419.02 p-value 0.000 0.000 0.000 GFI (Goodness of Fit) 0.825 0.827 0.822 CFI (Comparative Fit Index) 0.707 0.706 0.601 RMSEA (Root Mean Square Error of Approximation) 0.347 0.346 0.350 Number of observations 563 563 563 *** Statistically significant at p < 0.01, using a two-tailed test. This table shows the results of estimating structural equation model (1) separately for each financial outcome measure: revenue (REV), operating costs (COST), and profit (PROFIT). SKILL is retirement costs per employee, INTRO is new product or service introductions, CUST is customer satisfaction, and MKTSH is relative market share. All variables are estimated relative to the industry and deflated by total assets. The model is estimated using maximum likelihood. To adjust for skewness and kurtosis we use the natural log of all variables except for SKILL, INTRO, and PROFIT. We use the square root of SKILL and INTRO and square PROFIT. The sample is 563 firm-years with the required hand-collected data over the period 1994-1998. All p-values are bootstrapped. TABLE 3 Structural Equation Model of Balanced Scorecard Framework: Partially Mediated Model Panel A: Standardized Coefficients Independent Standardized Dependent Variable ([R.sup.2]) Variable Coefficients INTR[O.sub.t] (2.8%) SKIL[L.sub.t] 0.175 *** COS[T.sub.t] (53.4%) SKIL[L.sub.t] 0.737 *** INTR[O.sub.t] 0.105 *** MKTS[H.sub.t] (61.8%) SKIL[L.sub.t] -0.050 INTR[O.sub.t] 0.000 CUS[T.sub.t] 0.820 *** FI[N.sub.t] RE[V.sub.t](51.3%) SKIL[L.sub.t] 0.046 INTR[O.sub.t] 0.005 CUS[T.sub.t] 0.094 MKTS[H.sub.t] 0.612 ** CUS[T.sub.t] (50.1%) SKIL[L.sub.t] 0.067 INTR[O.sub.t] 0.009 CUS[T.sub.t] 0.039 MKTS[H.sub.t] 0.637 *** PROFI[T.sub.t] (9.7%) SKIL[L.sub.t] -0.025 INTR[O.sub.t] 0.103 ** CUS[T.sub.t] 0.193 ** MKTS[H.sub.t] 0.110 Panel B: Model Fit Statistics All Models [chi square] 0.676 p-value 0.411 GFI (Goodness of Fit) 1 CFI (Comparative Fit Index) 1 RMSEA (Root Mean Square Error of Approximation) 0 n 563 **, *** Statistically significant at p < 0.05 and p < 0.01, respectively, using a two-tailed test. This table shows the results of estimating structural equation model (2) separately for each financial outcome measure: revenue (REV), operating costs (COST) and profit (PROFIT). SKILL is retirement costs per employee, INTRO is new product or service introductions, CUST is customer satisfaction, and MKTSH is relative market share. All variables are estimated relative to the industry and deflated by total assets. The model is estimated using maximum likelihood. To adjust for skewness and kurtosis we use the natural log of all variables except for SKILL, INTRO, and PROFIT. We use the square root of SKILL and INTRO and square PROFIT. The sample is 563 firm-years with the required hand-collected data over the period 1994-1998. All p-values are bootstrapped. Due to model saturation, the path from INTRO to MKTSH is restricted to zero. The [R.sup.2] is the squared multiple correlation calculated by AMOS 4.0. TABLE 4 Standardized Direct and Indirect Effects for the Partially Mediated Model Dependent Independent Variable Variable Direct Indirect Total INTR[O.sub.t] SKIL[L.sub.t] 0.175 *** 0.000 0.175 *** CUS[T.sub.t] SKIL[L.sub.t] 0.737 *** 0.018 ** 0.755 *** INTR[O.sub.t] 0.105 *** 0.000 0.105 *** MKTS[H.sub.t] SKIL[L.sub.t] -0.050 0.619 *** 0.570 *** INTR[O.sub.t] 0.000 0.086 ** 0.086 * CUS[T.sub.t] 0.820 *** 0.000 0.820 *** FI[N.sub.t] RE[V.sub.t] SKIL[L.sub.t] 0.046 0.421 *** 0.467 *** INTR[O.sub.t] 0.005 0.063 *** 0.068 CUS[T.sub.t] 0.094 0.502 *** 0.596 *** MKTS[H.sub.t] 0.612 *** 0.000 0.612 *** COS[T.sub.t] SKIL[L.sub.t] 0.067 0.394 *** 0.461 *** INTR[O.sub.t] 0.009 0.059 *** 0.068 CUS[T.sub.t] 0.039 0.522 *** 0.562 *** MKTS[H.sub.t] 0.637 *** 0.000 0.637 *** PROFI[T.sub.t] SKIL[L.sub.t] -0.025 0.227 0.201 *** INTR[O.sub.t] 0.103 ** 0.030 0.133 *** CUS[T.sub.t] 0.193 ** 0.091 * 0.284 *** MKTS[H.sub.t] 0.110 0.000 0.110 **, *** Statistically significant at p < 0.05 and p < 0.01, respectively, using a two-tailed test. This table shows the direct and indirect effects for the structural equation model results presented in Table 3. SKILL is retirement costs per employee, INTRO is new product and service introductions, CUST is customer satisfaction, and MKTSH is relative market share. The financial outcome measure is defined in three separate ways: revenue (REV), operating costs (COST) and profit (PROFIT). All variables are estimated relative to the industry and deflated by total assets. The model is estimated using maximum likelihood. To adjust for skewness and kurtosis we use the natural log of all variables except for SKIL[L.sub.t] INTR[O.sub.t] and PROFIT. We use the square root of SKILL and INTRO and square PROFIT. The sample is 563 firm-years with the required hand-collected data over the period 1994-1998. Total effects may not equal the sum of direct and indirect effects due to rounding. TABLE 5 Moderating Effects of the Performance Measurement System Panel A: Standardized Coefficients Sample Sample Firms Firms Using Both Using Only Financial and Dependent Independent All Sample Financial Nonfinancial Variable Variable Firms Measures Measures INTR[O.sub.t] SKIL[L.sub.t] 0.175 *** 0.233 *** 0.265 *** CUS[T.sub.t] SKIL[L.sub.t] 0.737 *** 0.732 *** 0.838 *** INTR[O.sub.t] 0.105 *** 0.094 ** 0.063 MKTS[H.sub.t] SKIL[L.sub.t] -0.049 0.001 0.103 INTR[O.sub.t] -0.022 -0.096 *** 0.019 CUS[T.sub.t] 0.825 *** 0.896 *** 0.642 *** FI[N.sub.t] RE[V.sub.t] SKIL[L.sub.t] 0.046 0.083 0.009 INTR[O.sub.t] 0.005 0.012 0.076 CUS[T.sub.t] 0.094 -0.226 ** 0.323 *** MKTS[H.sub.t] 0.612 *** 0.937 *** 0.248 *** COS[T.sub.t] SKIL[L.sub.t] 0.067 0.100 0.013 INTR[O.sub.t] 0.009 0.025 0.072 CUS[T.sub.t] 0.039 -0.294 ** 0.283 *** MKTS[H.sub.t] 0.637 *** 0.974 *** 0.282 *** PROFI[T.sub.t] SKIL[L.sub.t] -0.025 0.128 -0.112 INTR[O.sub.t] 0.103 ** 0.027 0.233 *** CUS[T.sub.t] 0.193 ** 0.081 0.251 *** MKTS[H.sub.t] 0.110 0.109 0.008 Panel B: Total Effect of each Coefficient on Profit (by Sample Split) Sample Firms Using Both Sample Firms Financial and Using Only Nonfinancial Financial Measures Measures Total Effects on Profit Model SKIL[L.sub.t] 0.268 *** 0.170 *** INTR[O.sub.t] 0.034 0.249 *** CUS[T.sub.t] 0.179 * 0.257 *** MKTS[H.sub.t] 0.109 0.008 Number of Observations 281 228 *, **, *** Statistically significant at p < 0.10, p < 0.05, and p < 0.01, respectively, using a two-tailed test. This table shows the results of estimating structural equation model (2) separately for each financial outcome measure: revenue (REV), operating costs (COST), and profit (PROFIT). SKILL is retirement costs per employee, INTRO is new product or service introductions, CUST is customer satisfaction, and MKTSH is relative market share. All variables are estimated relative to the industry and deflated by total assets. The model is estimated using maximum likelihood. To adjust for skewness and kurtosis we use the natural log of all variables except for SKIL[L.sub.t] INTR[O.sub.t] and PROFIT. We use the square root of SKILL and INTRO and square PROFIT. The sample is 563 firm-years with the required hand-collected data over the period 1994-1998. All p-values are bootstrapped. Since we are not as interested in fit statistics as much as we are interested in path coefficients and significance, we estimate a saturated model and remove the restriction that required the path from INTRO to MKTSH to be zero. Thus, there may be small differences in the coefficient values between Table 3 and Table 5.
We thank Peter Easton, Frederick Lindahl, Frank Selto, and Naomi Soderstrom for their helpful comments on earlier drafts of this paper. We also gratefully acknowledge comments received from workshop participants at the 2000 American Accounting Association Annual Meeting, from The Ohio State University, and from the EIASM 6th Manufacturing Accounting Research Conference at Twente, The Netherlands. Finally, this paper has benefited from the constructive suggestions of two anonymous reviewers.
(1) The terms "lowest level in the hierarchy" and "highest-level perspective" refer to the timing of managerial actions and not to the importance of the perspective.
(2) As described later, our sample is biased toward large firms and has a higher proportion of food, textile, chemical, transportation, and utilities firms than does the Compustat population.
(3) Retirement expense and salary expense are significantly correlated (r = 0.791, p < 0.01). Since salary expense is available for only a limited number of firms in our sample, we use retirement expense to calculate the measure. We run a sensitivity test using salary expense and discuss the results in the sensitivity part of the "Results" section.
(4) As a sensitivity analysis, we also investigate employee productivity as an alternative measure of the learning and growth perspective. Employee productivity is a measure commonly used in performance measurement systems. A recent survey of performance measurement practices (AICPA and Maisel 2001) showed that 47 percent of the firms surveyed used productivity as a nonfinancial measure in their performance measurement system. However, depending on the overall design of their BSC, firms might consider it an outcome measure of either the learning and growth or financial perspectives (Niven 2002). We discuss this sensitivity analysis in the "Results" section.
(5) Ideally, we would model all four components of the internal business process perspective; however, using archival data places constraints on the availability of proxies. We do examine an alternative proxy, inventory turnover, as a measure of the efficiency of the operational process. We discuss this proxy in the sensitivity analysis portion of the "Results" section.
(6) Although profit captures both revenues and costs, we model all three financial measures in order to provide more in-depth insights. For example, customer satisfaction may not be significantly associated with profits, yet it may be significantly associated with both revenues and costs. Knowledge of the underlying associations provides more precise managerial implications.
(7) While studies have found some of the measures to be leading indicators of performance (e.g., Ittner and Larcker 1998), the timing has been within a one-year period. Furthermore, many of the studies that analyze our measures have focused on a contemporaneous analysis (e.g., Kekre and Srinivasan 2002). Thus, we focus on a contemporaneous analysis of multiple measures because (1) our data is annual, (2) this approach follows the prior literature, and (3) there is no theoretical or empirical guidance as to the length of timing effects. While the examination of timing effects is an interesting exploratory analysis, it is beyond the scope of this study.
(8) Insurance firms are often holding companies with many subsidiaries. This makes it difficult to match Compustat data, which is typically at the holding company level, to the ACSI score, which is typically at the subsidiary level.
(9) Pooling observations over time can result in serial autocorrelation. The Newey-West autocorrelation consistent covariance estimator (Greene 1997, 506; Newey and West 1987) can be used when serial autocorrelation is present; however, the Newey-West estimator is not available in AMOS graphics (SEM). Since our model is fully recursive, we can estimate our model using ordinary least squares and obtain consistent and efficient estimates (Greene 1997, 737). Therefore, we replicate Tables 3 and 5 using ordinary least squares and the Newey-West estimator. We find that the results are qualitatively consistent with the results reported in Tables 3 and 5 (using SEM) and conclude that, with one exception, serial autocorrelation is not driving the results. Using the Newey-West estimator, we find that the paths from CUST to both REV and COST for the subset of firms using only financial measures in their performance measurement system are not significantly different from zero. This does not change the overall conclusion, though, that the design of the performance measurement system moderates the relation between customer satisfaction and financial outcomes.
(10) We ran the partially mediated revenue model on the entire dataset. We then reviewed the Mahalanobis [D.sup.2] scores, which are used to assess multivariate outliers. This score indicates the difference between the sample means and the individual case means. In accordance with guidance that a conservative test be used, we only removed cases that had a p < 0.01, which indicates that these cases exert undue influence on the sample (Kline 1998).
(11) The number of firms in each subsample does not add up to the total sample because we were unable to find information on the compensation plan used for 54 of the firm-years.
(12) For firms with defined benefit plans, the pension expense is comprised of a number of components that are unrelated to employee compensation, such as the return on pension plan assets. For these firms, we include only the portion of the pension cost related to employee service during the year by subtracting out the total pension cost and adding back the service cost component.
(13) We do not know the exact percentage of product introductions that are publicly announced. If firms have product introductions significant enough to affect the relationship with customer satisfaction, market share, or financial outcomes and do not publicly announce the introduction, then this will bias against our finding results. However, we expect that the sample firms will make a public announcement in most cases. This method of estimating the extent of new product introductions has been previously used in the literature (see Chaney et al. 1991).
(14) This measure follows the marketing and management literature procedure of defining market share relative to the top competitors in the market (see, e.g., Bharadwaj and Menon 1993; Markell et al. 1988; Varadarajan 1985; Buzzell and Wiersema 1981). This measure is preferable when cross-sectional data is pooled across industries (Varadarajan and Dillon 1982).
(15) We define operating costs in this way to obtain a consistent measure across firms. There is significant variation across firms and industries with respect to which costs are allocated to cost of goods sold or selling, general and administrative expenses.
(16) Because many of our variables are hand-collected, industry averages reflect only the firms in our sample. We use the one-digit SIC code because we have a number of two-digit SIC codes with only one or two firms.
(17) We use total assets instead of book value of equity because total assets are not related to financing decisions. As a robustness check, we repeat the sample creation steps using book value of equity as the deflator. Using the revenue model as a sensitivity test, we find that the statistical inferences do not change for eight of the links. We find that product introductions and employee skills are directly associated with revenue. Kurtosis of approximately 18 in the INTRO variable may be the reason for the significance of these two additional paths, but transformations did not alleviate this effect. See footnote 19 for further explanation of normality. Overall, we conclude that using total assets as the deflator is not driving our results.
(18) Structural equation modeling (SEM) is a recommended technique for analyzing a model that includes multiple dependent variables (Kline 1998). See Widener (2004) and Anderson and Young (1999) for examples of management accounting research papers that use SEM, and for further information on the underlying statistical method see Byrne (2001) and Kline (1998).
(19) Kurtosis and skewness indicate that the data are not within tolerable levels of univariate normality. Kline (1998) suggests that skewness greater than 3.0 and kurtosis greater than 10.0 may suggest a problem with the data. We use the natural log of customer satisfaction, market share, revenue, and cost. Due to mathematical properties, we cannot use the natural log of skills, product introductions, or profit, so we take the square root of skills and product introductions and square profit. These transformations are suggested by Kline (1998, 83) and Judd and MeClelland (1989) to correct for nonnormal data. Kline (1998, 83) suggests that multivariate nonnormality can usually be identified through univariate procedures; thus by removing outliers and transforming the variables, we assume that our data is multivariate normal. To be conservative, we report bootstrapped p-values, a technique that does not assume multivariate normality (Kline 1998).
(20) In addition to checking for nonnormality and outliers, we also test for multicollinearity using two approaches. First, we run a simple bivariate correlation between the variables in Table 2 (employee skills, product introductions, customer satisfaction, market share, revenue, cost, and profitability). Although there are many significant correlations, none exceeds the 0.85 threshold suggested by Kline (1998). To assess multivariate multicollinearity, we also review the variance inflation and tolerance factors (using SPSS). Kline (1998) suggests that tolerance factors less than 10 percent and/or variance inflation factors greater than 10 may indicate redundancy. All variables are well within the tolerable limits.
(21) We chose to restrict the path from product introductions to market share to have zero value because doing so has no effect on the statistical inferences that we draw from Table 3. As a robustness check, we also ran the partially mediated model without restricting the path from product introductions to market share to have zero value and find that this path is not significant. Furthermore, we find that the statistical inferences drawn from the remaining paths in the model are unchanged.
(22) Although this model is a significant improvement over the fully mediated model (Table 2), there is no assurance that this is the "best" model because other, more parsimonious models could potentially fit the data equally well.
(23) The direct effects are the coefficients from the path model reported in Table 3. The indirect effects are reported by the SEM program and calculated as the product of the coefficients represented in the path. The statistical significance of the direct effects is equivalent to that reported for the coefficients in Table 3. The significance of the indirect effect of a path that involves three variables (e.g., two paths) is calculated as (a * b)/S[E.sub.ab] (where the coefficients of the two paths are "a" and "b"). The statistical significance of more complex paths is considered significant if all of the component paths are significant. The significance of the total effects is derived from regression results that remove the mediating or intervening variable from the regression. For more information on calculating the significance of indirect and total effects, see Kline (1998, Appendix 5.B).
(24) The association between productivity and profitability may depend on capital intensity. If the firm is highly capital intensive, increases in employee productivity may be offset by increases in depreciation expense and other capital expenses (e.g., repairs and maintenance). We split the sample into two groups: those that have high capital intensity versus those that are low in capital intensity and find that capital intensity moderates the relationship between productivity and profitability. We find that firms that are low in capital intensity have a positive, significant relationship between productivity and profitability, while those firms that are high in capital intensity have a negative, significant relationship between productivity and profitability. However, we find that capital intensity does not moderate the relationship between the level of employee skills and productivity, so there is no effect on our base model.
Acemoglu, D. 2002. Directed technical change. Review of Economic Studies 69: 781-809.
American Institute of Certified Public Accountants (AICPA), and L. S. Maisel. 2001. Performance Measurement Practices Survey Results. Jersey City, NJ: AICPA.
Anderson, E. W., C. Fornell, and D. R. Lehmann. 1994. Customer satisfaction, market share, and profitability: Findings from Sweden. Journal of Marketing 58: 53-66.
Anderson, S. W., and S. M. Young. 1999. The impact of contextual and process factors on the evaluation of activity-based costing systems. Accounting, Organizations and Society 24: 525-559.
Antle, R., and J. Demski. 1988. The controllability principle in responsibility accounting. The Accounting Review (October): 700-718.
Balkcom, J. E., C. D. Ittner, and D. F. Larcker. 1997. Strategic performance measurement: Lessons learned and future directions. Journal of Strategic Performance Measurement (April/May): 22-32.
Banbury, C. M., and W. Mitchell. 1995. The effect of introducing important incremental innovations on market share and business survival. Strategic Management Journal 16:161-182.
Banker, R. D., G. Potter, and D. Srinivasan. 2000. An empirical investigation of an incentive plan that includes non-financial performance measures. The Accounting Review (January) 75: 65-92.
--, C. Konstans, and R. Mashruwala. 2001. Information content of employee satisfaction in predicting future earnings. Working paper, The University of Texas at Dallas and Washington University.
Bayus, B. L., and W. P. Putsis. 1999. Product proliferation: An empirical analysis of product line determinants and market outcomes. Marketing Science 18:137-153.
--, G. Erickson, and R. Jacobson. 2003. The financial rewards of new product introductions. Management Science 49: 197-210.
Behn, B. K., and R. A. Riley. 1999. Using non-financial information to predict financial performance: The case of the U.S. airline industry. Journal of Accounting, Auditing and Finance 14: 29-56.
Bester, H., and E. Petrakis. 2003. Wages and productivity growth in a competitive industry. Journal of Economic Theory 109: 52-69.
Bharadwaj, S. G., and A. Menon. 1993. Determinants of success in service industries: A PIMS based empirical investigation. Journal of Services Marketing 4:19-40.
Bushman, R. M., and R. I. Indjejikian. 1993. Stewardship value of "distorted" accounting disclosures. The Accounting Review 68: 765-782.
Buzzell, R. D., and F. D. Wiersema. 1981. Modeling changes in market share: Across-sectional analysis. Strategic Management Journal 2: 27-42.
Byrne, B. N. 2001. Structural Equation Modeling with AMOS. Mahwah, NJ: Lawrence Erlbaum Associates.
Capon, N., J. U. Farley, and S. Hoenig. 1990. Determinants of financial performance: A meta analysis. Management Science 36:1143-1159.
Chancy, P., T. Devinney, and R. Winer. 1991. The impact of new product introductions on the market value of firms. Journal of Business 64:573-610.
Chow, C. W., K. M. Haddad, and J. E. Williamson. 1997. Applying the balanced scorecard to small companies. Management Accounting (August): 21-27.
Copeland, T., T. Koller, and J. Murrin. 2000. Valuation: Measuring and Managing the Value of Companies. 3rd edition. New York, NY: John Wiley and Sons, Inc.
Davis, S., and T. Albright. 2004. An investigation of the effect of balanced scorecard implementation on financial performance. Management Accounting Research 15:135-153.
Delaney, J. T., and M. A. Huselid. 1996. The impact of human resource management practices on perceptions of organizational performance. Academy of Management Journal 39: 949-969.
Doms, M., T. Dunne, and K. R. Troske. 1997. Workers, wages and technology. The Quarterly Journal of Economics 112: 253-290.
Feltham, G. A., and J. Xie. 1994. Performance measure congruity and diversity in multi-task principal/agent relations. The Accounting Review 69: 429-453.
Fornell, C., M. D. Johnson, E. W. Anderson, J. Cha, and B. E. Bryant. 1996. The American customer satisfaction index: Nature, purpose and findings. Journal of Marketing 60: 7-18.
Foster, G., and M. Gupta. 1999. The customer profitability implications of customer satisfaction. Working paper, Stanford University and Washington University in St. Louis.
Gersbach, H. 1998. On the equivalence of general and specific control in organizations. Management Science (May): 730-737.
Greene, W. H. 1997. Econometric Analysis. 3rd edition. Upper Saddle River, NJ: Prentice Hall.
Hemmer, T. 1996. On the design and choice of "modern" management accounting measures. Journal of Management Accounting Research 8:87-116.
Heskett, J. L., T. O. Jones, Ca W. Loveman, W. E. Sasser, and E. A. Schlesinger. 1994. Putting the service profit chain to work. Harvard Business Review 72:164-174.
--, W. E. Sasser, Jr., and L. A. Schlesinger. 1997. The Service Profit Chain. New York, NY: The Free Press.
Hilton, R. W., M. W. Maher, and F. H. Selto. 2003. Cost Management: Strategies for Business Decisions. New York, NY: McGraw-Hill.
Hoque, Z., and W. James. 2000. Linking balanced scorecard measures to size and market factors: Impact on organizational performance. Journal of Management Accounting Research 12:1-17.
Huselid, M. A. 1995. The impact of human resource management practices on turnover, productivity, and corporate financial performance. Academy of Management Journal 38: 635-672.
Ittner, C. D., and D. F. Larcker. 1998. Are non-financial measures leading indicators of financial performance? An analysis of customer satisfaction. Journal of Accounting Research (Supplement) 36: 1-35.
--, and M. V. Rajan. 1997. The choice of performance measures in annual bonus contracts. The Accounting Review (April): 231-255.
--, and --. 2001. Assessing empirical research in managerial accounting: A value based management perspective. Journal of Accounting and Economics 32:349-4 10.
--, and T. Randall. 2003. Performance implications of strategic performance measurement in financial services firms. Accounting, Organizations and Society 28:715-741.
Judd, C. M., and G. H. McClelland. 1989. Data Analysis: A Model-Comparison Approach. Orlando, FL: Harcourt Brace Jovanovich Publishers.
Kaplan, R. S., and D. P. Norton. 1996. The Balanced Scorecard. Boston, MA: Harvard Business School Press.
--, and --. 2001a. Transforming the Balanced Scorecard from performance measurement to strategic management: Part I. Accounting Horizons 15:87-104.
--, and --. 2001b. The Strategy Focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment. Boston, MA: Harvard Business School Press.
Kekre, S., and K. Srinivasan. 2002. Broader product line: a necessity to achieve success? Management Science 36: 1216-1231.
Kline, R. B. 1998. Principles and Practice of Structural Equation Modeling. New York, NY: The Guilford Press.
Lev, B. 2001. Intangibles: Management, Measurement, and Reporting. Washington, D.C.: Brookings Institution Press.
Lipe, M. G., and S. E. Salterio. 2000. The balanced scorecard: Judgmental effects of common and unique performance measures. The Accounting Review 75 (July): 283-298.
Malina, M. A., and F. H. Selto. 2002. Management control and performance measurement: An empirical test of causal relations. Working paper, Naval Postgraduate School and the University of Colorado at Boulder.
Markell, S. J., S. E. Neeley, and T. H. Strickland. 1988. Explaining profitability: Dispelling the market share fog. Journal of Business Research 16: 189-196.
McNair, C. J., R. L. Lynch, and K. F. Cross. 1990. Do financial and non-financial performance measures have to agree? Management Accounting (November): 28-36.
Milkovich, Ct T., and J. M. Newman. 2002. Compensation. New York, NY: McGraw-Hill.
Newey, W. K., and K. D. West. 1987. A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica 55 (May): 703-708.
Niven, P. R. 2002. Balanced Scorecard Step by Step: Maximizing Performance and Maintaining Results. New York, NY: John Wiley and Sons, Inc.
Russell, J. S., J. R. Terborg, and M. L. Powers. 1985. Organizational performance and organizational level training and support. Personnel Psychology 38: 849-863.
Rust, R., and A. Zahorik. 1993. Customer satisfaction, customer retention and market share. Journal of Retailing 69: 145-156.
--, V. A. Zeithaml, and K. N. Lemon. 2000. Driving Customer Equity. New York, NY: The Free Press.
Said, A. A., H. R. HassabElnaby, and B. Wier. 2003. An empirical investigation of the performance consequences of nonfinancial measures. Journal of Management Accounting Research 15: 193-223.
Smith, M. J. 2002. Gaming nonfinancial performance measures. Journal of Management Accounting Research 14:119-133.
Smith, R. E., and W. F. Wright. 2004. The determinants of customer loyalty and financial performance. Journal of Management Accounting Research: 16: 183-205.
Stivers, B. P., T. J. Covin, N. G. Hall, and S. W. Smalt. 1998. How non-financial performance measures are used. Management Accounting 79: 44-49.
Szymanski, D. M., S. G Bharadwaj, and p. R. Varadarajan. 1993. An analysis of the market share profitability relationship. Journal of Marketing 57: 1-18.
Varadarajan, P. R., and W. R. Dillon. 1982. Intensive growth strategies. Journal of Business Research 9: 49-64.
--. 1985. A two factor classification of competitive strategy variables. Strategic Management Journal 6: 357-375.
Wallman, S. M. H. 1995. The future of accounting and disclosure in an evolving world: The need for dramatic change. Accounting Horizons 9(3): 81-91.
Westbrook, R. A., P. Kumar, and M. J. Epstein. 2000. The drivers of customer and corporate profitability: Modeling, measuring and managing the causal relationships. Advances in Management Accounting 9: 43-72.
Widener, S. K. 2004. An empirical investigation of the relation between the use of strategic human capital and the design of the management control system. Accounting, Organizations and Society 12: (377-399).
University of Oregon
Denise A. Jones
College of William & Mary
Sally K. Widener
|Printer friendly Cite/link Email Feedback|
|Author:||Bryant, Lisa; Jones, Denise A.; Widener, Sally K.|
|Publication:||Journal of Management Accounting Research|
|Date:||Jan 1, 2004|
|Previous Article:||Aggregation and measurement errors in performance evaluation.|
|Next Article:||The value of activity-based costing in competitive pricing decisions.|