Decision Aids for Generating Analytical Review Alternatives: The Impact of Goal Framing and Audit-Risk Level.
An auditor generating potential explanations for an unusual variance in analytical review may utilize a decision aid, which provides many explanations. However, circumstances of budgetary constraints and limited cognitive load deter an auditor from using a lengthy list of explanations in an information search. A two-way between-subjects design was created to investigate the effects of two complementary approaches to trimming down the lengthy list on the number of remaining explanations carried forward into an information search. These two approaches, which represent the same goal (reducing the list) but framed differently, are found to result in a significantly different number of remaining explanations, in both low- and high-risk audit environments. The results of the study suggest that the extent to which an auditor narrows the lengthy list of explanations is important to the implementation of decision aids in analytical review.
Consider the common judgment scenario wherein one is given a large number of alternatives and must decide which of the alternatives is most likely correct. When the correct answer is not known, one will generally begin by reducing the longer list of alternatives to a shorter, more meaningful list of potentially correct answers. Ponder, for example, the judgment process that an eyewitness goes through in attempting to identify a criminal from the scene of a crime (Yaniv and Schul 1997). The eyewitness is asked to filter through a large stack of photos and create a reduced set of likely suspects. The eyewitness may cognitively approach this task by either including likely photos or eliminating less likely photos from the larger set. A pertinent question is whether the same potential criminals will be submitted as suspects under either approach.
A decision maker's goal in reducing a large number of alternatives to a shorter list can be framed in two alternative ways--include or eliminate. The processes of elimination and inclusion are logically equivalent (i.e., those items eliminated should be exactly those items not included). Regardless of whether one uses elimination or inclusion, he or she should arrive at the same reduced set of potential correct answers. However, a simple adjustment to the way in which the task is framed has been shown to have profound effects on the judgment process, and consequently, on the makeup of the reduced list (cf., Lichtenstein and Slovic 1973; Huber et al. 1987; Jaradat and Tollefson 1988; Medin et al. 1990; Westenberg and Koele 1992; Yaniv and Schul 1997, 2000; Levin et al. 1998; Prosansky and Levin 1999). Specifically, studies have shown that the inclusion (elimination) process leads to a reduced set that contains a smaller (greater) number of alternatives (cf., Huber et al. 1987; Yaniv and Schul 1997, 2000; Levin et al. 1998; Prosansky and Levin 1999).
Within accounting, the effects of goal framing may have implications for judgments involving lengthy lists of alternatives (e.g., a lengthy list of citations obtained in tax research or a lengthy list of investment alternatives in personal financial planning). In auditing, one such instance is the use of a decision aid in analytical review. A decision aid may provide an auditor with a substantial number (e.g., 20) of potential causes for an unusual fluctuation in a ratio. In preparing for an efficient information search, an auditor may reduce the large number of potential explanations to a more manageable number of most likely alternatives. An auditor proceeds by either eliminating those deemed least likely or including those deemed most likely, to achieve a reduced set of explanations. Dissimilar reduced sets, caused by the two different approaches, may lead to different initial searches for information and potentially different outcomes of the analytical procedure--one of which may be less effective.
Although the psychology literature reports goal-framing effects, studies have either addressed general decision making (e.g., trivia questions in Yaniv and Schul ) or have addressed field-specific decision making by utilizing student participants (e.g., students making human resource decisions in Huber et al. ). In tempering their results, Huber et al. (1987) caution that decision-making processes of college students may not model expert decision makers or decision makers operating in their area of expertise (e.g.. human resource managers making human resource decisions). It has also been conjectured that the discrepancy between the elimination and inclusion frames may disappear in decision scenarios that carry a high level of accountability (Levin et al. 1998). Because of the accountability and expertise necessary in analytical review judgments. it is unclear whether an auditor using a decision aid in analytical review will be susceptible to the effects of goal framing. Furthermore, a recent revi ew of audit judgment research suggests that "auditing's unique aspects...means that auditors cannot simply accept and apply findings from other disciplines such as psychology and decision sciences" (Bamber et al. 1995, 85). This study investigates goal framing by examining the resultant number of explanations after an auditor reduces a lengthy list provided by a decision aid in analytical review.
This study also considers the reduced number of explanations in the context of varied client risk, which is of particular concern when performing analytical review in the planning stages of the audit. (1) A risky audit environment heightens auditor skepticism and increases audit effort, as predicted by the audit-risk model. Heightened skepticism may lead an auditor to consider more of the feasible explanations from the lengthy list provided by the decision aid. Furthermore, a natural increase in audit effort that follows from high client risk may translate into the pursuit of more explanations than in a low-risk environment. Because varied risk in the audit environment may influence the size of the reduced set and also moderate the effect of goal framing on the size of the reduced set, two levels of risk are studied.
LITERATURE REVIEW AND HYPOTHESES
Significance of Number of Explanations to Analytical Review
Decision aids provide a means for auditors to initially consider a greater number of hypotheses. This is important to the diagnostic process, as an auditor will likely conclude that one or more of the initial hypotheses explain the fluctuation. Moreover, research has shown that even an optimal information search strategy cannot compensate for a poor hypothesis set (Asare and wright 1998).
The hypothesis set affects both the effectiveness and efficiency of analytical review. Effectiveness is essentially the need for the set to contain the correct hypothesis. whereas efficiency is represented by the number of hypotheses that must be investigated (size of the set) (Koonce 1993). The link between efficiency and effectiveness and the different sources an auditor may use to generate hypotheses (e.g., memory, client, decision aid) is that a decision aid may yield a larger hypothesis set that is less efficient, yet more effective, than a set developed from other sources. Indeed, a thorough decision aid may provide an auditor with substantially more explanations (e.g., 15 to 20 explanations as in Anderson et al. [1995, 1997]) compared to the number of explanations that could be self-generated by auditors (e.g., five to six explanations, or fewer, as in Hirst and Koonce , Koonce , Anderson et al. ).
Two issues arise that make it difficult for an auditor to investigate all hypotheses given by the decision aid-increased cognitive load and greater budgetary constraint. Decision makers are known to scale back the amount of information considered to (1) keep cognitive processing to a manageable level (Just and Carpenter 1992; Baddeley 1986); and (2) reduce time pressure, such as that resulting from budgetary constraints. Psychology research has shown that cognitive (or information) overload is often countered with a filtration strategy, whereby a decision maker screens information and subjectively identifies the most important information for further consideration (Miller 1960). (2)
This study examines the extent to which an auditor narrows a large hypothesis set given by a decision aid in the initial hypothesis generation stage of analytical review. The auditor's judgment is examined in the context of a theoretical framework from classical decision theory that has been adjusted for the effects of goal framing.
Framing, as related to a task, represents alternative descriptions of a task that do not alter the true substance of the task, but cause a change in the decisionmakers' thinking about the task (Huber et al. 1987; Jamal et al. 1995). This study focuses on a specific type of framing effect attributable to the manner in which the decision goal is framed. Studies from psychology show inconsistent decision making due to goal framing in tasks that contain multiple answer alternatives. Specifically, in each scenario, a goal framed as an exclusion procedure resulted in a larger reduced set of alternatives than a goal framed as an inclusion procedure (Huber et al. 1987; Levin et al. 1998; Prosansky and Levin 1999; Yaniv and Schul 2000).
Yaniv and Schul's Theoretical Framework to Explain Goal-Framing Effects
Yaniv and Schul (1997) proposed a theoretical framework to explain how a decision maker arrives at differing outcomes under alternative, but logically equivalent, decision goals. The framework explains the judgment process of reducing a list of alternatives in order to reach a correct answer when the correct answer is not known. The framework proposes a screening model consistent with classical decision theory and a modified model to account for the effects of goal framing.
The model consistent with classical decision theory predicts equivalent reduced sets under either goal frame. Those alternatives included under inclusion should be exactly those hypotheses not eliminated under elimination. The model may be defined as follows. Let [E.sub.s], a continuous variable, represent the strength of evidence regarding an alternative. In analytical review, [E.sub.s] may be viewed as the participant's perceived likelihood of a potential explanation for an unusual fluctuation. Also, let [C.sub.Incl] ([C.sub.Elim]) represent the criterion value that [E.sub.s] in favor of a given alternative must surpass (fall below) in order for that alternative to be included in (excluded from) the reduced set of alternatives. In other words, if a judge is using an inclusion approach, then only alternatives for which [E.sub.s] > [C.sub.Incl] will be in the reduced set; if using an elimination approach, then alternatives for which < [C.sub.Elim] will be eliminated, leaving the remaining alternatives in the reduced set. In analytical review. [C.sub.Incl] and [C.sub.Elim] represent likelihood thresholds. Therefore, if the participant considers the explanation to be likely (unlikely) enough to surpass (fall below) the threshold, then the explanation will be included in (eliminated from) the reduced set.
If the screening model is to yield equivalent reduced sets under inclusion and elimination, then [C.sub.Incl] must equal [C.sub.Elim] as illustrated in Figure 1, Panel A. In terms of probability, the model states that the probability of an alternative being included and the probability of the same alternative being eliminated are complementary and sum to 1, giving the model the property of complementarity. In other words, a participant arrives at the same conclusion for an alternative regardless of the framing of the goal.
To model the findings of the goal-framing literature. Yaniv and Schul (1997) proposed an adapted model, in which the criteria values are set apart. They concluded that the framing of the goal causes decision makers to infer where the "burden of proof lies, which in turn affects the setting of the criteria ([C.sub.Incl]) or [C.sub.Elim]) (Yaniv and Schul 1997, 2000). For instance, under inclusion, the decision maker feels as if he or she is accountable for including an explanation because leaving an explanation outside the decision set is the default. Alternatively, under elimination, the decision maker must justify eliminating an explanation because the default is to leave explanations inside the decision set. Such inferences drawn from the goal framing cause the decision maker to set the inclusion and elimination criteria apart, as illustrated in Figure 1, Panel B.
As before, if [E.sub.s] falls above (below) [C.sub.Incl] or ([C.sub.Elim]), an alternative will be included (eliminated). However. with the criteria set apart, some alternatives will not have evidence at either extreme. For those "in-between" alternatives, whether they are inside the choice set depends on the framing of the decision process. The inclusion participants will not include the "in-between" alternatives for lack of evidence to include them. However, under elimination, these very same alternatives will not be eliminated for lack of good reason to eliminate them, leaving them (by default) inside the choice set.
Yaniv and Schul (1997) call this model the subcomplementarity model, as the probability of an alternative being included and the probability of the same alternative being eliminated sum to less than 1. The prediction of the model is that the same participant can arrive at two different lists of alternatives depending on the way the goal is framed. Specifically, elimination will lead to a larger reduced set of alternatives, as it will include the "in-between" alternatives.
Application of Framework to Analytical Review
In the performance of analytical review, an auditor will hypothesize the cause(s) of any unexpected fluctuation in order to determine what type of and how much additional evidence is needed. A decision aid provides a means for auditors to generate and initially consider a greater number of alternatives than could otherwise be generated. Under time and cost constraints, an auditor will likely narrow down the list given by assessing the likelihood of the individual explanations on the full list.
Yaniv and Schul (1997) provide a framework to explain different judgments due to the manner in which the auditor's reduction goal is framed. The theoretical framework predicts that the likelihood threshold (how likely an explanation must be to make it into the reduced set) will be different under the inclusion and elimination frames. Inclusion will induce a higher, or stricter, threshold than will elimination. Therefore, fewer explanations will "make the cut" under the inclusion frame. The reduced set of explanations under the elimination frame will contain those that are somewhat less likely due to the lower threshold induced by that frame. As a result of the different thresholds, the inclusion frame will lead to a smaller reduced set than will the elimination frame.
H1: When using a decision aid in an analytical review task, auditors using an inclusion goal frame will conclude with a smaller set of explanations than will auditors using an elimination goal frame.
An auditor must contend with risk throughout the audit. However, in the early stage of an audit, when first attempting to determine required audit effort, auditors are particularly concerned with the riskiness of a client. Auditors are advised by professional guidance to assess the risk of material misstatement in a client's financial statements in conjunction with analytical review in the planning stage of the audit (AICPA 1983, 1988). Because this study investigates an auditor's judgment in this stage, we also consider the effects of client risk on the reduced list of explanations, in addition to the effects of goal framing.
Two related consequences of heightened risk might similarly impact the auditor's reduction of the lengthy list. Empirical evidence has shown that auditors are indeed sensitive to risk in audit planning, and will generally increase audit effort to counter heightened risk (e.g., Mock and Wright 1999; Davidson and Gist 1996; Bell et al. 1994; Biggs et al. 1988). This reaction is consistent with the prescription of the audit-risk model, which calls for increased audit effort to counterbalance heightened inherent and controls risks, if audit risk is to be held to a low level. In the analytical review stage of the audit, when an auditor is evaluating a lengthy list of explanations provided by a decision aid, increased effort will likely translate into investigation of a greater number of explanations, regardless of the manner in which the auditor's evaluation task (goal) is framed.
Second, heightened client risk is related to an auditor's conservatism heuristic. Conservatism takes many forms depending on the audit context, but is generally driven primarily by the substantial implications for audit failure (e.g., loss of reputation or legal liability). Conservatism serves as a functional heuristic used by an auditor to minimize economic losses (Smith and Kida 1991), and may go beyond the awareness called for by the standards (e.g., SAS No. 47). In terms of an auditor evaluating a lengthy list of explanations provided by a decision aid, conservatism may cause the auditor to be less inclined to discount any plausible explanation from the lengthy list, even if it is perhaps less likely than the others. The auditor may make this judgment regardless of the manner in which the evaluation task (goal) is framed.
Whether viewed in terms of increased audit effort or heightened conservatism, high risk may lead the auditor to retain a larger number of the explanations from the lengthy list than in low risk, regardless of the goal frame.
H2: When using a decision aid in an analytical review task, auditors in a high-risk environment will conclude with a larger set of explanations than will auditors in a low-risk environment.
Risk may moderate the influence of goal framing on the size of the reduced set of explanations, tempering the main effect of goal framing. As presented earlier, a high-risk scenario may prompt an auditor to act more conservatively, which is described as a loss protection mechanism. Based upon a review of judgment heuristics and biases in auditing, Smith and Kida (1991) concluded that, in some cases, conservatism overrides heuristics and biases found in general judgment settings, if the task performed is analogous to a typical audit task. (3) Thus, despite the robustness of the goal-framing effect in the psychology literature, in a high-risk environment, an auditor's conservatism might be expected to lessen the effect of goal framing. Specifically, in a high-risk/inclusion scenario, the influence of high-risk in enlarging the set of explanations (conservatism) may overcome the influence of the inclusion goal frame in reducing the set of explanations (psychological phenomenon). This potential interaction is sta ted as follows:
H3: When using a decision aid in an analytical review task, the effect of an inclusion goal frame reducing the set of explanations will be less in a high-risk environment than in a low-risk environment.
Participants were auditors from various offices of four of the Big 5 CPA firms. Panel B of Table 1 shows that six separate offices of the four firms participated, but with no more than two offices from any one firm. There were no significant firm/office effects on the dependent measures.
Supplemental participant data was collected regarding general auditing experience and experience with material errors/irregularities and ratio analysis (Panel A of Table 1). The participant group had a mean level of experience of 54.2 months, ranging from staff to partner. On average, participants had experienced an accounting error or material inventory error within the past 10 months. Participants reported auditing manufacturing clients about 23 percent of the time. On average, they reported analyzing the inventory turnover ratio in preliminary audit planning with a frequency of 3 on a 10-point scale and using any type of ratio analysis with a frequency of 5 on a 10-point scale. With the exception of auditors' frequency of use of ratio analysis, there were no significant effects of experience on the dependent measures. (4)
The case, adapted from Anderson et al. (1997), began with background information regarding a hypothetical client. Included in the background information were references to the client's corporate structure, management, controls, and other miscellaneous facts. Also included in the background information was a set of prior-year audited financial statements and current year unaudited financial statements. The participant was informed that he or she was performing analytical review for planning purposes and that the inventory turnover ratio had significantly decreased from the prior year to the current year. The ratio was computed for the participant drawing on the data provided in the financial statements.
Following the client background data, participants were told about analytical procedures software that had been developed by their hypothetical audit firm. The software was described as having been designed according to actual error occurrence, as documented in firm audit working papers over a period of eight years, along with the input of audit managers. The extensive piloting process for the software was described in order to make the participants comfortable with the reliability of the software. Participants were informed that they had attended a training session on the use of the software and were comfortable with its use. After the decision aid was described, participants were presented with a screenshot from the decision aid listing 20 potential explanations for the decrease in the inventory turnover ratio. The explanations evaluated in the case were actually generated as plausible error and nonerror explanations by audit managers in a previous study (Anderson et al. 1992), and are also reported in prev ious research (Anderson et al. 1995, 1997).
The participants were presented with two experimental tasks, each of which involved evaluating the list of explanations provided by the decision aid. The tasks will be explained further as the manipulated variables are described below. The final part of the case instrument consisted of manipulation check and demographic questions.
Participants were randomly assigned to one of four versions of the case instrument, corresponding to a two-way between-subjects design. The two manipulated variables are client risk (either high or low) and goal framing (either inclusion or elimination). The client-risk manipulation was achieved by arying the background information provided in the case. The high-risk client was described using inherent and control risk red flags," whereas the low-risk client was described using low-risk terminology. (5)
The goal-framing manipulation was achieved by varying the manner in which participants completed the first task of evaluating the list of provided explanations for the inventory turnover ratio. Participants were asked to mark the explanations that they considered either "likely" or "not likely" to be the correct explanations for the decrease in the inventory turnover ratio. The former represents the inclusion treatment and the latter, the elimination treatment. The responses from this task provided the dependent measure, reduced set size, used to test all hypotheses. A similar measure has been used in psychology to test for the effects of goal framing on decisions (cf., Huber et al. 1987; Yaniv and Schul 1997, 2000; Levin et al. 1998).
An additional dependent measure resulted from the second experimental task. Participants were once again presented the 20 explanations. They were asked to evaluate how likely each explanation was, for the cause of the fluctuation in the inventory turnover ratio using a 100-point Likert-type scale with endpoints labeled "Not Very Likely" and "Very Likely." Each subject, therefore, had 20 individual likelihood ratings, one for each of the explanations. The ratings were used to form the dependent measure, likelihood criterion, that was used in supplemental analysis to provide evidence of "how likely" participants required an explanation to be for inclusion in (or no elimination from) the reduced set of explanations. For each participant, the explanations in his/her reduced set were ranked according to his/her assessed likelihood of those explanations. The lowest likelihood assessment from among those explanations was the participant's likelihood criterion. (6) The likelihood criterion represents [C.sub.Incl] and [C.sub.Elim] for the inclusion and elimination participants, respectively, in Yaniv and Schul's (1997) goal-framing framework. Similar critical values have been used in psychology studies (cf., Huber et al. 1987; Yaniv and Schul 1997, 2000) and in auditing research (cf., Asare 1992).
Tests of Hypotheses
A two-way analysis of covariance (ANCOVA) was used to test the hypotheses. The original analysis of variance model included two levels of goal framing (inclusion and elimination) and two levels of client risk (high and low). Although the participants were randomly assigned to treatment conditions, auditors' frequency of using ratio analysis was not distributed evenly across the treatment conditions. Panel C of Table 2 shows the unbalanced distribution of this participant attribute in the treatment conditions (F = 8.24, p = 0.006). It is present to a greater extent in the inclusion cells than in the elimination cells and ranges from a mean of 4.46 in the elimination/high-risk condition to 5.79 in the inclusion/high-risk condition (as measured on a scale from 0 to 10). Frequency of using ratio analysis is inversely correlated with the size of the reduced set of explanations; thus, auditors who use ratio analysis more frequently would tend to have smaller reduced set sizes. In order to mitigate the unintended ef fect of frequency of using ratio analysis on the dependent measure, it was included as a covariate in the analysis. (7)
Before performing the tests of hypotheses, six participants' responses were eliminated as they failed the manipulation check for the goal-framing treatment. Preliminary analysis verified that the manipulation of risk was successful for the remaining 59 participants. (8) Also in the preliminary analysis, the assumptions for the use of ANCOVA were tested, all of which were met by the data.
It was predicted in H1 that participants experiencing the inclusion goal frame would produce, from a lengthy list of explanations provided by a decision aid, a smaller reduced set of explanations than those participants experiencing the elimination frame. A significant main effect for goal framing was found (Panel A of Table 2, F = 23.00, p <.001). Descriptive statistics (Panel B of Table 2) reveal that participants experiencing the inclusion goal frame concluded with about eight explanations, while those experiencing elimination concluded with approximately 13 explanations. Thus, goal framing caused the elimination participants to purge fewer explanations from the lengthy list than the inclusion participants, resulting in significantly smaller reduced sets for the inclusion participants. This result supports H1.
Hypothesis 2 predicted that a high-risk audit environment would lead to a larger reduced set of explanations than a low-risk audit environment. An inspection of the descriptive statistics, in Panel B of Table 2, discloses that high-risk participants had approximately 12 explanations, whereas low-risk participants had approximately nine explanations in their reduced sets. This difference is statistically significant (Panel A of Table 2, F = 5.06, p = 0.029), thus supporting H2.
Hypothesis 3 addressed the interaction of goal framing and risk, predicting that the riskiness of the audit environment would moderate the effects of goal frame on the size of the reduced set of explanations.
Specifically. the effects of goal framing were expected to be greater in a low-risk audit environment than in a high-risk environment. In a high-risk environment, the difference in reduced set sizes of inclusion and elimination should be significantly less distinct as even the inclusion participants should have a large reduced set. Treatment means (Panel B, Table 2) for the set sizes of the inclusion and elimination groups in the high-risk condition are 9.3 and 14.8, respectively--a difference of approximately 5.5 explanations. Treatment means (Panel B, Table 2) for the set sizes of the inclusion and elimination groups in the low-risk condition are 6.7 and 12.2--a difference of 5.5 explanations. There was no significant statistical interaction (Panel A, Table 2, p = .523) and therefore, the effect of goal framing in the high-risk condition was essentially the same as in the low-risk condition. Hypothesis 3 is not supported.
Various supplemental analyses were performed to further investigate the effects of goal framing and risk on participants' reduced explanation sets. The first analysis tested the tenet of Yaniv and Schul's (1997) theoretical framework that participants set thresholds for inclusion or elimination of the explanations according to the burden of proof (or default action) inferred from the goal frame. Inclusion (elimination) implies that the default is to leave explanations outside (inside) the reduced set unless they are likely (unlikely) enough to be included (eliminated). Inclusion should, therefore, result in a higher criterion value than elimination because explanations would have to be assessed as more likely to be in the reduced set under the former (as shown in Figure 1, Panel B).
Using the dependent measure likelihood criterion, supplemental analysis was conducted to gain insight regarding the criterion values, [C.sub.Incl] and [C.sub.Elim] being used by inclusion and elimination participants, respectively. Recall that these likelihood criteria represent the minimum likelihood of any explanation, required by a participant, for admittance into the reduced set. In order to support the underlying judgment processes of Yaniv and Schul's (1997) framework, the likelihood criterion for inclusion participants ([C.sub.Incl]) should be higher (more strict) than the likelihood criterion for elimination participants ([C.sub.Elim]). A t-test of the likelihood criterion values revealed that participants in the inclusion treatment required the explanations to be significantly more likely than participants in the elimination treatment, as shown by the respective mean likelihood criterion values of 47.7 and 31.2 (T = 2.92, p = 0.0026). (9)
In an attempt to portray a realistic decision aid in this case, it was designed to provide both error and nonerror causes to participants. This allowed analysis of the type of explanations (i.e., error or nonerror) included or eliminated by participants. Wright and Bedard (2000) recently found that auditors self-generated more errors when client inherent risk was high than when it was low. Of interest in the current study is whether participants placed a greater concentration on the error explanations from the decision aid when client risk was high than when it was low. The number of errors selected by inclusion participants or retained by elimination participants was examined. Results support the findings of Wright and Bedard (2000), in that participants experiencing high client risk selected (or retained) 7 error explanations while those experiencing low client risk selected (or retained) about 4.5 error explanations (F = 7.6 15, p = .008). (10)
DISCUSSION AND IMPLICATIONS
A decision aid is an example of technology that can provide auditors with a more complete listing of potential explanations for a fluctuation, reducing the reliance on the client and on memory, which is admittedly fallible (cf., Arkes 1981, 592; Smith and Kida 1991; Nesca 1997). Furthermore, a decision aid can provide these explanations independently of the client. In order for decision aids to be effective, however, they must be implemented without the introduction of (or at a minimum, with an awareness of) problematic decision-making biases, such as the goal-framing bias investigated in this study.
Bamber et al. (1995, 75) point out that "the mixed results on effectiveness of decision aids are not surprising given the lack of theory to guide their development and implementation." This study attempts to lend guidance by investigating the effects of goal framing on an auditor's use of a decision aid. All auditors began with a list of 20 explanations for a decrease in the inventory turnover ratio. As predicted, those experiencing an inclusion goal frame were found to derive a significantly smaller reduced set of alternatives (about 8 explanations) than auditors experiencing an elimination goal frame (about 13 explanations). Thus, the elimination auditors would be expected to begin their information search with essentially 5 more explanations than the inclusion auditors.
Determining which group of auditors is most likely to reach the correct explanation is outside the scope of this study and remains for future research. Prior research regarding hypothesis generation and information search provides no clear resolve. The elimination strategy may lead to a more effective analytical review, in that information search is begun with more "potentially correct" explanations (Asare and Wright 1998: Asare et al. 1997). However, auditors maybe comparably effective while more efficient at investigating fewer explanations, as produced by the inclusion strategy (Bhattacharjee et al. 1999). (11) Future research could empirically address whether inclusion or elimination leads to a more effective information search and what efficiency trade-offs are made. Auditors may need to be prompted by the decision aid to use a "standard" strategy (Jamal et al. 1995) or trained on the decision aid using a particular strategy, in order to mitigate the effects of goal framing.
Future research could also address the fate of the "abandoned" explanations--those that do not make it into the reduced set. Although prior research has addressed this question or explored it post hoc, evidence is inconsistent (Asare and Wright 1997; Bedard and Biggs 1991a; Johnson et al. 1991). Furthermore, the question has not been addressed in the setting of a lengthy list of explanations provided by a decision aid. Whether the abandoned hypotheses are once again considered may impact the effectiveness and/or efficiency of the analytical review under the two goal-framing approaches.
In addition to the effects of goal framing on the auditor's use of a decision aid, a more familiar influence was also addressed--risk. Auditors reacted to increased client risk in accordance with the audit-risk model, which stipulates an increase in audit effort for an increase in inherent and/or control risk for a given level of audit risk (AICPA 1983). Auditors were conservative in that they were less inclined to discount a plausible explanation for an unusual fluctuation when the audit environment is risky than when it is not. Furthermore, auditors in the high-risk scenario reacted to the "red flags" in the client description by considering a greater number of error explanations than auditors in the low-risk scenario. Future research might address whether auditors using a decision aid achieve greater effectiveness than auditors relying on self-generation in high-risk analytical review scenarios.
Finally, it was hypothesized that the client-risk effect would overcome the goal-framing effect. It was expected that auditors in the inclusion/high-risk condition would not reduce the list to the degree that would be expected under inclusion due to the influence of conservatism and the need to increase audit effort in a period of high risk. The results showed no significant interaction. Goal-framing effects were very similar at each level of risk--even in high risk. Thus, the goal-framing effect was sufficiently robust to overcome conservatism and the call for increased audit effort during high risk when these were needed. The implication of this finding is that caution should be exercised in a high-risk scenario if an inclusion frame is used. The original list may be reduced to a greater degree than desirable (because of the effect of the inclusion frame), as more explanations may deserve further consideration in the high-risk environment. Future research should test whether effectiveness of the analytical review is hindered in the high-risk/inclusion scenario.
A secondary finding of the study is additional information on auditor's consideration of hypotheses. Auditors using the decision aid created a set, on average, of about eight explanations--more than has been found possible relying on memory and the client (Hirst and Koonce 1996; Koonce 1993: Anderson et al. 1992). This finding suggests that decision aids may indeed be useful as a means for considering a greater number of plausible explanations.
This research is not without certain limitations. The external validity of a laboratory experiment is limited to the degree that it abstracts a scenario from the real world. As an experiment, this study placed auditors in a simulated audit environment without the ability to consult other information sources (e.g., other audit team member, firm policy, client, professional guidance, etc.). It is also important to note that this study examined only the judgment in the initial hypothesis acquisition of analytical review; it did not consider the iterations of information search and modification to the hypothesis set nor the final conclusion made by an auditor regarding the fluctuation. This may be viewed as a limitation because the study cannot definitively make conclusions regarding whether an auditor eventually reaches the "correct" diagnosis. This is an area outside the scope of this study and a topic for future research.
TABLE 1 Demographic Data for Subjects Panel A: Continous Measures Attribute Scale n Mean Auditing Experience Months 65 54.2 Frequency of Use of Ratio Analysis Likert-type (0-10) 65 5.1 Frequency of Analysis of Inventory Likert-type (0-10) 65 3.3 Turnover Ratio Months Since Experienced any Material Months 65 10.2 Accounting Error Number of Material Inventory Errors Number 65 0.9 during Past Two Years Months Since Experienced Material Months 65 10.0 Inventory Error Percentage of Time Involving Percentage 65 23.5 Manufacturing Clients Attribute Std Dev Auditing Experience 49.7 Frequency of Use of Ratio Analysis 3.0 Frequency of Analysis of Inventory 3.1 Turnover Ratio Months Since Experienced any Material 21.8 Accounting Error Number of Material Inventory Errors 1.6 during Past Two Years Months Since Experienced Material 23.0 Inventory Error Percentage of Time Involving 27.5 Manufacturing Clients Panel B: Discrete Measures Attribute Rank within Firm (a) Staff/ Senior/ Manager/ Partner/ n = 19 20 20 2 Firm (Office) I/ II/ III (A)/ III (B)/ IV (A)/ n = 9 6 9 10 22 Attribute Rank within Firm (a) Total/ 61 Firm (Office) IV (B)/ Total/ 9 65 (a)Four subjects did not indicate rank within the firm. TABLE 2 Analysis of Covariance for the Dependent Measure Reduced Set Size (a) Panel A: ANCOVA with Frequency Using Ratio Analysis as Covariate Sum of Source of Variation df Squares F P Goal Framing 1 425.500 23.00 <0.001 Client Risk 1 93.660 5.06 0.029 Goal Framing x Client Risk 1 7.630 0.41 0.523 Frequency Using Ratio Analysis 1 152.390 8.24 0.006 Error 54 998.910 Total 58 1744.750 Panel B: Adjusted Treatment Means Source Mean Std. Dev. n Goal Framing Inclusion 8.00 0.763 32 Elimination 13.44 0.832 27 Client Risk High Risk 11.78 0.803 29 Low Risk 9.25 0.788 30 Goal Framing x Client Risk Inclusion/High Risk 9.30 1.086 16 Inclusion/Low Risk 6.70 1.075 16 Elimination/High Risk 14.82 1.199 13 Elimination/Low Risk 12.16 1.150 14 Panel C: Distribution of Frequency Using Ratio Analysis in Treatment Cells (b) Source Mean Std. Dev. Inclusion/High Risk 5.79 3.052 Inclusion/Low Risk 5.00 2.989 Elimination/High Risk 4.46 2.817 Elimination/Low Risk 4.86 3.371 (a)Reduced Set Size is the number of explanations selected by inclusion participants or the number of explanations not eliminated by elimination participants. (b)Frequency Using Ratio Analysis is the participant's rating on a Likert-type scale with 0 labeled "Not Very Frequently" and 10 labeled "Very Frequently."
(1.) Client Risk as used in this study refers to the inherent and control risks of an auditee (i.e., the uncontrollable components of audit risk).
(2.) Decision makers have been observed applying a filtration strategy In a variety of decision settings (Payne et al. 1988: Ben Zur and Breznitz 1981: wright 1974), including accounting (Glover 1997; Spilker and Prawitt 1997).
(3.) The effects of conservatism were found to mitigate confirmatory strategies (Trotman and Sng 1989: Anderson 1988: Kida 1984b), neglect of base rate information (Kida 1984a), and insensitivity to source (Bamber 1983).
(4.) General auditing experience was tested various ways: (1) by using rank as a covariate in the analysis of variance, (2) by using months of experience as a covariate in the analysis of variance, (3) by using various groupings of participants according to rank as independent categorical variables in the analysis of variance, and (4) by using various groups of participants according to months of experience as Independent categorical variables in the analysis of variance. Prior literature has found performance differences between more and less experienced auditors in the analytical review task of sew-generating explanations due perhaps to the more complete memory structure of experienced auditors based on exposure to analytical review fluctuations (Libby and Frederick 1990; Bedard and Biggs 199la. 199lb; Wright and Wright 1997; Solomon et al. 1999). The task of self-generation is fundamentally different from this study's task of reducing a list of provided explanations (all plausible) in that there is little reliance on memory to generate plausible explanations. Therefore, more experienced auditors would not necessarily be expected to perform differently than less experienced auditors.
(5.) The risk manipulation was based on the risk manipulation in Anderson et al. (1997), which reviewed both academic and professional literature to identify factors associated with an increased probability of error in the financial statements (AICPA 1988: Albrecht et al. 1980: Albrecht and Romney 1986: National Commission on Fraudulent Financial Reporting 1987). The instrument was piloted for understandability and realism with auditors, and there were no comments indicating that the high-risk manipulation was too strong or unreal.
(6.) For elimination participants, in cases where there is a gap in likelihood assessments between explanations eliminated and those included (by default), two possible methods of establishing the likelihood criterion exist. One alternative Is to use the greatest likelihood from among explanations eliminated, while another is to use the lowest likelihood from among explanations included by default. The two methods were each evaluated and yielded similar results. The most conservative result (lowest F-statistic) is reported.
(7.) Other participant attributes were examined as covariates but were not significant.
(8.) A univariate analysis of variance verified the manipulation of client risk as the mean (standard deviation) riskiness" ratings were 8.58(0.93) and 3.48 (1.57) for the high-and low-risk groups. respectively. on a Likert-type scale of 0 to 10. These ratings are significantly different (F = 252.40, p = .000).
(9.) Of the 59 participants available for analysis, four were removed, as their criterion values were inconsistently applied as cutoff values to all 20 explanations evaluated. The first subject. for example, established a criterion value of 20 percent such that all explanations rated at least 20 percent likely by the participant should be in the reduced set. If the participant fails to include explanations she or he rates at least 20 percent likely, then the criterion value is invalid because it was not applied to explanations consistently.
(10.) In order to control for the total number of explanations in participants' reduced sets, a separate analysis was performed using the proportion of a participant's reduced set composed of en-or explanations as a dependent measure. Results of that analysis confirmed the previous finding.
(11.) A conclusion for the current study cannot be drawn directly from Bhattacharjee et al. (1999). however, as their participants lost some efficiency in generating a large number of explanations and participants in the current study were provided explanations by a decision aid, diminishing the "brainstorming" time required.
Albrecht, W. S., D. J. Cherrington, I. R. Payne, A. V. Roe, and M. B. Romney. 1980. Auditor involvement in the detection of fraud. In Management Fraud: Detection and Deterrence, edited by R. K. Elliott, and J. J. Willingham. Princeton, NJ: Petrocelli Books.
-----, and M. B. Romney. 1986. Red-flagging management fraud: A validation. Advances in Accounting: 323-334.
American Institute of Certified Public Accountants (AICPA). 1983. Audit Risk and Materiality in Conducting an Audit. Statement on Auditing Standards No. 47. New York, NY: AICPA.
-----. 1988. Analytical Procedures. Statement on Auditing Standards No. 56. New York, NY: AICPA.
Anderson, B. 1988. An examination of factors affecting auditors' hypothesis testing strategies. Doctoral dissertation, University of Massachusetts.
Anderson, J. C., S. E. Kaplan. and P. M. J. Reckers. 1992. The effects of output interference on analytical procedures judgments. AUDITING: A Journal of Practice & Theory 11 (Fall): 1-13.
-----, M. M. Jennings. S. E. Kaplan. and P. M. J. Reckers. 1995. The effect of using diagnostic decision aids for analytical procedures on judges' liability judgments. Journal of Accounting and Public Policy 14 (Spring): 33-48.
-----, S. E. Kaplan. and P. M. J. Reckers. 1997. The effects of interference and availability from hypotheses generated by a decision aid upon analytical procedures judgments. Behavioral Research in Accounting 9 (Supplement): 1-20.
Arkes, H. R 1981. Impediments to accurate clinical judgment and possible ways to minimize their impact. In Judgment and Decision Making: An Interdisciplinary Reader. edited by H. R. Arkes. and K. R. Hammond. Cambridge. MA: Cambridge University Press.
Asare, S. K. 1992. The auditor's going-concern decision: Interaction of task variables and the sequential processing of evidence. The Accounting Review 67 (April): 379-393.
-----, G. Trompeter. and A. M. Wright. 1997. The effect of accountability and time budget pressure on auditors' hypothesis testing strategy. Working paper, Boston College.
-----, and A. M. Wright. 1997. Hypothesis revision strategies in conducting analytical procedures. Accounting. Organizations and Society 22 (8): 737-755.
-----, and -----. 1998. The interdependence of hypothesis generation and Information search in diagnostic reasoning. Working paper, Boston College.
Baddeley, A. D. 1986. Working Memory. New York, NY: Oxford University Press.
Bamber, E. M. 1983. Expert judgment in the audit team: A source reliability approach. Journal of Accounting Research 21 (2): 396-412.
-----, P. R. Gillett. T. J. Mock, and K. T. Trotman. 1995. Audit judgment. In Auditing Practice, Research and Education: A Productive Collaboration, edited by T. B. Bell. and A. M. Wright. New York, NY: AICPA.
Bedard, J. C., and S. Biggs. 1991a. Pattern recognition, hypotheses generation, and auditor performance in an analytical task. The Accounting Review 66 (3): 622-642.
-----, and -----. 1991b. The effect of domain-specific experience on evaluation of management representations in analytical procedures. AUDITING: A Journal of Practice & Theory 10 (Supplement): 77-90.
Bell, T., W. R. Knechel. and J. Willingham. 1994. An exploratory analysis of the determinants of audit engagement resource allocations. Proceedings of Deloitte & Touche/University of Kansas Symposium on Auditing Problems. 49-67. Lawrence. KS: University of Kansas.
Ben Zur, H., and S. J. Breznitz. 1981. The effects of time pressure on risky choice behavior. Acta Psychologica (February): 89- 104.
Bhattacharjee, S., T. Kida. and D. M. Hanno. 1999. The impact of hypothesis set size on the time efficiency and accuracy of analytical review judgments. Journal of Accounting Research (Spring): 83-100.
Biggs, S., T. Mock, and P. Watkins. 1988. Auditor's use of analytical review in audit program design. The Accounting Review (January): 148-161.
Davidson, R., and W. Gist. 1996. Empirical evidence on the functional relation between audit planning and total audit effort. Journal of Accounting Research (Spring): 111-124.
Glover, S. M. 1997. The influence of time pressure and accountability on auditors' processing of nondiagnostic information. Journal of Accounting Research (35) 2: 213-226.
Hirst, D. E., and L. Koonce. 1996. Audit analytical procedures: A field investigation. Contemporary Accounting Research 13 (Fall): 457-486.
Huber. V. L., M. A. Neale, and G. B. Northcraft. 1987. Decision bias and personnel selection strategies. Organizational Behavior and Human Decision Processes (40): 136-147.
Jamal, K., P. E. Johnson, and R. G. Berryman. 1995. Detecting framing effects in financial statements. Contemporary Accounting Research 12 (Fall): 85-105.
Jaradat, D., and N. Tollefson. 1988. The impact of alternative scoring procedures for multiple-choice items on test reliability, validity, and grading. Educational and Psychological Measurement 48: 627-635.
Johnson, P. E., K. Jamal, and R. G. Berryman. 1991. Effects of framing on auditor decisions. Organizational Behavior and Human Decision Processes 50: 75-105.
Just, M. A., and P. A. Carpenter. 1992. A capacity theory of comprehension: Individual differences in working memory. Psychological Review (January): 122-149.
Kida, T. 1984a. The effect of causality and specificity on data use. Journal of Accounting Research (22) 1: 145-152.
-----. 1984b. The impact of hypothesis testing strategies on auditors' use of judgment data. Journal of Accounting Research 22 (1): 332-340.
Koonce, L. 1993. A cognitive characterization of analytical review. AUDITING: A Journal of Practice & Theory 12 (Supplement): 57-76.
Levin, I. P., J. D. Jasper, and W. S. Forbes. 1998. Choosing versus rejecting options at different stages of decision making. Journal of Behavioral Decision Making 11:193-210.
Libby, R., and D. M. Frederick. 1990. Experience and the ability to explain audit findings. Journal of Accounting Research (Autumn): 348-367.
Lichtenstein, S., and P. Slovic. 1973. Reversals of preference between bids and choices in gambling decisions. Journal of Experimental Psychology (89): 46-55.
Medin. D. L., R. L. Goldstone, and D. Gentner. 1990. Similarity involving attributes and relations: Judgments of similarity and difference are not inverses. Psychological Science 1 (1): 64-69.
Miller, J. G. 1960. Information input overload and psychopathology. American Journal of Psychiatry (February): 695-704.
Mock, T., and A. Wright. 1999. Are audit program plans risk-adjusted? AUDITING: A Journal of Practice & Theory 18(1): 55-76.
National Commission on Fraudulent Financial Reporting (the Treadway Commission). 1987. Report of the National Commission on Fraudulent Financial Reporting. Washington, D.C.: Government Printing Office.
Nesca, M. 1997. Seeking a middle ground on the recoverability of memories controversy. American Journal of Forensic Psychology 15 (3): 3-22.
-----, W. Bettman, and E. Johnson. 1988. Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition (July): 534-552.
Prosansky, C., and I. P. Levin. 1999. Inclusion and exclusion in prescreening options for a positive and negative decision task. Working paper, The University of Iowa.
Smith. J. F., and T. Kida. 1991. Heuristics and biases: Expertise and task realism in auditing. Psychological Bulletin 109 (3): 472-489.
Solomon, I., M. D. Shields. and O. R. Whittington. 1999. What do industry-specialist auditors know? Journal of Accounting Research (Spring): 191-208.
Spilker, B. C., and D. F. Prawitt. 1997. Adaptive responses to time pressure: The effects of experience on tax information search behavior. Behavioral Research in Accounting 9: 172-198.
Trotman. K., and J. Sng. 1989. The effect of hypothesis framing, prior expectations and cue diagnosticity on auditors' information choice. Accounting, Organizations and Society 14 (5/6): 565-576.
Westenberg, M. R. M., and P. Koele. 1992. Response modes, decision processes and decision outcomes. Acta Psychologica 80: 169-184.
Wright. A., and J. Bedard. 2000. Decision processes in audit evidential planning: A multistage investigation. AUDITING: A Journal of Practice & Theory 19 (Spring): 123-143.
Wright. P. 1974. The harassed decision maker: Time pressures, distractions and the use of evidence. Journal of Applied Psychology (October): 555-561.
Wright, S., and A. Wright. 1997. The effect of industry experience on hypothesis generation and audit planning decisions. Behavioral Research in Accounting (9): 273-294.
Yaniv, I., and Y. Schul. 1997. Elimination and inclusion procedures in judgment. Journal of Behavioral Decision Making 10: 211-220.
-----, and -----. 2000. Acceptance and elimination procedures in choice: Noncomplementarity and the role of implied status quo. Organizational Behavior and Human Decision Processes 82: 293-313.
We thank each of the following for helpful comments in the preparation of this paper: Barry J. Epstein, Kimberly K. Moreno, Reza Barkhi, Robert M. Brown, Robert Williges, and participants of research seminars at virginia Polytechnic Institute & State University, Auburn University. The University of Tennessee, and Florida State University. The paper has also benefited from the comments of an anonymous reviewer and the editor. We are grateful for the support provided by the John E. Peterson Doctoral Research Fellowship.
|Printer friendly Cite/link Email Feedback|
|Author:||Mueller, Jennifer M.; Anderson, John C.|
|Publication:||Behavioral Research in Accounting|
|Date:||Jan 1, 2002|
|Previous Article:||Investors' Use of Analysts' Recommendations.|
|Next Article:||Evidence of an Association between Error-Specific Experience and Auditor Performance during Analytical Procedures.|