An Investigation of Multidimensional Knowledge Structure and Computer Auditor Performance.
This study assesses the internal control knowledge structure dimensions and antecedents, as well as the relationship between structure and performance, for a set of computer auditors. The study extends prior research in two ways. First, in an effort to investigate the impact of multiple knowledge structures on auditor performance, a multidimensional model is employed through the use of multidimensional scaling (MDS). An added benefit of this methodology is that it facilitates the evaluation of the extent of auditor knowledge structure. Second, a more complete understanding of the influence of experience and education on auditor Performance is gained through investigating the role of knowledge structure as a mediator.
Results suggest that computer auditors employ both control objective and transaction flow structural dimensions simultaneously, as well as a third dimension not previously identified, which represents the degree of computerization of a particular procedure. Transaction flow was identified as the dominant structural dimension for most computer auditors.
Computer auditor knowledge structures were found to be associated with college education and prior computer information systems (CIS) work experience. The extent (weight) of auditors' control objective and transaction flow dimensions was related to their performance on an internal control review task. However, no significant interactions were found.
Key Words: Knowledge structure, Internal control, Computer auditor, Multidimensional scaling, Performance.
Data Availability: Contact first author.
In external auditing, the evaluation of internal control is required for purposes of assessing control and fraud risk, as well as for planning related substantive tests (AICPA 1995, 1983, 1979). A significant body of research has explored the ways in which auditors perform internal control reviews and the individual factors required for expert performance at this task (Bonner and Pennington 1991). In this literature, one determinant of expert performance is auditor knowledge structure (Bedard and Chi 1993; Libby and Luft 1993). Knowledge structure is thought to impact task performance through its role in the storage and recall of knowledge, as well as in the formation of problem representations.
Research has shown that auditors can develop different knowledge structure patterns through training (Bonner et al. 1997). Additionally, higher performance is associated with a match between dominant knowledge structure and task structure (Nelson et al. 1995). However, an interesting characteristic of knowledge structure is that one person can simultaneously possess multiple patterns of organization of their knowledge (Ford 1995; DeJong and Ferguson-Hesler 1986; Frederick et al. 1994; Kintsch and Greeno 1985). For example, internal control knowledge can be organized in different ways, such as by transaction flow or by control objective. If a person's internal control knowledge has both patterns, then either pattern has the potential to be accessed and retrieved, with the dominant pattern being the most likely one retrieved first.
While prior studies have recognized the multidimensional nature of accountant knowledge structure (Ingraham and Reneau 1999; Bonner et al. 1997; Frederick et al. 1994; Bonner and Pennington 1991; Choo and Trotman 1991; Pei and Reneau 1990), those studies have looked at only one dimension for each experimental participant at a time. (1) If access and retrieval of a single knowledge structure is sufficient for acceptable job performance, then audit-training programs should reinforce the identified structure. However, if multiple knowledge structures are concurrently associated with higher job performance, then audit-training programs should provide a multidimensional presentation of audit knowledge. Existing research has not addressed the impact of accessing and retrieving multiple knowledge structures during performance of tasks. The primary goal of this study is to further investigate the link between knowledge structure and performance in the evaluation of internal control, specifically in the evaluation of the relevance of multiple structural dimensions on auditor performance.
A supplemental goal of this study is to extend previous research regarding the antecedents of auditor knowledge structure. While it is assumed that knowledge structure is developed through education and experience, research to date has neither investigated the differential impact of varying education and experience in diverse disciplines, nor has it evaluated whether educational and experiential impacts have lasting effects. This is important because the backgrounds of auditors are becoming more diverse. For example, the 150-hour rule has resulted in a variety of formulas for undergraduate and graduate accounting education; the increase in non-traditional students in accounting has led to an increase in the average age of accounting firm new-hires (and thus greater experience in other areas prior to joining public accounting); and recent increases in demand for both entry-level and senior-level auditors has influenced accounting firms to expand their employment searches to areas not previously considered, such as internal auditing.
Auditors with differing educational and work experiences may bring diverse and, in some cases highly developed, knowledge and related knowledge structures to the firm training program (Butt 1988). It is likely that, despite common firm training and audit approaches, the dissimilar pre-employment backgrounds and post-employment experiences of auditors may induce differences in knowledge structure, resulting in performance disparities (Vera-Munoz 1998; Libby and Frederick 1990). Findings on this issue are important for employment selection and the design of training programs.
In exploring antecedents of knowledge structure, it is useful to have a diverse sample of auditors. Because diversity of employment for financial auditors is a relatively new phenomenon, we have chosen computer auditors as the subjects of this study for four reasons. First, the review of internal control represents a significant portion of their job responsibilities. Second, computer auditors receive the same firm training for application control evaluation (albeit more technical, perhaps), and they use the same decision aids and audit approaches as financial auditors. Third, their backgrounds have always been varied (2) and so present the rich array of influences on structural development, at all levels, that the pool of financial auditors is expected to exhibit in the future (Boyton and Kell 1996; Champlain 1995). Fourth, and finally, little research has addressed computer auditors' knowledge base, despite their increasing importance to audit practice.
THEORY AND HYPOTHESES
Internal Control Knowledge Structure Dimensions for Computer Auditors
Prior research has found that auditors employ three primary patterns for organizing and accessing knowledge: control objective, transaction flow, and transaction cycle (Bonnet et al. 1997; Borthick et al. 2000; Frederick 1991; Nelson et al. 1995; Tubbs 1992; Weber 1980). Although the basic concepts of control objectives (for example, completeness, accuracy, and authorization) and transaction flow (for example, input, processing, and output) remain constant across cycles, the ways in which control objectives are achieved and transactions are processed differ between cycles. Thus, control objective and transaction flow are fundamentally different patterns for organizing the same internal control knowledge within any given cycle. Frederick (1991) explained that these patterns correspond to two cognitive theories of knowledge organization: taxonomic organization for control objective and schematic organization for transaction flow.
The ability to organize internal control knowledge in both of these patterns appears to be critical to auditor appraisal of internal controls. On one hand, accounting systems are constructed in a transaction flow (TF) organization and documentation is often presented in this way using narratives, flowcharts, and other presentation tools (Bodnar and Hopwood 1998; Romney et al. 1997). Therefore, it seems reasonable that the extent to which auditors can retrieve internal control knowledge in transaction flow structure impacts their ability to map details from the current client context, including related internal controls, to their existing knowledge and its structure in order to build a mental representation of that system (Christ 1993; Plumlee 1985). The adequacy of this mental representation should impact the effectiveness of the auditor evaluation of that system of internal controls (Borthick et al. 2000). On the other hand, many firms have restructured their internal control questionnaires to rate control adequacy according to control objectives (CO), which map to financial statement assertions (Borthick et al. 2000). Auditor ability to organize internal control knowledge in a control objective pattern should enhance their ability to assess the extent to which control objectives are met or critical financial statement assertions are at risk. Thus, because the evaluation of internal control requires computer auditors to apply their internal control knowledge in both transaction flow and control objective perspectives, experienced computer auditor internal control knowledge organization is expected to include both of these dimensions. (3)
H1: Experienced computer auditors' application control knowledge structure includes both control objective and transaction flow dimensions.
A second issue concerning knowledge structure is the identification of the dimension auditors tend to employ most frequently, the "dominant dimension." A thorough understanding of dominant dimensions is important because mismatches between knowledge and task structure may result in reduced performance (Nelson et al. 1995).
In previous research, Frederick (1991) found that, when asked to free-sort control procedures, experienced auditors used a control objective pattern only slightly more often than a transaction flow pattern (55 percent used CO, while 45 percent used TF). In a similar free-sort task, Frederick et al. (1994) found that experienced auditors use control objective as the primary or dominant dimension to structure financial-statement-error knowledge. Nelson et al. (1995) also found that, on average, the audit objective dimension was dominant for experienced auditors, and Tubbs (1992) found that auditors become more aware of audit objectives as they gain experience. Conversely, Borthick et al. (2000) found that senior accounting students used transaction cycle as their dominant structural dimension. Thus, it appears that accountants, employed directly out of school, enter the profession with transaction cycle structures and are gradually acclimated to the control objective approach to internal control evaluation, first through firm training and then through ongoing experience with a control objective approach to the task.
However, what is not known is, first, how strongly experience and education received prior to firm training resist conversion to the control objective way of structuring knowledge and, second, what impact experience received after initial firm training has on structural development. Thus, an important step to understanding auditor knowledge structure is to determine whether experienced auditors, with similar firm training yet dissimilar education and experience, exhibit their firm-preferred structure as the dominant dimension for their internal control knowledge. Absent prior research to the contrary, we assume:
H2: The majority of computer auditors will exhibit a dominant dimension equivalent to their firm training and decision-aid structure.
Relationship between Auditor Knowledge Structure and Performance
Prior accounting research regarding knowledge structure and performance has taken two different approaches: studies classifying subjects as to which structural dimension is dominant, and those measuring the extent of subjects' structure. The former studies have emphasized the impact of dimensional classification on performance. For example, Nelson et al. (1995) and Bonner et al. (1996) found that performance may be diminished when the structure of the task does not match the dominant structure employed by the subject. These studies evaluated auditors' ability to apply error frequency knowledge (stored in audit objective structure) in the allocation of hours in an audit plan (a task requiring a transaction-cycle orientation).
Research measuring the extent of individual structure has evaluated the relationship between amount of structure and performance, and has generally found a positive association between these two factors. For example, Choo and Trotman (1991) found that the clustering of recall on the basis of typical/atypical items was significantly correlated with experienced auditors' inferences ("filling in the blanks") about a current problem context. Pratt (1982) found the extent of students' knowledge structure to be positively related to prediction accuracy when using complex annual reports. Borthick et al. (2000) found that greater structure, along either of the trained dimensions, was associated with better performance on an internal control task.
This line of research suggests that more enhanced internal control knowledge structure, in dimensions relevant to the review of internal controls, would result in greater performance in the relevant task. We have argued previously that transaction flow structure is instrumental in the construction of a mental representation of accounting systems, and control objective structure aids the evaluation of controls contained in that representation. Therefore, we expect the extent of both structural dimensions to be positively related to auditor performance in the review of internal controls.
H3: Greater (lesser) structure for either control objective or transaction flow dimensions is associated with greater (lesser) performance in the review of internal controls.
Antecedents of Computer Auditor Internal Control Knowledge Structures
The psychology literature has proposed that both education and experience influence the organization of knowledge (Alba and Hasher 1983). Christ (1993) and Chi et al. (1989) have found that education provides the initial foundation for structuring knowledge, while the findings of Bonner et al. (1997), DeMarie-Dreblow (1991), and others support the notion that differing education leads to different forms of knowledge structure. Frederick et al. (1994) also posited that experienced auditors' knowledge organization is affected by the structures presented both in college and professional training materials.
Education and Knowledge Structure
Some researchers question whether educational influences on auditor knowledge structure can be detected after auditors have gained a certain amount of experience. However, cognitive psychology theory suggests that the initial structure provided by education should give auditors a foundation upon which later experiences build, resulting in those later experiences having a greater impact on the extent of knowledge and its structure than for those with less initial foundation (West et al. 1985). Alba and Hasher (1983) assert that the critical condition for the acquisition of new knowledge is the existence of previously acquired relevant knowledge and that, in the absence of such knowledge, memory is generally poor. Thus, if prior exposure to a set of concepts improves ability to remember and understand the concepts, then education that includes exposure to a set of concepts should have a positive influence on the further development of that knowledge when later re-exposed to the concepts.
Since accounting students are exposed to both transaction flow and control objective concepts in their classes, it is posited that computer auditors with relatively more accounting education would also have greater initial foundations upon which to build control objective and transaction flow structure through training and experience. Computer information systems (CIS) education should similarly influence individuals' transaction flow knowledge structure, since these classes expose students to concepts related to the flow of data and the design of procedures for computer applications. If education continues to influence knowledge structure beyond its initial exposure, we would expect to find that, for experienced auditors, more education in these areas is associated with greater structure for the two dimensions.
H4a: Computer auditors' college semester hours in accounting are positively related to the extent of transaction flow knowledge structure dimension.
H4b: Computer auditors' college semester hours in accounting are positively related to the extent of control objective knowledge structure dimension.
H4c: Computer auditors' college semester hours in CIS are positively related to the extent of transaction flow knowledge structure dimension.
Experience and Knowledge Structure
Chi and Rees (1983) note that, as individuals gain experience at domain-related tasks, they add new factual knowledge to existing knowledge structure, add to or change the perceived relation between knowledge elements, and change the way knowledge elements are categorized. Computer auditors may have two relevant work experiences that impact their knowledge structures: auditing and CIS.
Experience in the computer-auditing field should influence the development of both transaction flow and control objective knowledge structures. In a card-sort task, Frederick et al. (1994) found that experienced auditors had more extensive control objective structure than did less experienced auditors, while, in a recall task employing cophenetic correlations, Frederick (1991) found that experienced auditors had relatively greater structure for both transaction flow and control objective structures than did less experienced auditors. If experience in computer-audit provides similar influence on knowledge structure as financial audit, we expect those with more computer-audit experience to more strongly exhibit both of these structures.
Just as education in CIS is thought to make individuals more aware of the flow of data through a computer system, experience in the CIS field should help computer auditors refine their knowledge of transaction flow and give them practice in the design and development of transaction processing systems. Therefore, we expect computer auditors with more CIS-related experience prior to joining the audit profession to structure their knowledge in the transaction flow dimension.
H5a: Computer auditors' years of computer-audit work experience are positively related to the extent of their transaction flow knowledge structure dimension.
H5b: Computer auditors' years of computer-audit work experience are positively related to the extent of their control objective knowledge structure dimension.
H5c: Computer auditors' years of CIS-related work experience are positively related to the extent of their transaction flow knowledge structure dimension.
Fifty-eight computer auditors, employed by a single Big 6 public accounting firm, participated in this study. The participants worked in several offices of various sizes across the country. See Table 1 for a summary of subject demographic characteristics. Forty-four auditors completed the research tasks during controlled experimental sessions and the remainder completed the tasks as survey instruments. (4) This study included auditors from only one firm in order to control for differences in firm employment and training, as well as variation in relevant task structure and design.
TABLE 1 Participant Demographics Average Low High Experience within firm: years 3.8 0.20 18.0 Computer audit experience: years 4.3 0 25.0 Number of application control reviews performed 23.0 0 100.0 CIS work experience: years 2.0 0 12.0 Accounting education: semester hours 17.2 0 54.0 CIS education: semester hours 21.0 0 60.0 Number Level: Staff 21 Seniors 16 Managers 22 CISA certification 18 CPA certification 16
In the identification of stimuli for use in the study, our goal was to build a set of concepts that could be classified along a number of dimensions. Prior studies have found that auditors tend to structure their internal control knowledge in identifiable patterns, such as the organizational schemes used in training manuals and decision support tools (Frederick 1991; Weber 1980). Thus, we looked at empirical research in auditing, as well as other sources including computer auditor training and firm policy manuals, for likely structural dimensions. A set of nine application control concepts, which could reasonably be classified along any of the likely dimensions, was identified. These are described in Figure 1. Note that, as the number of concepts increases, the number of all possible pairs for paired comparisons increases geometrically ((n * (n - 1)) * 2). One limitation of this methodology, therefore, is that the number of concepts that can be compared is limited, while maintaining the interest and attention of participants. Two very experienced computer-audit specialists, described at a later point, determined that this was a reasonable subset of controls to employ.
FIGURE 1 Application Control Procedures 1. Document Numbers (DN): Users review a sequential list of input document numbers to identify missing records or incorrect document numbers. 2. MasterFile Balances (MB): Run-to-run totals are produced and automatically verified for important numeric master file fields, such as account balances, to help ensure accuracy of update. 3. Confidential Reports (CR): Extremely sensitive reports are automatically delivered to users electronically via EMail. 4. Generated Transactions (GT): Automatic journal entries or other generated transactions are reviewed against supporting information by users. 5. Key Fields (KF): Editing and reasonableness checks are performed on key fields during initial processing. 6. Receipt of Reports (RR): The list of users authorized to receive reports is verified periodically and users are queried as to actual receipt of all reports produced. 7. Validation Rejects (VR): Transactions rejected by data validation routines are automatically placed in suspense for corrective action. 8. Completeness of Processing (CP): Users compare record counts generated during update processing to input control totals to ensure completeness of production processing. 9. Completeness of Output (CO): All output is reviewed for completeness by the data control group.
The participants were first asked to read definitions of the internal controls in the stimuli set, perform a related matching test, and then check their responses with an answer key. (5) The purpose of the test was to encourage the auditors to thoroughly familiarize themselves with the way in which the controls were implemented in this study and to eliminate individual differences related to varying definitions of the concepts. The participants next performed a knowledge elicitation exercise in which they compared all possible pairs of the application control concepts (36 pairs), rating the pairs as to their relative similarity on a scale of 1 to 9. The criterion for judging similarity was intentionally unstated, thus allowing the participants to use their own "perceptual frameworks" to evaluate the concepts (Libby 1979). Subjects were randomly assigned to one of two versions of the elicitation instrument, which differed only in presentation order of the rating pairs. The auditors then performed an internal control review task, in which they first read a case describing a highly computerized disbursement accounting system and its related internal controls, and then identified internal control weaknesses and possible financial statement errors resulting from these weaknesses. Finally, the auditors completed a demographics questionnaire containing detailed questions regarding their education and experience. (6)
Analysis Using Multidimensional Scaling
Multidimensional Scaling (MDS) is a "factor-analysis-like technique" (Bailey et al. 1983) that produces spatial representations of similarity judgments (Schvaneveldt et al. 1985). The assertion that these representations depict knowledge structure dimensions is based on the notion that concepts in memory differ in their relatedness or psychological proximity (Collins and Loftus 1975) and that relatedness judgments by individuals indicate organizing structures for these concepts (Cooke et al. 1986). MDS has often been employed in psychology (Markman and Gentner 1996; Mallins and Kimbrough 1988), marketing (Droge and Darmon 1987; Moore and Lehmann 1982) and management (Boal and Perry 1985) research and, less frequently, in accounting research (Tubbs 1992; Pratt 1982; Libby 1979).
Input to MDS is constructed by converting individual judgments regarding the similarity (psychological proximity) of all possible pairs of a set of concepts to symmetrical n-by-n matrices. As depicted in Figure 2, the auditors' paired comparisons were converted to 9 x 9 matrices that served as input to weighted MDS analysis. Note, for example, the first matrix shown in Phase 1 for Figure 2; the data in the matrix indicates the subject S1 rated the relative difference between the concepts DN and KF to be 3 on a scale of 1 (very similar) to 9 (very dissimilar), while they perceived the relative difference between MB and CO to be 5 on the scale of 1 to 9.
FIGURE 2 Multidimensional Scaling Depiction of Analytical Process for Identification of Structural Dimensions Phase 1: Individual subjects' similarity ratings are formed into matrices, which serve as input to MDS. S1 DN VR KF CP MB GT CO RR CR DN 0 VR 2 0 KF 3 2 0 CP 9 3 5 0 MB 7 5 6 2 0 GT 6 8 9 9 3 0 CO 3 2 4 4 5 2 0 RR 9 8 7 6 6 2 1 0 CR 4 5 6 5 6 7 7 8 0 S2 DN VR KF CP MB GT CO RR CR DN 0 VR 9 0 KF 7 8 0 CP 2 8 4 0 MB 2 5 2 2 0 GT 9 5 2 2 2 0 CO 9 8 7 6 5 4 0 RR 5 3 7 8 8 4 2 0 CR 5 4 3 2 1 2 7 7 0 etc. Phase 2: MDS first identifies group dimensions for all matrices combined. Dim 1 Dim 2 Dim 3 DN 0.84 -1.08 0.53 VR 0.63 0.97 -1.36 KF 0.52 1.78 0.20 CP 0.55 -0.58 0.77 MB 0.78 0.74 0.77 GT 0.11 0.75 0.33 CO 0.21 -0.84 0.55 RR -1.77 -0.88 0.45 CR -1.88 -0.90 -2.24 Phase 3: MDS then computes dimensional weights for each subject on each group dimension. Subject Dim 1 Dim 2 Dim 3 S1 1.98 1.54 0.04 S2 1.52 0.43 0.72 S3 1.58 0.68 0.21 S5 1.61 0.45 0.41 S6 1.10 0.86 1.07 S7 0.88 0.44 0.49 S8 1.39 0.79 0.67 S9 1.55 0.23 0.78 S10 1.19 1.17 0.45 S21 1.11 1.31 0.29 S22 1.20 0.96 1.03 S23 0.32 1.09 1.30 etc. See Figure 1 for abbrevations.
The SAS MDS procedure (SAS Institute Inc. 1992), employing the Euclidean distance metric, was instructed to search for three dimensions and to treat the data as continuous. Various authoritative sources regarding MDS analysis (Dillon and Goldstein 1984; Kruskal and Wish 1978) suggest that three is the maximum number of dimensions that can be reliably measured from the paired comparison of nine concepts. MDS identifies dimensions common to all input matrices and depicts a dimension by indicating the relative position of each concept along a continuum or vector. The labeling of identified dimensions is solely the responsibility of the researcher and is facilitated by the use of a visual representation of the dimension, such as a graph. By considering the concepts at the extremes of the dimensions, and by evaluating the order and groupings of items along the continuum, the researcher should be able to match the graphed dimension with a particular characteristic of the concepts.
Knowledge Structure Dimensions
To determine what knowledge structure dimensions computer auditors exhibit, the paired comparison matrices were analyzed with MDS. For comparison purposes, the dimensions for the staff-level participants and for the manager-level participants were determined separately. This contrasted subjects at the lowest and highest levels of experience, excluding, for now, the middle-level seniors. The staff group had an average of one year of computer-audit experience and had performed an average of eight application control reviews, while the manager group bad an average of 7.5 years of computer-audit experience and had performed an average of 45 application control reviews. Their application control dimensions are depicted in Figures 3 and 4. (Abbreviations are explained in Figure 1.)
[FIGURES 3-4 OMITTED]
In analyzing input from multiple subjects, MDS can identify dimensions that fit pooled data for that subject group. In the current study, separate MDS analyses were performed on pooled paired-comparison matrices from two subject groups in order to contrast those of relatively more and less experience: managers and staff. The analyses identified three distinct application control dimensions for each group. The vector for each identified dimension (depicted as the x-axis in Figures 3 and 4) is defined by the unique positioning of the nine application control concepts on that vector. The dimensional vector to emerge first is considered the most dominant dimension for a particular subject group. Dimensional vectors can vary in length, with longer vectors possessing greater distinction in the grouping of concepts. There is no pre-defined maximum or minimum vector length.
We initially labeled the dimensions identified by MDS based on our knowledge of internal controls. We then asked two highly experienced computer-audit specialists, (7) who had not participated in the experiment and who had no prior knowledge of the MDS results, to sort the control procedures based on any dimensions they thought appropriate. For each dimension, visual comparison suggests close agreement between the way the highly experienced auditors organized the concepts, for each dimension and the managers' structural dimensions. For example, both highly experienced auditors identified control objective as a dimension they thought computer auditors should perceive. Within this dimension, they identified the categories of Accuracy, Authorization, and Completeness, with one adding the category of Confidentiality.
In evaluating the first manager dimension, shown in Figure 3a (of Figure 3), the elements appear to be arranged along a continuum according to the flow of processing. At one end of the continuum are input procedures: Key Fields, Document Numbers, and Validation Rejects. In the middle are processing procedures: Completeness of Processing and Master File Balances. At the other end of the continuum are output procedures: Completeness of Output, Generated Transactions, Receipt of Reports, and Confidential Reports. This dimension was labeled "Transaction Flow."
The elements in the second dimension, shown in Figure 3b, appear to be ordered according to the control objective they satisfy. The controls related to accuracy occur first: Key Fields, MasterFile Balances, and Generated Transactions. The second set of controls on the continuum is related to completeness. The first control in this category, Validation Rejects, is primarily a completeness control, although its positioning suggests that managers consider it to be related somewhat to accuracy, also. The next two controls on the continuum, Completeness of Processing and Completeness of Output, also satisfy the completeness control objective. The third category on this dimension is validity or authorization. Document Numbers is primarily a validity/authorization control, although its positioning to the left of the other controls in this category indicates that managers may consider it to satisfy, to some extent, the objective of completeness. The remaining elements on the continuum, Confidential Reports and Receipt of Reports, satisfy the validity/authorization objective.
In the third dimension, depicted in Figure 3c, the elements appear to be aligned according to whether they are automated or manual procedures. At one end of the continuum are controls that are a blend of manual and automated actions (based on the definition of the concepts contained in the experimental materials): Completeness of Processing and MasterFile Balances. Next to these are elements that are completely manual procedures: Completeness of Output, Document Numbers, Receipt of Reports, and Generated Transactions. These are followed by automated control procedures: Key Fields, Validation Rejects, and Confidential Reports. This dimension was labeled "Control Method" (manual vs. automated).
In summary, we found transaction flow (TF), control objective (CO), and control method (CM) dimensions in the computer audit managers' application control knowledge structures. Two of these dimensions, CO and TF, are those identified in financial-audit research (Frederick 1991; Weber 1980). This supports HI, which proposes that experienced computer auditors possess both transaction flow and control objective knowledge structures.
The staff auditors' dimensions, depicted in Figure 4, were generated next in order to facilitate the comparison of the knowledge structures of more experienced and less experienced computer auditors. The staff auditors' first dimension, Figure 4a, presents the application control elements in roughly the same groupings as the managers' second dimension, labeled Control Objective. For example, Key Fields, Generated Transactions, and MasterFile Balances, presented together at one end of the continuum, satisfy the control objective of accuracy. Validation Rejects is included in this grouping, suggesting that staff auditors consider Validation Rejects to be related to accuracy. The completeness controls, including Completeness of Output and Completeness of Processing, are grouped together, with Document Numbers closely following. The final grouping, validity/authorization, includes Receipt of Reports and Confidential Reports. Document Numbers is presented between completeness and validity/authorization controls, suggesting that staff auditors consider this control to be related to both objectives. The identification of this dimension as a control objective is supported by computing the correlation between the managers' second dimension and the staffs first dimension. The Pearson's Product Moment correlation coefficient was 0.737 (p < .02).
The second staff dimension appears to be organized according to control method, the managers' third dimension. As Figure 4b shows, the manual controls of Generated Transactions, Completeness of Output, Document Number, and Receipt of Reports are grouped together to the left of the continuum. The automated controls of Validation Rejects, Key Fields, and Computer Reports are grouped together to the right. The two controls that managers grouped separately, apparently because they were a mix of manual and automated procedures, included Completeness of Processing and MasterFile Balances. Staff auditors grouped these differently, including Completeness of Processing with manual procedures and MasterFile Balances with automated procedures. The identification of this dimension as control method is supported by computing the correlation between the managers' third dimension and the staff's second dimension (Pearson Product Moment correlation coefficient: 0.699; p < .04).
The identification of the staff auditors' third dimension is more problematic. Correlating it with the managers' three dimensions yielded correlation coefficients of 0.50, 0.37, and 0.37, respectively. The fact that the dimension is approximately equally correlated with all three of the managers' dimensions suggests that there is a poor match between this dimension and any of the managers' dimensions. A qualitative review of the dimensional graph, presented in Figure 4c, suggests the dimension may resemble the managers' transaction flow dimension layout, although two controls are misplaced on the continuum: Generated Transactions is an output procedure that is placed toward the front in this dimension and Validation Rejects is an input procedure that is placed at the end of this dimension.
In summary, staff auditors appear to possess distinct control objective and control method knowledge structures, although their arrangement of control procedures along these dimensions differs from the managers' arrangement in several respects. The staff auditors' third dimension is less distinct, although it may arguably be considered to be arranged in transaction flow organization.
Measuring Extent of Structure
All further research questions require a quantitative measure of the extent of each individual's structure. To achieve this, MDS was employed to compute dimensional weights for all 58 subjects, including auditors at the manager, senior, and staff levels. Dimensional weights indicate the degree to which each individual's paired comparison data incorporates each identified dimension and are considered to depict the extent to which each individual "perceives" each dimension (Kruskal and Wish 1978). First, in order to determine all participants" weights on a set of meaningful dimensions, the most experienced subjects were identified and their MDS input was analyzed. The dimensions extracted from this pool of subjects became our "gold standard." (8) The MDS input of all participants was then compared against the "gold standard" dimensions. (9) Descriptive statistics for dimension weights of the 58 subjects are presented in Table 2. (10)
TABLE 2 Descriptive Statistics for Internal Control Dimensions for Sample of 58 Computer Auditors Correlation between Dimensions Percent of Mean of Sample with Dimension Dimension Dimension Weights Dim 2 Dim 3 as Dominant (1) Transaction Flow 1.41 -.197 -.772 64 (2) Control Objective 0.51 -.191 25 (3) Control Method 0.52 11 Average Computer Audit Percent of Experience When Sample with Dimension is Dimension Dimension Dominant as Secondary (1) Transaction Flow 4.7 14 (2) Control Objective 3.5 46 (3) Control Method 2.9 40
Hypothesis 2 relates to the dimension most salient or "'dominant" for computer auditors' internal control knowledge. By identifying which dimension carries the greatest weight for each individual, the most dominant dimension for each individual can be identified (Dillon and Goldstein 1984). Table 2 depicts the distribution of the dimensions among participants. As this table indicates, transaction flow was the dominant dimension for 64 percent of the sample. Hypothesis 2 predicted that the subjects would possess dominant dimensions similar to their firms' approach to internal control. Thus, control objective was expected to be the dominant dimension for the majority of computer auditors' internal control knowledge. Hypothesis 2 is, therefore, not supported.
Knowledge Structure Dimensions and Performance
Hypothesis 3 predicted a positive association between both transaction flow and control objective dimensions and auditor performance. To measure performance in an internal control review task, the case required participants to identify weaknesses in a system of internal controls, and errors that could result from those control weaknesses. Having subjects identify both weaknesses and errors was considered important because these two types of evaluations are often performed by auditors at different experience levels. Libby and Frederick (1990) assert that the actual procedures in an audit program are typically performed by less experienced auditors, while explaining and interpreting the implications of those results for further testing and possible audit adjustment is usually left to more seasoned auditors. Therefore, the possibility exists that structural dimensions differ in their relationship to the various performance tasks (Tubbs 1992).
Five performance measures (two quantitative and three qualitative) were created (Bonner et al. 1992) from participant responses to the internal control review case. The first step in creating these variables was the compilation of the complete set of all weaknesses and errors identified by participants. The two highly experienced auditors then rated each weakness as to its validity and its relative importance to the overall control structure of the company, and rated each error as to its validity, likelihood of occurrence, and risk to the fair presentation of the financial statement if it did occur. The assessments were based on the experimental case used by participants. Validity ratings were coded as yes or no, while ratings of importance, likelihood, and risk were made on a scale of 1 to 5. There were few differences in the two judges' validity ratings, and their qualitative scores of importance, likelihood, and risk were highly correlated (0.77, p < .0001; 0.47, p < .0001; and 0.43, p < .0001, respectively). Mean scores on importance, likelihood, and risk were used for analysis when differences in these ratings occurred. Finally, quantitative and qualitative performance measures for each participant were computed. Quantitative variables included the total number of valid weaknesses (W-CNT) and valid errors (E-CNT) identified by each participant. Qualitative measures were created by summing the highly experienced auditors' rating scores of relative importance of each valid weakness (W-IMP), likelihood of each valid error occurring (E-LIKE), and the risk (E-RISK) of each valid error identified by each participant. (11) Table 3 presents summary statistics for the performance variables.
TABLE 3 Summary Statistics for Performance Variables Variable Standard Name Variable Description Mean Deviation W-CNT Count of weaknesses identified 3.69 2.36 W-IMP Summed importance ratings for weaknesses identified 12.36 7.94 E-CNT Count of errors identified 3.52 2.27 E-LIKE Summed likelihood ratings for errors identified 11.53 9.79 E-RISK Summed risk ratings for errors identified 9.07 6.29 Variable Name Variable Description Min. Max. W-CNT Count of weaknesses identified 0 10 W-IMP Summed importance ratings for weaknesses identified 0 30 E-CNT Count of errors identified 0 9 E-LIKE Summed likelihood ratings for errors identified 0 57 E-RISK Summed risk ratings for errors identified 0 24
To facilitate the analysis of interactions, auditor weights on the control objective and transaction flow dimensions were split at the median (12) to create high/low variables. Analyses were initially performed for these two dimensions and their interaction, excluding control method. Because the dependent variables are correlated at a statistically significant level (all r > .83), MANOVA is used in the analysis. All associations between these two knowledge structure dimensions and the five performance measures, shown in Table 4, were positive. The MANOVA results for Control Objective were significant (p < .03), but were not significant for Transaction Flow (p < .14), possibly due to the small sample size (n = 59). Thus, the univariate results in regard to Transaction Flow (described next) must be interpreted with caution (Tabachnick and Fidell 1989). This unexpected difference in significance suggests that the relative importance of a single knowledge structure is contextual; i.e., each type of structure may vary in its relevance to a particular audit problem and task.
TABLE 4 MANOVA Analysis of Auditor Performance Univariate F-Tests Multivariate Dependent Effects Significance Variables F Sig. Transaction Flow .140 W-CNT 2.98 .045 ** W-IMP 4.35 .021 ** E-CNT 3.96 .026 ** E-LIKE 2.44 .062 * E-RISK 3.14 .041 ** Control Objective .028 W-CNT 4.86 .016 ** W-IMP 3.74 .029 ** E-CNT 5.28 .013 ** E-LIKE 1.08 .152 E-RISK 2.38 .064 * Trans Flow x Control .495 W-CNT 0.24 .314 Obj W-IMP 0.18 .342 E-CNT 0.46 .500 E-LIKE 0.26 .320 E-RISK 0.38 .264 *, ** Significant at the .10 and .05 level, respectively, for one-tailed test.
In reviewing the association between the structural dimensions and the two measures of auditor performance in the identification of weaknesses, both the transaction flow and control objective dimensions were related to the quantitative performance variable, W-CNT (TF: p < .05; CO: p < .02) and the qualitative performance variable, W-IMP (TF: p < .03; CO: p < .03). For auditor performance in the identification of potential errors, both knowledge structure dimensions were related to the quantitative error identification performance measure E-CNT (TF: p < .03; CO: p < .02) and the qualitative measure E-RISK (TF: p < .05; CO: p < .07). Only Transaction Flow was marginally related to the qualitative E-LIKE (p < .07) measure.
Interactions of the two knowledge structure dimensions were not significant for any of the performance variables. For post hoc analyses, control method was added to the model. No effects for this dimension were significant, and the relationship between Control Method and performance tended to be negative.
These results suggest that there is a positive association between the strength of computer auditor transaction flow and control objective knowledge structure dimensions and performance in the review of internal controls, supporting H3. Because the interactions were not significant, there is also evidence that the two dimensions operate independently in their relationship with performance.
Finally, to further support the notion that knowledge structure serves as a mediator between experience and performance (Libby and Luft 1993), post hoc analyses were performed to determine the extent to which differences in experience explain differences in computer auditor performance. Total audit experience (p > .40 for each), total EDP audit experience (p > .30 for each), and level within the firm (p > .20 for each) were regressed against each performance variable. All of these relationships were insignificant, thus supporting the theory that experience, rather than serving as a direct influence on auditor performance, instead affects an intermediate facility, cognitive capabilities, which in turn influence performance.
Antecedents of Computer Auditor Knowledge Structure Dimensions
Hypotheses 4 and 5 relate to the antecedents of computer auditor knowledge structure dimensions. From participant responses to the demographic questionnaire, we were able to identify many facets of prior education and experience. Participant total semester hours of accounting and computer information systems (CIS) courses, as well as total years of EDP audit and CIS-related work experience, were regressed against the "IT and CO dimensions. The overall regressions are significant in each case. Hypotheses 4a and 4b proposed that accounting hours would be positively related to both control objective and transaction flow dimensions. As Table 5 indicates, hours of accounting courses are positively related to participant weights on the control objective dimension (p < .03), but were not significantly related to the transaction flow dimension. Hypothesis 4c suggested that CIS hours would be positively related to the transaction flow dimension. Analyses depicted in Table 5 indicate that hours of CIS-related courses are not significantly related to participants' hours on the transaction flow dimension, in the presence of other structural determinants. These results provide support for H4b, and do not support H4a and H4c.
TABLE 5 Regression Analyses of Dimensional Antecedents Dimension 1: Transaction Flow Estimate t-value Hours of Accounting Courses -.0001 -0.03 Hours of Computer-Related Courses .0024 1.10 Years of Computer Audit Experience -.0011 -0.10 Years of CIS-Related Experience .0245 1.89 ** [R.sup.2] .118 Dimension 2: Control Objective Estimate t-value Hours of Accounting Courses .0082 2.11 ** Hours of Computer-Related Courses -.0030 -1.15 Years of Computer Audit Experience -.0147 -1.09 Years of CIS-Related Experience -.0193 -1.26 [R.sup.2] .134 *, ** p-value [less than or equal to] .10 and [less than or equal to] .05, respectively, for one-tailed test.
Post hoc analyses of accounting and CIS course hours with the control method dimensional weights reveal that both educational disciplines were negatively related to participant weights on this dimension, although these associations were not significant for a two-tailed test. It seems reasonable that education in these fields would make one more comfortable with computer and control systems, and thus place less emphasis on whether any particular procedure was computerized or manual.
Hypotheses 5a and 5b predicted positive relationships between computer-audit experience and both transaction flow and control objective dimensions. Analysis presented in Table 5 indicates that experience in computer auditing was not significantly related to either dimension. Therefore, H5a and H5b are not supported. This may be due to the diversity of computer auditing, which results in many different kinds of experiences (Viator and Curtis 1998). Thus, two computer auditors with the same number of years of experience in the field may have very different levels of experience for particular computer-audit tasks.
Hypothesis 5c proposed that CIS-related work experience would result in greater weights on the transaction flow dimension. Regression results indicate that prior experience in CIS is positively related to auditor transaction flow dimension weights (p < .04), supporting this hypothesis. Post hoc analysis found that CIS experience and control method dimensions were negatively correlated (p < .06), while no significant relationship was found between experience in computer auditing and the control method dimension (p > .32).
Table 6 summarizes the findings for all hypotheses.
TABLE 6 Summary of Hypotheses and Findings Hypothesis Supported? Principal Finding H1: Experienced computer Supported Structure includes auditors' CO and TF, as well application control as a third--Control knowledge structure Method includes both control objective and transaction flow dimensions. H2: The majority of Not supported Majority of computer computer auditors auditors employ TF will exhibit a as dominant dominant dimension structure. equivalent to their firm training and decision-aid structure. H3: Greater (lesser) Supported Dimensions related structure for to several different control objective measures of and transaction flow performance dimensions is associated with greater (lesser) performance in the review of internal controls. H4a: Computer auditors' Not supported Accounting education college semester not significantly hours in accounting related to TF are positively related to the extent of their transaction flow knowledge structure dimension. H4b: Computer auditors' Supported Accounting education college semester positively related hours in accounting to CO are positively related to the extent of their control objective knowledge structure dimension. H4c: Computer auditors' Not supported CIS education not college semester significantly hours in CIS are related to TF positively related to the extent of their transition flow knowledge structure dimension. H5a: Computer auditors' Not supported Computer audit years of computer experience not audit work significantly experience are related to TF positively related to the extent of their transaction flow knowledge structure dimension. H5b: Computer auditors' Not supported Computer audit years of computer experience not audit work significantly experience are related to CO positively related to the extent of their control objective knowledge structure dimension. H5c: Computer auditors' Supported CIS work experience years of CIS-related positively related work experience are to TF positively related to the extent of their transaction flow knowledge structure dimension.
DISCUSSION AND LIMITATIONS
The primary motivation for this research was to investigate the issue of whether access and retrieval of a single knowledge structure is sufficient for auditor job performance, or whether concurrent access of multiple knowledge structures support higher job performance. Through the use of a multidimensional approach, it was demonstrated that both control objective and transaction flow structures are concurrently associated with stronger performance in the evaluation of both internal control weaknesses and potential financial statement errors, considering both quantitative and qualitative measures of performance. The implication is that multiple knowledge structures, each strongly developed, facilitate higher auditor performance. In the current study, both transaction flow and control objective were concurrently, positively associated with auditor performance. In other audit tasks, the relative importance of the two structures on auditor performance may, however, differ.
These results have implications for accounting educators and professionals. First, the shift from transaction flow to control objective approach in the evaluation of internal control (Borthick et al. 2000) may result in a de-emphasis of transaction flow structure in the classroom and in training programs. This research suggests that a strong transaction flow knowledge structure significantly contributes to job performance, beyond that provided by control objective. Second, decision support aids, such as internal control questionnaires, designed to employ a control objective structure exclusively for this task may diminish the extent to which auditors employ transaction flow knowledge structure, resulting in weaker job performance.
A question raised in regard to computer auditors was whether knowledge structure differences attributable to diverse education and work experience backgrounds might be mitigated by exposure to uniform training. More experienced computer auditors were found to employ TF as their dominant dimension, while less experienced computer auditors appear to employ CO. Prior studies indicate that experienced financial auditors employ CO as their dominant structural dimension. Thus, despite common training approaches and task decision aids for financial and computer auditors, it appears that experienced computer auditors' dominant structural dimension diverges from financial auditors for internal control knowledge. This may be explained by differences in types of experience, types of job responsibilities (since financial auditors are also responsible for planning substantive audit procedures), or other factors. Nevertheless, it is possible that performance differences exist between the two groups (Nelson et al. 1995; Choo and Trotman 1991). Further research directly contrasting financial and computer auditors is warranted.
The finding that differing educational backgrounds are associated with differing structural development has implications for audit-firm hiring preferences and the design of educational/ training programs. For example, computer auditors with greater amounts of traditional accounting education exhibited enhanced control objective structure. Computer auditors appear to gradually change their structural emphasis to transaction flow as they gain experience. However, despite changing dominant structure, prior work experiences have a differential influence on the extent of each structural dimension throughout auditors' careers.
A limitation to this research is the tight restrictions researchers face in the selection of the internal control concepts used in MDS knowledge structure elicitation tasks. For example, the specific concepts included must be representative of every possible dimension in order to ensure that the dimensions actually employed by auditors can emerge, while the number of concepts included must be limited in order to maintain the mental alertness and interest of participants during the paired-comparison task. Thus, the identification of possible dimensions and relevant concepts that fit those dimensions is the responsibility of the researcher. Additionally, because the identification of subjects' actual structural dimensions is also the responsibility of the researcher, the labeling of these auditor structures is subject to possible researcher bias. Finally, while the use of one firm was necessary in order to control interfirm training differences, this limits the generalization of results to other organizations, particularly internal auditors or smaller public accounting firms that do not employ extensive training programs for their computer auditors.
This study suggests several possibilities for future research. First, the finding that multiple knowledge structures are concurrently associated with higher job performance raises interesting questions about the marginal contribution of each structure. Prior audit studies have focused on which knowledge structure is most closely aligned with job performance, but the results of this study suggest that secondary knowledge structures also are positively associated with job performance. To more fully understand audit effectiveness, accounting researchers should examine how the relative importance of multiple knowledge structures varies depending on the audit task. Second, since there is little question that enhanced knowledge structure results in better performance, specific training techniques designed to provide multidimensional presentation of audit knowledge should be investigated. Finally, the profession may be interested in identifying whether other forms of intervention, such as decision aids or working paper organization, can mitigate the influence of prior education and experience on structure development.
The authors are indebted to the two computer audit specialists who helped with this project as well as the computer audit professionals who participated and their firm that supported the research. Earlier versions of this paper benefited from comments by the participants of work. shops at the University of Kentucky, Temple University, and Georgia State University. The authors also thank Bob Ramsey, Bill Dilla, Mark Davis, the editors, and two anonymous reviewers for their insightful comments.
Alba, J. A., and L. Hasher. 1983. Is memory schematic? Psychological Bulletin 93: 203-231.
American Institute of Certified Public Accountants (AICPA). 1979. The Auditor's Study and Evaluation of Internal Control in EDP Systems. New York, NY: AICPA.
--. 1983. Statements on Auditing Standards. New York, NY: AICPA.
--. 1995. Statement on Auditing Standards No. 78. New York, NY: AICPA.
Bailey, K. E., J. H. Bylinski, and M. D. Shields. 1983. Effects of audit report wording changes on the perceived message. Journal of Accounting Research 21: 355-370.
Bedard, J., and M. T. H. Chi. 1993. Expertise in auditing. Auditing: A Journal of Practice & Theory (Supplement): 21-45.
Boal, K., and N. Perry. 1985. The cognitive structure of corporate social responsibility. Journal of Management 11: 71-82.
Bodnar, G. H., and W. S. Hopwood. 1998. Accounting Information Systems. 7th edition. Upper Saddle River, NJ: Prentice Hall.
Bonner, S. E., and N. Pennington. 1991. Cognitive processes and knowledge as determinants of auditor expertise. Journal of Accounting Literature 10: 1-50.
--, J. Davis, and B. Jackson. 1992. Expertise in corporate tax planning: The issue identification stage. Journal of Accounting Research (Supplement): 1-28.
--, R. Libby, and M. W. Nelson. 1996. Using decision aids to improve auditors' conditional probability judgments. The Accounting Review 71 (2): 221-240.
--, --, and --. 1997. Audit category knowledge as a precondition to learning from experience. Accounting, Organizations and Society 22 (5): 387-410.
Borthick, A. F., M. B. Curtis, and R. Sriram. 2000. Accelerating the acquisition of knowledge structure to improve performance in internal control evaluation. Working paper, Georgia State University.
Boyton, W. C., and W. G. Kell. 1996. Modern Auditing. 6th edition. New York, NY: John Wiley & Sons, Inc.
Butt, J. L. 1988. Frequency judgments in an auditing-related task. Journal of Accounting Research (Autumn): 315-330.
Champlain, J. J. 1995. Demystifying computer systems auditing. Internal Auditor (December): 28-33.
Chi, M. T. H., and E. T. Rees. 1983. A learning framework for development. Contributions in Human Development 9:71-107.
--, J. E. Hutchinson, and A. F. Robin. 1989. How inferences about novel domain-related concepts can be constrained by structured knowledge. Merrill-Palmer Quarterly 35: 27-62.
Choo, F., and K. T. Trotman. 1991. The relationship between knowledge structure and judgments for experienced and inexperienced auditors. The Accounting Review 66: 464-485.
Christ, M. Y. 1993. Evidence on the nature of audit planning problem representations: An examination of auditor free recalls. The Accounting Review 68 (April): 304-322.
Collins, A. M., and E. F. Loftus. 1975. A spreading activation theory of semantic processing. Psychological Review 82: 407-428.
Cooke, N. M., F. T. Durso, and R. W. Schvaneveldt. 1986. Recall and measures of memory organization. Journal of Experimental Psychology: Learning, Memory, and Cognition 12: 538-549.
Curtis, M. B., and A. F. Borthick. 1999. Evaluation of internal control from a control objective narrative. Journal of Information Systems (Spring): 63-82.
DeJong, T., and M. G. M. Ferguson-Hesler. 1986. Cognitive structures of good and poor novice problem solvers in physics. Journal of Educational Psychology (August): 279-288.
DeMarie-Dreblow, D. 1991. Relation between knowledge and memory: A reminder that correlation does not imply causality. Child Development 62: 484-498.
Dillon, W. R., and M. Goldstein. 1984. Multivariate Analysis Methods and Applications. New York, NY: John Wiley & Sons.
Droge, C., and R Y. Darmon. 1987. Associative positioning strategies through comparative advertising: Attribute versus overall similarity approaches. Journal of Marketing Research 24: 377-388.
EDP Auditors Association (EDPAA). 1992. Control Objectives. 4th edition. Carol Stream, IL: The EDP Auditors Foundation, Inc.
Ford, M. 1995. Two modes of mental representation and problem solution in syllogistic reasoning. Cognition 54 (January): 1-71.
Frederick, D. M. 1991. Auditors' representation and retrieval of internal control knowledge. The Accounting Review 66 (April): 240-258.
--, V. B. Heiman-Hoffman, and R. Libby. 1994. The structure of auditors' knowledge of financial statement errors. Auditing: A Journal of Practice & Theory (Spring): 1-21.
Ingraham, L. R., and J. H. Reneau. 1999. The effect of expert system explanatory facilities on knowledge acquisition and representation. Working paper, North Carolina State University.
Kintsch, W., and J. G. Greeno. 1985. Understanding and solving word arithmetic problems. Psychological Review 92 (1): 109-129.
Kruskal, J. B., and M. Wish. 1978. Multidimensional Scaling. Newbury Park, CA: Sage Publications.
Libby, R. 1979. Bankers' and auditors' perceptions of the message communicated by the audit report. Journal of Accounting Research (Spring): 99-122.
--, and D. M. Frederick. 1990. Expertise and the ability to explain audit findings. Journal of Accounting Research (Autumn): 348-367.
--, and J. Luft. 1993. Determinants of judgment performance in accounting settings: Ability, knowledge, motivation, and environment. Accounting, Organizations and Society 18: 425-450.
Markman, A. B., and D. Gentner. 1996. Commonalities and differences in similarity comparisons. Memory & Cognition 24 (2): 235-249.
Moore, W. L., and D. R. Lehmann. 1982. Effects of usage and name on perceptions of new products. Marketing Science 1 (4): 351-370.
Mullins, W. C., and W. W. Kimbrough. 1988. Group composition as a determinant of job analysis outcomes. Journal of Applied Psychology 73: 657-664.
Nelson, M. W., R. Libby, and S. E. Bonner. 1995. Knowledge structure and the estimation of conditional probabilities in audit planning. The Accounting Review 70 (1): 27-47.
Pei, B. K. W., and J. H. Reneau. 1990. The effects of memory structure on using rule-based expert systems for training: A framework and an empirical test. Decision Sciences 21: 263-286.
Plumlee, R. D. 1985. The standard of objectivity for internal auditors: Memory and bias effects. Journal of Accounting Research (Autumn): 46-58.
Pratt, J. 1982. Post-cognitive structure: Its determinants and relationship to perceived information use and predictive accuracy. Journal of Accounting Research 20: 189-209.
Rittenberg, L. E., and B. J. Schwieger. 1996. Auditing: Concepts for a Changing Environment. 2nd edition. Fort Worth, TX: The Dryden Press.
Romney, M. B., P. J. Steinbart, and B. E. Cushing. 1997. Accounting Information Systems. 7th edition. Reading, MA: Addison-Wesley Publishing Co., Inc.
SAS Institute Inc. 1992. SAS Technical Report P-229, SAS/STAT Software: Changes and Enhancements. Release 6.07. Cary, NC: SAS Institute Inc.
Schvaneveldt, R. W., F. T. Durso, T. E. Goldsmith, T. J. Breen, and N. M. Cooke. 1985. Measuring the structure of expertise. International Journal of Man-Machine Studies 23: 699-728.
Tabachnick, B. G., and L. S. Fidell. 1989. Using Multivariate Statistics. Second edition. New York, NY: Harper & Row, Publishers.
Tubbs, R. M. 1992. The effect of experience on the auditor's organization and amount of knowledge. The Accounting Review 67 (October): 783-801.
Vera-Munoz, S. C. 1998. The effects of accounting knowledge and context on the omission of opportunity costs in resource allocation decisions. The Accounting Review 73 (1): 47-72.
Viator, R. E., and M. B. Curtis. 1998. Computer auditor perception of the relative effectiveness of internal control procedures. Journal of Information Systems (Spring): 19-30.
Weber, R. 1980. Some characteristics of free recall of computer controls by EDP auditors. Journal of Accounting Research (Spring): 214-241.
West, L. H. T., P. J. Fensham, and J. E. Garrard. 1985. Describing the cognitive structures of learners following instruction in chemistry. In Cognitive Structure and Conceptual Change, edited by L. H. T. West, and A. L. Pines. Orlando, FL: Academic Press, Inc.
(1) For example, Bonner et al. (1997) trained students for either transaction cycle or control objective structure, Borthick et al, (2000) trained students for either transaction flow or control objective structure, and Nelson et al. (1995) considered experienced auditors to possess control objective structure as their primary (dominant) knowledge structure.
(2) While financial auditors, in the past, have tended to have accounting undergraduate degrees and little work experience prior to joining an audit staff, computer auditors are as likely to have computer information systems (CIS) degrees as well as accounting degrees, and often have prior experience in CIS and other fields (Viator and Curtis 1998).
(3) It is quite possible that auditors with diverse backgrounds will have other knowledge structures, as well. While TF and CO appear to be the most common knowledge structure dimensions, a review of audit text books and training manuals suggests other ways in which auditors may organize their application control knowledge. These include methods (manual vs. automated procedures), the particular technology being controlled, whether the control is preventive, detective, or corrective, and whether the control is discretionary (EDPAA 1992; Rittenberg and Schwieger 1996).
(4) Analyses of performance for those who completed the instrument in experimental settings and as surveys resulted in no significant differences between the two groups. In addition, measures of time spent at the task and of interest in the task are not significantly different between the two groups.
(5) The matching test required the participants to match a list of the internal control concepts to a set of definitions, on a one-to-one basis. The answer key listed each control concept, followed by the correct definition.
(6) Two versions of the elicitation instrument were completed, with the same pairs sorted in two different orders. No order effects were identified during analysis. The disbursements case was developed by the researchers, based on their own experience in computer auditing, and was organized to favor no particular structure. See Curtis and Borthick (1999) for a version of the case specifically rewritten for control objective structure. The entire experimental instrument was pretested with three computer audit seniors and two computer audit managers for face and content validity. Revisions were made based upon their feedback.
(7) The specialists were a managing partner for the firm's largest computer-audit group and a computer-audit manager recognized for his skill in this field. The two had over 20 and 12 years of computer-audit experience, respectively, and the partner was a co-author of a highly regarded computer-audit reference manual. These specialists were asked by the authors to participate in the study.
(8) The "gold standard" group included all auditors at the manager level in the top third of the computer-audit experience range and in the top third of the application control-review experience range. The second criteria was relevant because some computer auditors specialize in the review of general controls and, while possessing significant experience in computer auditing in general, are not as experienced in this particular task. This selection process resulted in a subset of ten managers. The dimensions of this subset of managers are similar to those described for the group of all managers.
(9) The order in which the gold standard (GS) dimensions are seeded into the overall MDS analysis does not bias the weights identified for individuals. The GS dimensions may be input into an MDS analysis in any order (e.g., CO, TF, CM or 'IF, CM, CO) and achieve the same individual weights on each dimension.
(10) These individual dimension weights also provide a second method for confirming the assigned dimensional labels. A question at the completion of the exercise asked that participants identify the basis for their comparisons during the knowledge structure elicitation exercise. While not all participants provided interpretable answers, many did. For example, seven auditors stated that they distinguished between the control procedures based upon the control objective each procedure satisfied. All seven of these participants had relatively high weights on the CO dimension. Additionally, seven auditors who stated that they distinguished between concepts based upon whether the controls were manual or automated had relatively high weights on the CM dimension and three auditors who stated that they distinguished based upon the phases of processing were relatively high on the TF dimension.
(11) Weanesses or errors that the specialists agreed were invalid, given the case, were not included in the auditors performance scows.
(12) The distributions were relatively uniform, although the TF weights were slightly skewed to the left. Sensitivity analysis suggests that all findings are robust across reasonable differences in the dimensional splits.
Mary B. Curtis is an Assistant Professor at the University of North Texas, and Ralph E. Viator is an Associate Professor at the University of Kentucky.
|Printer friendly Cite/link Email Feedback|
|Author:||Curtis, Mary B.; Viator, Ralph E.|
|Publication:||Auditing: A Journal of Practice & Theory|
|Date:||Sep 22, 2000|
|Previous Article:||The Effect of Auditor Attestation and Tolerance for Ambiguity on Commercial Lending Decisions.|
|Next Article:||The Audit Risk Model: An Empirical Test for Conditional Dependencies among Assessed Component Risks.|