Relationships between teacher education students' epistemological beliefs and their learning outcomes in a case-based hypermedia learning environment.

Personal epistemological beliefs, what individuals believe about the nature of knowledge, have been known to influence many aspects of learning. This study examines the relationships between the epistemological beliefs of teacher education students and their learning in a case-based hypermedia learning environment. Data were collected from a set of questionnaires, from pre- and postknowledge tests, and from evaluation of users' artifacts on problem-solving activities embedded within the case-based hypermedia learning environment (CBHLE). The results of this study revealed that epistemological beliefs had significant but low relationships with learning in a hypermedia learning environment. This article discusses the implications of this investigation for researchers and instructors and provides recommendations for future research.

**********

Personal epistemological development is a growing area of interest for psychologists and educators. Studies in epistemological beliefs--what individuals believe about the nature of knowledge (Hofer & Pintrich, 1997; Kardash & Scholes, 1996; Schommer, 1994a, 1998)--provide one lens through which the teaching-learning process in teacher education can be viewed. Epistemological beliefs have been shown to influence students in terms of their learning in ill-structured domains, their problem-solving approaches, and their persistence in facing difficult tasks (Kardash & Scholes; Scheurman, 1997; Schommer, 1994a, 1998).

Current school reforms in teacher education focus on an augmentation of opportunities from which students can gain more authentic classroom experience through field placement and technology components based on real classroom scenarios. As a part of these school reforms, a number of instructional approaches have been implemented, including case-based learning (CBL) and hypermedia learning. These two instructional approaches provide an interesting array of alternative materials and approaches to traditional teaching strategies for teacher education programs.

Purpose

The purpose of this study was to examine relationships between teacher education students' epistemological beliefs and these students' learning outcomes in a case-based hypermedia learning environment (CBHLE). Although the literature generally supports the conceptual value of epistemological beliefs, most researchers agree that empirical data are needed to clarify the relationships between epistemological beliefs and learning (Hartley & Bendixen, 2003; Land & Greene, 2000; Schommer, 1994b). Although a number of studies have documented the connection between epistemological beliefs and their effects on learning in a hypertext system, to date, there have been no studies focusing on students' epistemological beliefs and performance in CBHLE.

Previous studies have shown that well-designed CBHLE programs offer an equally effective learning environment for many diverse learners, with the only causal difference among the learners' achievements being the total time that learners are engaged in the program (Fitzgerald, Semrau, & Deasy, 1997; Fitzgerald & Semrau, 1998). By extending this line of research on learner differences in CBHLE to an examination of the effects that learners' epistemological beliefs have on learning outcomes with CBHLE materials, results may further explain how students learn from cases and hypermedia--an accomplishment that can, in turn, lead to effective implementation of case-based and hypermedia instruction in teacher education.

REVIEW OF THE LITERATURE

The following sections review research findings on the interweaving of epistemological beliefs with the design and the development of innovative case-based hypermedia learning environments.

A Brief Review of Epistemological Beliefs

Earlier conceptions of epistemological beliefs have been dominated by the work of Perry (1970). Perry hypothesized that students go through fixed stages attached to the development of epistemological belief: (a) dualism, (b) multiplism, (c) relativism, and (d) commitment. Expanding on the work of Perry, Schommer (1990) suggested that personal epistemology would be better portrayed as a multidimensional system. In a later study, Schommer (1994a) identified four dimensions within four perspectives of epistemological beliefs, ranging from naive to sophisticated:

1. Ability to Learn -- the ability to learn either is fixed at birth (naive) or can be changed (sophisticated);

2. Structure of Knowledge -- knowledge is made up of either unambiguous, isolated bits (naive) or highly interrelated concepts (sophisticated)

3. Speed of Learning -- learning is either quick or all-or-none (naive) or, on the other hand, gradual (sophisticated); and

4. Stability of Knowledge -- knowledge is either absolute and unchanging (naive) or evolving (sophisticated). Schommer has developed and tested the Epistemological Questionnaire (EQ) to measure an individual's epistemological beliefs in these four dimensions (Schommer, 1994a). Table 1 provides an explanation of these four dimensions of epistemological beliefs and a sample of questions in each dimension.

The Relationships Between Epistemological Beliefs and Hypertext/Hypermedia Learning

Researchers have investigated the connection between epistemological beliefs and their effects on learning in a hypertext system (Jacobson, Maouri, Mishra, & Kolar, 1996; Jacobson & Spiro, 1995; MacNeal, 2000). Compared to students with a simple set of epistemological beliefs, those with sophisticated epistemological beliefs were likely to learn and apply their knowledge efficiently, to persist in information-seeking beyond the necessity, and to spend more time in the completion of the hypertext activities (Jacobson & Spiro, 1995; Tolhurst & Debus, 2002; MacNeal, 2000).

"Hypermedia," an extension of "hypertext," describes learning environments that allow the nonlinear, interactive linking of multiple media, such as video and audio, in a document (Tergan, 1997; Tolhurst, 1995). According to the existing literature, hypermedia has been promoted as a type of advanced educational development because of its nonlinear nature, its ability to link complex information, and its interactivity with the learner. The self-paced, highly engaging method also provides a natural form of the representation of the human mind (Dillon & Gabbard, 1998). Cognitive Load Theory (Brunken, Plass, & Leutner, 2003) and Mayer's three assumptions of multimedia learning (Mayer, 2001; Mayer & Moreno, 2003) provide theoretical foundations for the incorporation of multiple media in education. These theories propose that human minds have several channels through which information is communicated. If information is presented through two or more of these channels, there will be an additional reinforcement of learning and greater retention of the material. Fundamentally, the results of hypermedia research are consistent with Paivio's dual-coding theory, according to which multimodal redundancy is effective for the accommodating of various learners' needs in a hypermedia environment (Paivio, 1986).

Research on hypermedia studies is in its early stages with regard to the relationships among learner characteristics, system usage, and learning outcomes. Learner characteristics reflect many user attributes: readiness, learning styles, and motivation to learn (Takacs, Reed, Wells, & Dombrowski, 1999). In the early 1990s, Tergan and colleagues (1992) suggested that, in order to support learners in task processing (e.g., modeling, coaching, and scaffolding), effective learning should result from a complex interaction of learner-based constraining conditions (e.g., learner ability, motivation, and learning styles) and of instructional methods. Following Tergan, Dillon and Gabbard (1998) began their comprehensive review of existing quantitative research literature on learners' characteristics, including learner comprehension, control, and style. They proposed that individual differences among learners might explain the effects of hypermedia on learning outcomes. However, learners' epistemological beliefs, being one attribute of learners' characteristics, remain unexplored in empirical studies.

The Use of CBHLEs in Teacher Education Programs

Interactive, multimedia case programs are typically used in higher education as supplements to teacher education courses because such case programs offer alternative ways to provide practical experiences similar to the hands-on training available in field placement requirements (Fitzgerald & Semrau, 1998). Examples of these rich, hypermedia-learning environments are the Teacher Problem Solving Skills in Behavioral Disorders (TPSS) series (Fitzgerald & Semrau, 1993-1997). The goals of these CBHLE materials are to enhance the problem-solving skills of teacher education students preparing to serve children who have emotional and behavioral disorders. Using these multimedia case materials, students in special education programs can develop knowledge and perspectives based on complex case information and practice realistic skills needed in professional practice. Each case program includes multimedia elements, case and procedural information, and performance support tools to help users create intervention materials. Also embedded in the program is an audit trail function that automatically records users' time, actions, and input.

Fitzgerald and Semrau have conducted a number of studies regarding the effectiveness of the learning outcomes of preservice teachers who use the TPSS case-based, hypermedia learning systems (Fitzgerald & Semrau, 1998; Fitzgerald, Semrau, & Deasy, 1997; Kraus, Reed, & Fitzgerald, 2000). Their studies have demonstrated that learners broaden their knowledge repertoires, improve their cognitive flexibility in approaching problems, and increase their problem-solving skills. Their research has centered on relationships between learner differences, usage patterns within hypermedia environments, and learning outcomes. Their findings revealed that these case-based, hypermedia learning environments accommodated diverse users and facilitated equivalent levels of learning regardless of age, learning styles, teaching experience, or prior computer experience.

The Case of Amy

The case of Amy is one of the TPSS cases in Program 3, Instruction and Management in Behavioral Disorders, that focuses on knowledge and skills to design and develop instruction and management plans for children with these learning needs. "Amy" is a 10-year-old girl with mild behavioral and emotional problems who has been placed in a resource room with instructional inclusion in a regular classroom and receives supportive instruction in social skills and language therapy. After logging into Amy, a user interacts in the hypermedia environment by accessing case and procedural information, viewing videos of specialized curriculum, and taking a role as Amy's teacher to analyze her needs, assess her setting, create a behavior management plan and materials, and plan her instructional program. User's work on these four problem-solving activities creates learning artifacts that can be evaluated by course instructors or researchers studying learning outcomes of CBHLE systems.

METHODOLOGY

Research Questions

The following two research questions were examined in this study:

1. With regard to knowledge acquisition, is there a relationship between the epistemological beliefs of teacher education students and their learning outcomes when such students use a case-based hypermedia learning environment?

2. With regard to the quality of casework artifacts, is there a relationship between the epistemological beliefs of teacher education students and their learning outcomes when such students use a case-based hypermedia learning environment?

Research Design

This study was a pre- and posttest design, involving a one-month intervention period during which graduate students in special education courses independently used the program Amy. The content in Amy corresponded with the focus of the methods courses; topics included behavior assessment, management interventions, and instructional curriculum and methods. The use of the Amy case was a course requirement and took approximately five to seven hours for the students to complete. Students were required to work on the program individually outside their class time and complete the case assignment in four weeks. During the research period, students were asked not to discuss the content with others or to complete assignments collaboratively. Points toward the students' grades were given for completion of the casework activities but not for the quality of their work on the those activities. Instructors were not allowed to discuss or integrate case activities until all research data were collected and submitted. At that point in the courses, instructors could integrate the case content and activities as desired.

The first step of the study involved the dissemination of a demographic survey, an Epistemological Questionnaire (EQ), and the pretest Knowledge Instrument prior to the use of the hypermedia program in the fifth week of the semester. The second step involved an ongoing collection of program usage data and artifacts from the problem-solving activities embedded within the hypermedia program. The participants' responses on the activities were stored on a floppy disk and mailed to the researchers for analysis. Four researcher-developed rubrics were used to score students' work on the problem-solving activities within Amy. The final step involved the administering of the posttest Knowledge Instrument.

To ensure the integrity of research implementation, the four instructors who allowed the researchers to collect data from their students filled out an integrity checklist halfway through and again at the end of the study. All participants and instructors completed the checklists. The data were used to verify the extent to which the research protocol had been implemented. In accordance with the results of checklists, it can be concluded that the instructors and the students among these four universities sufficiently followed the research protocols.

Data Collection

Data were collected from participants by both self-reporting instruments and the audit trail captured by the hypermedia program. A combination of quantitative and qualitative data methodologies were employed, including (a) quantitative data collected from the Epistemological Questionnaire (EQ) and the pre- and post Knowledge Instruments, and (b) quantitative and qualitative data collected from rubric scoring and analysis of the user artifacts generated by the on the four problem-solving activities (case consults, setting analysis, management plan, and instructional plan).

Participants

The subjects for this study included a convenience sample of teacher education students in special education methods courses at four universities in different areas of the United States. A convenience sample, rather than a randomized sample, was used owing to the limited availability of research subjects in courses with trained TPSS instructors who could provide the technology support needed for the Amy program. Although the researchers were not able to employ random selection, participants included a wide spectrum of teacher education students. The study was announced at the beginning of the course and volunteers were solicited to participate. Seventy-three out of the 98 students who were enrolled in the four methods courses agreed to participate in the study. Sixty students completed the pre- and posttest instruments and were used as the research subjects.

On the basis of the demographic survey for the study, these 60 teacher education students ranged in age from 19 to 55, ranged in class ranking from junior to graduate, and represented a range of classroom experience from none to 14 years as a certified teacher or school administrator. All subjects from these four universities were treated as one group with requirement to use Amy as a supplement to their special education methods course. The purpose of the study was to examine relationships among variables related to individuals' epistemological beliefs rather than to compare performances between groups. Despite the fact that students from four different groups or sites participated in this study, a predesigned protocol controlled the intervention to ensure that each group received as close to the same treatment as possible. The treatment was the independent use of the CBHLE case material, not the integration or use of the case material in their courses.

Instruments

The instruments used in this study included (a) the Epistemological Questionnaire (Schommer, 1998), (b) pre- and posttest Knowledge Instruments of interventions in behavioral management, (c) a demographic survey, and (d) case artifact rubrics.

Epistemological Questionnaire (EQ). This 63-item Epistemological Questionnaire is one of the most widely used instruments available for the examination of college students' epistemological beliefs as multidimensional constructs. The EQ was used in the building of the four factor scores of epistemological beliefs, consisting of Ability to Learn, Structure of Knowledge, Speed of Learning, and Stability of Knowledge. A number of empirical studies have documented the statistical rigor and theoretical soundness of this instrument as well as the sufficiency of the four-factor structure for a set of epistemological beliefs. Duell and Schommer-Aikins (2001) reported a .74 test-retest and.63 to a .85 inter-item correlations for items within each belief factor. Content validity was established by expert review of professionals in the field of educational psychology. Predictive validity has been verified in different contexts of learning as well. A copy of Schommer's Epistemological Questionnaire is provided as Appendix 3, used with the author's permission.

The questionnaire asked respondents to rate their level of agreement for each item on a five-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). These ratings give an indication of a participant's range of beliefs along a continuum from a higher score (naive belief) to a lower score (sophisticated belief). Beliefs measured by the questionnaire may be represented more accurately as flexible points along a continuum rather than as fixed points at the extremes of the range of scores.

Knowledge Instruments. The Knowledge Instruments consisted of 18 multiple-choice and five matched questions, constructed to examine the teacher education students' knowledge of behavior disorders. These questions were designed specifically to focus on content provided through the hypermedia program. To ensure content validity, two experts in emotional and behavioral disorders reviewed the Knowledge Instruments. The items on the pre- and postKnowledge Instruments were identical except for the questions concerning computer equipment and research integrity, which were included at the end of the posttest Knowledge Instrument. There was a four-week time lag between administrations of these two tests.

The range of possible scores was 0 (all scored incorrect) to 32 (all scored correct). Changes in the posttest Knowledge Instrument were made on the basis of the results and suggestions from participants in the pilot study. A paired-samples t-test was performed for an examination of whether systematic pretest differences existed among the participants. The results of the paired-sample test indicated a significant difference (p = .005) between the pre- and posttest scores. Therefore, it was appropriate to use the gain score (pre-to-posttest gain) to represent achievement levels among participants while controlling for prior knowledge.

Demographic Survey. Participants completed a demographic survey that included questions pertaining to their gender, age, academic major, type of teaching certification, and prior teaching experiences. This information was collected to verify group attributes as well as future analyses of epistemological beliefs among diverse age groups and diverse levels of teaching experience. However, analyses of relationships among ages and teaching experiences were not the focus of this study.

Rubrics for Scoring Casework Artifacts. Case artifacts were generated as the participants completed the four problem-solving activities while using the hypermedia program. These artifacts were saved in records stored on the participants' floppy disks. Four case artifact rubrics were developed to provide a common and reliable scoring procedure to evaluate the quality of user responses on the problem-solving activities (refer to Appendices 1 & 2 for rubrics and examples of Needs Analysis and Instructional Plan). The rubrics were developed by an expert in behavioral disorders. The range of possible scores for each artifact was 0 (scored 0 on all 5 scales) to 15 (scored 3 on all 5 scales) for each problem-solving activity. The two researchers then scored a random sample of case artifacts using the rubrics. The two scorers achieved 93.3% item-by-item reliability. Discrepant items on the rubrics were resolved by discussion and modifications were made after the discussion to clarify scoring rules for the rubrics. The first author then completed scoring of the remaining case artifacts.

Independent Variables

The independent variables consist of four factor scores from the Epistemological Questionnaire: (a) Ability to Learn, (b) Structure of Knowledge, (c) Speed of Learning, and (d) Stability of Knowledge. The results of the EQ scores indicated that the subjects of this study had sophisticated views on three of the four epistemological belief factors. The descriptive statistics of the EQ scores are listed in Table 2.

Dependent Variables

Gain Scores on the Pre-to-postknowledge Instrument. The gain scores of pre-to-posttest performance on the Knowledge Instruments were used to measure the levels of participants' knowledge acquisition.

Rubric Scores on the Case Activities. Four case artifacts were completed within the hypermedia program as part of the problem-solving activities for Amy. In the Needs Analysis problem-solving activity, a preformatted form is provided for the user to identify Amy's needs within 10 programmatic categories. The activity works similar to a behavior consultation by prompting the user to analyze problems and identify needs (Figure 1). In the Setting Analysis problem-solving activity, the user evaluates each setting factor and provides justification notes on possible changes to assist Amy (Figure 2). After identifying Amy's needs, the user creates behavior management plans (Figure 3) and instructional plans (Figure 4) using problem-solving templates. These entries are saved in the user records and provide the artifacts used in this study to evaluate performance in the CBHLE through the rubric scoring procedures. Examples of rubrics used to score the Needs Analysis Problem-solving Activity and the Instructional Plan Problem-solving activity are included in Appendices 1 and 2.

[FIGURE 1 OMITTED]

[FIGURE 2 OMITTED]

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

Data Analyses

The analyses of data included frequencies and percentages, means and standard deviations, t-tests, Pearson's correlations, and multiple regressions. The significance level was set at the .05 level to control Type I and Type II errors. Responses from the demographic survey, the EQ, the Knowledge Instruments, and rubric scores of case artifacts were gathered and entered into SPSS. During the preliminary data screening, four outliers were deleted, leaving 56 cases for the following analyses. Before the regression technique was performed, a set of correlation matrixes was used to analyze the relationships among variables.

RESULTS AND INTERPRETATION

The Relationship Between Epistemological Beliefs and Knowledge Acquisition

Table 3 shows the relationships among the strength of the four epistemological belief factors and the gain scores on the Knowledge Instruments. None of the variables reached the level of significance. The results indicated that there were no linear relationships between epistemological belief factors and knowledge acquisition as measured on the Knowledge Instruments.

The Relationship Between Epistemological Beliefs and Quality of Case Artifacts

Table 4 shows how the variables correlated to each other and the levels of importance among the four belief factors and the quality of work on the case artifacts as measured by the rubric scores. The more that teacher education students believed that the ability to learn is fixed and unchangeable, the poorer they performed in the problem-solving activities of Setting Analysis (r = -.31, p < 0.05) and Management Plan (r = -.38, p < 0.01). On the contrary, the more that they believed that learning is quick or all-or-none, the better they performed in the Instructional Plan activity (r = .33, p < 0.05) for Amy.

Following the analysis of paired correlation, a series of stepwise regressions were conducted on the four factors in epistemological beliefs and on three case artifacts (Setting Analysis, Management Plan, and Instructional Plan). As shown in Tables 5, 6, and 7, the regression models revealed that students' beliefs on Ability to Learn were the strongest predictor of quality for both Setting Analysis ([R.sup.2] = .07) and Management Plan ([R.sup.2] = .12) and that students' beliefs on Speed of Learning were the strongest predictor for Instructional Plan ([R.sup.2] = .09). That is, among the four factors of epistemological beliefs, only two variables (Ability to Learn and Speed of Learning) entered the final regression model in predicting students' performances in the problem-solving activities.

The current study found that less than 10% of variation in problem-solving activities could be explained by the self-reported epistemological beliefs. An investigation of other demographic variables (i.e., gender and class rank) that are combined with epistemological beliefs might generate better prediction for learning that takes place within the CBHLE program.

CONCLUSIONS

Epistemological beliefs, as measured in this study, did not relate to knowledge acquisition for teacher education students who were using this case-based hypermedia learning environment. When looking at the average gains (pre-to-post scores) on the Knowledge Instruments, 1.44 points was the average gain score across all subjects, with a range of -8 points to +11 points. It is possible that the achievement in the preKnowledge Instrument was saturated, so that the pre-to-post gains did not show a significant amount of change. It is also possible that the Knowledge Instrument was not a sensitive measure of learning in this case environment. During this four-week intervention, students needed to work on the program during their own time; this work did not directly correspond to their course requirements, as points were given for completing the casework but not for the quality of their efforts. The low average gain scores might indicate that students neither worked for mastery of the content nor took the testing seriously.

These results did not replicate previous research findings that suggest that the presence of students with naive views toward Structure of Knowledge predicts the degree of students' comprehension of text-based materials (Duell & Schommer-Aikins, 2001; Schommer, 1990). Contradictions regarding achievement suggest that additional research should focus on factors that support knowledge acquisition in hypermedia-based learning environments.

Data from the scores on the case artifacts indicated that belief in Ability to Learn had a negative relationship with the quality of work on some of the problem-solving activities embedded in this CBHLE. The more that teacher education students agreed that ability to learn is fixed at birth (higher score in Ability to Learn), the poorer the quality of both their analysis of setting factors and their recommendations for managing Amy's behaviors from multiple perspectives to meet Amy's needs. Limited investigations have been conducted focusing on the relationships between belief in Ability to Learn and other aspects of learning. However, the findings of this study seem to support the assertion in Schommer's study (Schommer, 1990) that learners with more sophisticated views toward Ability to Learn (lower score in Ability to Learn) tend to value persistence in hard work in the face of academic difficulties; hence, such learners would be expected to perform better on problem-solving activities. Students with more sophisticated views toward Ability to Learn created more in-depth artifacts in Setting Analysis and generated more plausible behavioral management procedures to assist Amy.

The relationship between belief in Speed of Learning and the quality of work on intervention planning revealed inconsistent results from earlier research. Earlier studies (Schoenfeld, 1983; Schommer, 1990; Schommer-Aikins, 2002) suggested that naive learners in Speed of Learning perform more poorly because they are less inclined to engage and persist in difficult problems. Schraw, Dunkle, & Bendixen (1995) and colleagues (1995) found that belief in Speed of Learning relates to students' problem solving of ill-sdefined content. However, this study did not replicate the previously reported results for naive learners in the problem-solving activity of instructional planning. In general, the structure and the complexity of the Instructional Plan activity are parallel to those of the Management Plan activity.

Nevertheless, there are major differences in the content of the program that may account for this unexpected finding. Procedural information on behavior management is only provided through text in this CBHLE, whereas information on instructional planning is provided through video demonstration and expert commentary. The program features audio overviews of four curricular approaches (social skills training, conflict resolution, cognitive restructuring, and anger control training) and classroom videos that demonstrate implementation of these approaches in the classroom. Although it is true that the text information on behavioral management includes performance support tools and strategy information that are meant to support the learner, these components are less accessible to the user. To use these resources, users who are in the main program must relocate to the tool and resource programs; although these programs are linked to the main program, they are not easily available when users are constructing behavioral management artifacts. These two factors may contribute to difficulties that naive learners encounter in utilizing program resources to create good management plans. Even though naive learners spent less time integrating ideas, they demonstrated a better ability to apply relevant information to the creating of goals. This view would be consistent with the cognitive load theory, which suggests that learning is enhanced when processing involves two or more channels in learning (Paivio, 1986).

Recommendations for Further Study

First, to address the generalizability of the findings, it is recommended that future studies use a larger sample size and include a range of instructors who, in terms of teaching, may not be familiar with hypermedia cases. Research needs to accomplish two things: an analysis of generalizations across age and an embrace of groups of students beyond those included in this study. Studies need to include participants representing the full spectrum of epistemological beliefs.

Second, the organization of components in the case-based hypermedia program in this study might not apply to all case-based hypermedia programs. Limited generalizability should be taken between this hypermedia program and others with different content and organizational structures.

Third, even though this study took place over approximately a one-month period, the average usage time equaled 3.81 hours, with a range of 1.08 to 8.42 hours. A more ideal situation would be to use a similar research construct over a longer period of time as participants continue their work in case-based problem solving in hypermedia environments.

Fourth, the EQ can provide an overview of how students form beliefs--or what it is that students believe--in relation to a series of questions, but a rigorous understanding of epistemological beliefs requires the use of supplementary approaches such as observation or individual or group interviews (Hofer & Pintrich, 1997; MacNeal, 2000; Schraw et al., 2002; Duell & Schommer, 2001; Schommer, 2002; Wood, Kitchener, & Jensen, 2002). In conducting future research, scholars should use multiple measurement approaches, such as unobtrusive audit trail analyses or indepth interviews, and incorporate the use of paper-and-pencil instruments. It appears, therefore, that personal epistemological beliefs are so complex that mixed methodologies might provide better insight into the questions raised in this study.

Implications for Researchers, Theorists, and Instructors

The low-significance or nonsignificant relationships between epistemological beliefs and learning outcomes are consistent with other studies conducted with TPSS programs. Referring to a recent review of empirical studies conducted with the TPSS cases, Riedel, Fitzgerald, Leven, and Toenshoff (2003) reported that well-designed learning systems, particularly those built from constructivist learning principles, might provide equally effective learning environments for special teacher education students, irrespective of learner differences.

Prior research shows that students with naive beliefs in Speed of Learning tend to overrate their ability, knowledge, and experiences (MacNeal, 2000; Schommer, 1990). Therefore, multiple approaches (observation, interview, and unobtrusive audit trail) in combination with self-report instruments should be used in the assessment of epistemological beliefs and in the systematic exploration of the effects that these belief structures have on both knowledge acquisition and knowledge application in computer-based or non-computer-based learning systems (Jacobson et al., 1996).

The findings of this study are consistent with findings about the relationships that link epistemological beliefs to learning; namely, that there are low to moderate relationships between epistemological beliefs and knowledge application but not between epistemological beliefs and knowledge acquisition. Finally, the findings do not support the multi-dimensional view, put forth in Schommer's studies, of the epistemological beliefs of several independent factors.

These results suggest that advocates of the present theory on epistemological beliefs need to continue to examine the multi-dimensionality of epistemological beliefs. Researchers should then study these multiple dimensions to differentiate more specifically the types of learning that are affected in different learning environments, including environments that feature mediating variables (technology-based or nontechnology-based learning) and implementation strategies (learner-centered or instructor-mediated). Future research should focus on a broader learning context, ranging from lower-level learning (knowledge acquisition and comprehension) to higher-order learning (knowledge application, analysis, synthesis, and evaluation). In the future, the design of epistemological theory studies should facilitate the examination of whether epistemological beliefs might not affect learning outcomes in a general sense but might depend on other situational instructional variables.

On the basis of this study, it is apparent that students' epistemological beliefs have some significant but low relationships with learners' abilities to apply knowledge in a case-based hypermedia program. When encountering ill-structured problems, students with naive epistemological beliefs seem to have difficulties integrating ideas and perform poorly in problem-solving activities (Duell & Schommer-Aikins, 2001). Instructors might apply knowledge of students' epistemological beliefs to the design of instructional supports for students. For those students with more naive beliefs, instructors might provide additional support with which to assist them in the completing of problem-solving activities, such as brief orientations about embedded scaffolds within the hypermedia program (Teacher Tools and Tool Resources) and rubrics that guide the self-evaluation of work.

To motivate students and better assess learning and performance, instructors using hypermedia case-based programs should integrate the materials and the content of these programs into their course instruction in a meaningful way. Integration can occur through class discussions of the material and through student learning groups that, from multiple perspectives, work on problem-solving activities. Use of the program can be made a course requirement so that the program's features can be integrated into the grading structure of classrooms; furthermore, this integration would be based on quality of work as well as on any effort expended in the completion of casework. Through these means, motivation can be improved externally through the assignment of grades, collaboratively through class discussions for the social construction of knowledge, and internally through personal reflection on performance and growth.

References

Brunken, R., Plass, J., & Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educational Psychologies, 38(1), 53-61.

Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative research literature on learner comprehension, control and style. Review of Educational Research, 68(3), 322-349.

Duell, O.K., & Schommer-Aikins, M. (2001). Measures of people's beliefs about knowledge and learning. Educational Psychology Review, 13, 419-449.

Fitzgerald, G.E., & Semrau, L.P. (1993-1997). An interactive videodisc program to enhance teacher problem-solving skills for behavioral disorders (Project # H029K30210). Washington, DC: United States Department of Education: Office of Special Education Programs.

Fitzgerald, G.E., Semrau, L.P., & Deasy, G.S. (1997). Interactive multimedia training materials for assessment of students with behavioral problems: Design and outcomes. Journal of Technology and Teacher Education, 5(2/3), 231-250.

Fitzgerald, G.E., & Semrau, L.P. (1998). The effects of learner differences on usage patterns and learning outcomes with hypermedia case studies. Journal of Educational Multimedia and Hypermedia, 7(4), 309-331.

Hartley, K., & Bendixen, L.D. (2003). The use of comprehension aids in a hypermedia environment: Investigating the impact of metacognitive awareness and epistemological beliefs. Journal of Educational Multimedia and Hypermedia, 12(3), 275-289.

Hofer, B.K., & Pintrich, P.R. (1997, Spring). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67, 88-140.

Jacobson, M.J., Maouri, C., Mishra, P., & Kolar, C. (1996). Learning with hypertext learning environments: Theory, design and research. Journal of Educational Multimedia and Hypermedia, 5(3/4), 239-281.

Jacobson, M.J., & Spiro, R.J. (1995). Hypertext learning environments, cognitive flexibility, and the transfer of complex knowledge: An empirical investigation. Journal of Educational Multimedia and Hypermedia, 12(5), 301-333.

Kardash, C.M., & Scholes, R.J. (1996). Effects of preexisting beliefs, epistemological beliefs, and need for cognition on interpretation of controversial issues. Journal of Educational Psychology, 88(2), 260-271.

Kraus, L., Reed, W., & Fitzgerald, G.E. (2000). The effects of learning style and hypermedia prior experience on behavioral disorders knowledge and time on task: A case-based hypermedia environment. Computers in Human Behavior, 17(1), 125-140.

Land, S.M., & Greene, B.A. (2000). Project-based learning with the world wide web: A qualitative study of resource integration. Educational Technology Research and Development, 48(1), 45-68.

MacNeal, L.G. (2000). Epistemic beliefs and comprehension in a hypertext system. Widener University, Chester, PA.

Mayer, R.E. (2001). Multimedia learning. New York: Cambridge University Press.

Mayer, R.E., & Moreno, R. (2003). Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52.

Perry, W.G. (1970). Forms of intellectual and ethical development in the college years: A scheme. New York: Holt, Reinhart and Winston.

Paivio, A. (1986). Mental representation: A dual coding approach. New York: Oxford University Press.

Riedel, J., Fitzgerald, G.E., Leven, F., & Toenshoff, B. (2003). The design of computerized practice fields for problem solving and contextualized transfer. Journal of Educational Multimedia and Hypermedia, 12(4), 377-398.

Scheurman, G. (1997). Using principles of Constructivism to promote reflective judgment: A model lesson. Journal on Excellence in College Teaching, 8(2), 63-86.

Schoenfeld, A.H. (1983). Beyond the purely cognitive: Belief systems, social cognition, and metacognitions as driving forces in intellectual performances. Cognitive Science, 7, 239-363.

Schommer, M. (1990, September). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82, 498-504.

Schommer, M. (1994a). An emerging conceptualization of epistemological beliefs and their role in learning. In R. Garner & P. A. Alexander (Eds.), Beliefs about text and instruction with text (pp. 25-40). Hills-dale, NJ: Lawrence Erlbaum.

Schommer, M. (1994b). Synthesizing epistemological beliefs research: Tentative understandings and provocative confusions. Educational Psychology Review, 6(4), 293-319.

Schommer, M. (1998). The influence of age and education on epistemological beliefs. British Journal of Educational Psychology, 68, 551-562.

Schommer-Aikins, M (2002). An evolving theoretical framework for an epistemological belief system. In B. K. Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (Vol. 1, pp. 103-118). Mahwah, NJ: Lawrence Erlbaum.

Schraw, G., Dunkle, M.E., & Bendixen, L.D. (1995). Cognitive processes in well-defined and ill-defined problem solving. Applied Cognitive Psychology, 9, 523-528.

Takacs, J., Reed, W., Wells, J.G., & Dombrowski, L. (1999). The effects of online multimedia project development, learning style, and prior computer experiences on teachers' attitudes toward the Internet and hypermedia. Journal of Research on Computing in Education, 31(4), 341-355.

Tergan, S.O. (1997). Misleading theoretical assumptions in hypertext/hypermedia research. Journal of Educational Multimedia and Hypermedia, 6(3/4), 257-283.

Tergan, S.O., Sparkes, J.J., Hitchkock, C., Kaye, A.R., Hron, A., & Mandl, H. (1992). Computer-based systems for open learning: State of the art. In S.O. Tergan, J.J. Sparkes, C. Hitchkock, A.R. Kaye, A. Hron & H. Mandl, et al. (Eds.), Open learning and distance education with computer support. Multimediales Lernen in der Berufsbildung (Vol. 4, pp. 97-182). Nurnberg: BW Bildung und Wissen.

Tolhurst, D. (1995). Hypertext, hypermedia, multimedia defined? Educational Technology, 35(2), 21-26.

Tolhurst, D., & Debus, R.L. (2002). The influence of prior knowledge, attitude, ability and activity structure on students' learning and use of software. Journal of Educational Computing Research, 27(3), 275-313.

Wood, P., Kitchener, K., & Jensen, L. (2002). Considerations in the design and evaluation of a paper-and-pencil measure of epistemic cognition. In B.K. Hofer & P.R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (Vol. 1, pp. 277-294). Mahwah, NJ: Lawrence Erlbaum.

APPENDIX 1

Rubric and Example of Needs Analysis Problem-Solving Activity

Identify Needs

Student: Student 1

Date Completed:

Date Scored: 1/30/2003</p> <pre> Beginning -- 1/3 Developing -- 2/3 Stated Objective Performance

Performance or Performance characteristics

characteristics reflecting a beginning reflecting a development level of performance

toward mastery Indicator 1 Indicator 2 Completeness Only 1 need is identified 2-3 needs are identified (4/5 steps completed). (4/5 steps completed). Score = 1 Score = 2 Identification Description of need Desired alternative has a Correctness includes case-based direct correspondence to Analysis of an information. identified need. academic need. Scored Identification Description of need

Desired alternative has a Correctness includes case-based

direct correspondence to Analysis of information.

identified need. behavioral need. Scored

Scored Identification Description of need Desired alternative has a Correctness includes case-based direct correspondence to Analysis of information. identified need. social skill need. Identification Description of need

Desired alternative has a Correctness includes case-based

direct correspondence to Analysis of any information.

identified need. other (one) Scored analysis (Choose the most complete or "best" for scoring). Accomplished -- 3/3 Points Stated Objective Performance characteristics reflecting a 0-3 or Performance master of performance

Indicator 3 Completeness 4-5 needs are identified (4/5 steps 3 completed).

Score = 3 Identification Analysis demonstrates sensitivity to the 1 Correctness child's perspective. Analysis of an academic need. Identification Analysis demonstrates sensitivity to the 3 Correctness child's perspective. Analysis of Scored behavioral need. Identification Analysis demonstrates sensitivity to the 0 Correctness child's perspective. Analysis of social skill need. Identification Analysis demonstrates sensitivity to the 2 Correctness child's perspective. Analysis of any Scored other (one) analysis (Choose the most complete or "best" for scoring). Comments:

9 9 needs were identified. </pre> <p>APPENDIX 2

Rubric and Example of Instructional Plan Problem-Solving Activity

Instructional Plan & Reflection

Student: Student 4

Date Completed:

Date Scored: 2/2/2003</p> <pre> Beginning -- 1/3 Developing -- 2/3 Stated Objective or Performance

Performance Performance characteristics characteristics reflecting a beginning reflecting a development level of performance toward mastery Indicator 1 Indicator 2 Completeness

Only one instructional Two instructional plans

plan is designed (4/6 are designed (4/6 areas

areas completed). completed). Information

Curriculum areas are Curriculum areas are Information

selected that can be selected that were provided in the

observed in the videos described in the case case is used to

showing child's records and/or IEP (must identify

instruction. address documented needs). curriculum.

Scored Scored * Instruction Child's background Instructional approach Planning information is relevant and/or Complete planning to the instruction. materials/activities process is evident. Scored "fit" the instruction.

Scored Specified

At least one area of The proposed plan is based Curriculum

cognitive-behavioral  on steps of direct Curriculum

instruction is instruction. planning includes developed (4 out of 6 Scored cognitive- areas). behavioral

Scored instruction. * Reflections Reflections are

Reflections are provided Instructional plans provided that are that demonstrate critical demonstrate consistent with thinking by applying movement to recommendations made by strategies for transfer thinking like an the experts (make

(make overall judgment). expert. overall judgment on Scored similarity to expert

thinking). Scored

Accomplished -- 3/3 Points Stated Objective or Performance characteristics reflecting a 0-3 Performance

mastery of performance Indicator 3 Completeness

Three instructional plans are designed (4/6 3

areas completed). Information Curriculum areas are selected that were 3 Information described in the needs analysis activity. provided in the Scored case is used to identify curriculum. * Instruction The evaluation strategy "fits" the

3 Planning instruction. Complete planning process is evident. Specified The evaluation strategy "fits" the

3 Curriculum cognitive-behavioral skill. Curriculum

Scored planning includes cognitive- behavioral instruction. * Reflections Answers to best practice prompts are 3 Instructional plans consistent with the plan. demonstrate

Scored movement to thinking like an expert. Comments:

15 * Select the one most complete or "best" plan for scoring and use that plan for all indicators. This student completed four plans. </pre> <p>APPENDIX 3

Epistemological Questionnaire

Schommer Epistemological Questionnaire (Second revision) [For college students]

c Copyright by Marlene Schommer, Ph. D. 1989

As of December 1998: The College Version of This Questionnaire Will Be Published in The British Journal of Educational Psychology, 68, 551-562. (This journal should be cited when using this questionnaire)

This questionnaire was used as an instrument in my dissertation, which is available through University Microfilm Incorporated (UMI) (1-800-521-0600). The dissertation number is 89-24938. A journal article from this dissertation has been published in 1990 in the Journal of Educational Psychology, 82(3), 498-50.

For further inquiry write to:

Dr. Marlene Schommer-Aikins

College of Education #123

Wichita State University 

Wichita, KS 67260-0123

[phone]

E-mail: [e-mail]

Epistemological Questionnaire

Second Draft

By Marlene Schommer-Aikins

Directions: There are no right or wrong answers for the following questions. We want to know what you really believe. For each statement fill in the circle on the answer sheet for the degree to which you agree or disagree. (FN: QUESA.DIS)</p> <pre> Strongly Disagree

Strongly Agree 1 2

3 4 5 No. Questions 1. If you are ever going to be able to understand 1 2 3 4 5 something, it will make sense to you the first time you hear it. 2. The only thing that is certain is uncertainty 1 2 3 4 5

itself. 3. For success in school, it's best not to ask too many 1 2 3 4 5 questions. 4. A course in study skills would probably be valuable. 1 2 3 4 5 5. How much a person gets out of school mostly depends 1 2 3 4 5 on the quality of the teacher. 6. You can believe almost everything you read. 1 2 3 4 5 7. I often wonder how much my teachers really know.

1 2 3 4 5 8. The ability to learn is innate.

1 2 3 4 5 9. It is annoying to listen to a lecturer who cannot 1 2 3 4 5 seem to make up his mind as to what he really believes. 10. Successful students understand things quickly. 1 2 3 4 5 11. A good teacher's job is to keep his students from 1 2 3 4 5 wandering from the right track. 12. If scientists try hard enough, they can find the 1 2 3 4 5 truth to almost anything. 13. People who challenge authority are over-confident. 1 2 3 4 5 14. I try my best to combine information across chapters 1 2 3 4 5 or even across classes. 15. The most successful people have discovered how to 1 2 3 4 5 improve their ability to learn. 16. Things are simpler than most professors would have 1 2 3 4 5 you believe. 17. The most important aspect of scientific work is 1 2 3 4 5 precise measurement and careful work. 18. To me studying means getting the big ideas from the 1 2 3 4 5 text, rather than details. 19. Educators should know by now which is the best

1 2 3 4 5 method, lectures or small group discussions. 20. Going over and over a difficult textbook chapter 1 2 3 4 5 usually won't help you understand it. 21. Scientists can ultimately get to the truth. 1 2 3 4 5 22. You never know what a book means unless you know the 1 2 3 4 5 intent of the author. 23. The most important part of scientific work is

1 2 3 4 5 original thinking. 24. If I find the time to re-read a textbook chapter, I 1 2 3 4 5 get a lot more out of it the second time. 25. Students have a lot of control over how much they 1 2 3 4 5 can get out of a textbook. 26. Genius is 10% ability and 90% hard work. 1 2 3 4 5 27. I find it refreshing to think about issues that 1 2 3 4 5 authorities can't agree on. 28. Everyone needs to learn how to learn. 1 2 3 4 5 29. When you first encounter a difficult concept in a 1 2 3 4 5 textbook, it's best to work it out on your own. 30. A sentence has little meaning unless you know the 1 2 3 4 5 situation in which it is spoken. 31. Being a good student generally involves memorizing 1 2 3 4 5 facts. 32. Wisdom is not knowing the answers, but knowing how 1 2 3 4 5 to find the answers. 33. Most words have one clear meaning. 1 2 3 4 5 34. Truth is unchanging.

1 2 3 4 5 35. If a person forgot details, and yet was able to come 1 2 3 4 5 up with new ideas from a text, I would think they were bright. 36. Whenever I encounter a difficult problem in life, I 1 2 3 4 5 consult with my parents. 37. Learning definitions word-for-word is often 1 2 3 4 5 necessary to do well on tests. 38. When I study, I look for the specific facts. 1 2 3 4 5 39. If a person can't understand something within a 1 2 3 4 5 short amount of time, they should keep on trying. 40. Sometimes you just have to accept answers from a 1 2 3 4 5 teacher even though you don't understand them. 41. If professors would stick more to the facts and do 1 2 3 4 5 less theorizing, one could get more out of college. 42. I don't like movies that don't have an ending. 1 2 3 4 5 43. Getting ahead takes a lot of work.

1 2 3 4 5 44. It's a waste of time to work on problems which have 1 2 3 4 5 no possibility of coming out with a clear-cut and unambiguous answer. 45. You should evaluate the accuracy of information in a 1 2 3 4 5 textbook, if you are familiar with the topic. 46. Often, even advice from experts should be 1 2 3 4 5 questioned. 47. Some people are born good learners, others are just 1 2 3 4 5 stuck with limited ability. 48. Nothing is certain, but death and taxes.

1 2 3 4 5 49. The really smart students don't have to work hard to 1 2 3 4 5 do well in school. 50. Working hard on a difficult problem for an extended 1 2 3 4 5 period of time only pays off for really smart students. 51. If a person tries too hard to understand a problem, 1 2 3 4 5 the will most likely just end up being confused. 52. Almost all the information you can learn from a 1 2 3 4 5 textbook you will get during the first reading. 53. Usually you can figure out difficult concepts if you 1 2 3 4 5 eliminate all outside distractions and really concentrate. 54. A really good way to understand a textbook is to 1 2 3 4 5 re-organize the information according to your own personal scheme. 55. Students who are "average" in school will remain 1 2 3 4 5 "average" for the rest of their lives. 56. A tidy mind is an empty mind.

1 2 3 4 5 57. An expert is someone who has a special gift in some 1 2 3 4 5 area. 58. I really appreciate instructors who organize their 1 2 3 4 5 lectures meticulously and then stick to their plan. 59. The best thing about science courses is that most 1 2 3 4 5 problems have only one right answer. 60. Learning is a slow process of building up knowledge. 1 2 3 4 5 61. Today's facts may be tomorrow's fiction. 1 2 3 4 5 62. Self-help books are not much help.

1 2 3 4 5 63. You will just get confused if you try to integrate

1 2 3 4 5 new ideas in a textbook with knowledge you already

have about a topic. </pre> <p>Note

From Duell, O. K., & Schommer-Aikins, M. (2001). Measures of people's beliefs about knowledge and learning. Educational Psychology Review, 13, 419-449.

Copyright by Marlene Schommer, Ph. D., 1989

Reprinted with permission.

HSINYI PENG 

National Chiao Tung University 

Hsinchu, Taiwan

[e-mail]

GAIL E. FITZGERALD

University of Missouri-Columbia

Columbia, MO USA

[e-mail]

Table 1 Factors and Sample Items from the EQ

Factor            Subset                     Sample Items

Ability to Learn  Can't learn how to learn   Self-help books are not
                                             much help.
                  Success unrelated to hard  The really smart students
                  work                       don't have to work hard to
                                             do well in school.
                  Learn the first time       Almost all the information
                                             you can learn from a
                                             textbook you will get
                                             during the first reading.
                  Innate ability             Students who are "average"
                                             in school remain "average"
                                             for the rest of their
                                             lives.
Structure of      Avoid ambiguity            It is annoying to listen to
Knowledge                                    a lecture who cannot seem
                                             to make up his mind as to
                                             what he really believes.
                  Seek single answers        Most words have one clear
                                             meaning.
                  Avoid integration          When I study, I look for
                                             the specific facts.
                  Depend on authority        How much a person gets out
                                             of school mostly depends on
                                             the quality of the teacher.
Speed of          Quick Learning             Successful students
Learning                                     understand things quickly.
                  Don't criticize authority  You can believe almost
                                             everything you read.
Stability of      Certain Knowledge          Truth is unchanging.
Knowledge

Table 2 Distribution of EQ Scores (N = 56)

                                PreEQ Scores
                        Mean   SD    Minimum  Maximum

Ability to Learn         9.14  1.75  5.30     13.32
Structure of Knowledge  13.38  1.68  9.10     17.29
Speed of Learning        4.61   .66  3.37      6.30
Stability of Knowledge   2.75   .66  1.00      4.20

Table 3 Intercorrelations between Epistemological Beliefs and Gain
Scores in Pre-to Postknowledge Instrument (N = 56) (a)

Subscale                Gain Scores on Pre- to Postknowledge Instrument

Ability to Learn        -.04
Structure of Knowledge  -.18
Speed of Learning       -.05
Stability of Knowledge  -.04

(a) * p < .05, one-tailed

Table 4 Intercorrelations between Epistemological Beliefs and Case
Artifact Scores (N = 56) (b)

                   Artifact        Artifact          Artifact
Subscale           Identify Needs  Setting Analysis  Management Plan

Ability to Learn    .02            -.31 *            -.38 **
Structure of       -.08             .09              -.08
  Knowledge
Speed of Learning   .03            -.02               .00
Stability of        .05             .11               .13
  Knowledge

Subscale            Artifact Instructional Plan  Total Artifact Scores

Ability to Learn    -.11                         -.01
Structure of        -.09                         -.16
  Knowledge
Speed of Learning    .33 *                       -.04
Stability of         .13                         -.20
  Knowledge

(b) ** p < .01, one-tailed; * p < .05, one-tailed

Table 5 Summary of Stepwise Regression Analysis for Variables Predicting
Score of Artifact Setting Analysis (N = 56) (c)

                        Unstandardized Coefficients  Standardized
                                                     Coefficients
Variable                B      Std. Error            Beta

Model 1
  (Constant)            16.18  3.81
  Pre Ability to Learn   -.85   .41                  -.31

(c) [R.sup.2] = .09 and Adjusted [R.sup.2] = .07 for Model 1

Table 6 Summary of Stepwise Regression Analysis for Variables Predicting
Score of Artifact Management Plan (N = 56) (d)

                        Unstandardized Coefficients  Standardized
                                                     Coefficients
Variable                B      Std. Error            Beta

Model 1
  (Constant)            16.73  2.36
  Pre Ability to Learn   -.64   .25                  -.38

(d) [R.sup.2] = .15 and Adjusted [R.sup.2] = .12 for Model 1

Table 7 Summary of Stepwise Regression Analysis for Variables Predicting
Score of Artifact Instructional Plan (N = 56) (e)

                               Unstandardized Coefficients  Standardized
                                                            Coefficients
Variable                       B     Std. Error             Beta

Model 1
  (Constant)                    .27  3.84
  Pretest score in Ability to  1.81   .84                   .33
    Learn

(e) [R.sup.2] = .11 and Adjusted [R.sup.2] = .09 for Model 1