The use of comprehension aids in a hypermedia environment: investigating the impact of metacognitive awareness and epistemological beliefs.

         This is a descriptive study that analyzes learners' use of
         comprehension aids such as objectives, a glossary, and links
         between pages in a hypermedia tutorial. Students read a short
         hypermedia tutorial on the topic of E. coli. Relationships
         between the use of the comprehension aids and individual
         characteristics such as metacognitive awareness and
         epistemological beliefs were evaluated. It was hypothesized
         that the more metacognitively skilled learners and those with
         more sophisticated beliefs about learning would use the aids
         more frequently. The findings indicate that the uses of some
         comprehension aids are significantly related to some
         epistemological beliefs (e.g., innate ability, omniscient
         authority, and quick learning) and certain types of
         metacognitive knowledge (e.g., regulation of cognition).

**********

JI. of Educational Multimedia and Hypermedia (2003) 12(3), 275-289

Hypermedia tutorials are widely used in classrooms throughout the country. While the textbook is still the primary source of course content, hypermedia documents are frequently supplementing and in some cases replacing the book. Web pages are an example of hypermedia--artifacts displayed by a computer that use multiple media and click-able links to present related information. With the increasing popularity of the Internet as an educational tool, learners are being asked more frequently to use hypermedia materials to access information and learn important concepts. These instructional materials often provide a variety of comprehension aids to assist the reader. The most common aid is a glossary that can be accessed by clicking on a hyperlinked word to see the definition or a representative image. Others include a list of objectives, advanced organizers, self-test questions and links to related text. Hypermedia formats provide the opportunity to access these aids "when needed." In other words, they will not appear unless the learner chooses to see them. It is assumed that the inclusion of these tools can improve comprehension.

Learning With Hypermedia

Research on learning in a hypermedia environment is becoming more prevalent and what has been learned largely comes from research concerned with outcomes rather than approaches. For example, Mayer (2001) developed a set of multimedia learning principles, which are based on research comparing student achievement after using a variety of tutorials. Much of Mayer's research is based upon multimedia tutorials on topics such as the formation of lightning or how a tire pump operates. After viewing these tutorials with a variety of interventions, participants in these studies are asked to recall as much as possible (near transfer) and apply what was learned to new situations (far transfer). This line of research is typical in multimedia studies where outcomes are emphasized while processes (i.e., approach to learning) and learner characteristics are controlled.

A review by Dillon and Gabbard (1998) explored the hypermedia research literature with a focus on learning comprehension, learner control, and style (i.e., learner characteristics). The reviewed studies used, almost exclusively, achievement measures as dependent variables. They concluded that there is substantial evidence that individual characteristics such as ability, passive/active dispositions, field independence/dependence tendencies and deep/shallow processors play a role in learning from hypermedia instruction (Dillon & Gabbard, 1998). There is a strong need to understand why these characteristics may impact comprehension in a hypermedia environment.

One possible source of variability in the successful use of a hypermedia tutorial is the use of comprehension aids. Like traditional textbooks, hypermedia instructional materials often contain comprehension aids such as advanced organizers, lists of objectives, a glossary and self-check questions. The value of comprehension aids such as advanced organizers has been clearly demonstrated in reading research (Moore & Readance, 1984). Studies that incorporate advanced-organizers into computer based instruction have also demonstrated a positive effect (Kenny, 1993). One important consideration is the use of these comprehension aids in a hypermedia environment. In a hypermedia tutorial, providing the option to access these tools does not ensure use. Two attributes that may impact the use of comprehension aids include metacognitive knowledge and epistemological beliefs (Hartley & Bendixen, 2001).

Important Learner Characteristics

The list of learner attributes that may impact successful learning in a hypermedia environment is certainly long and the interactions complex. The Dillon and Gabbard review (1998) provided compelling evidence that learner characteristics play a key role in learning with computer based instructional materials. While much of the available literature has investigated aptitude (e.g., reading comprehension or prior achievement) or cognitive styles (e.g., active/passive learners or field dependence) very little research has investigated implications of more general attributes such as epistemological beliefs and metacognitive knowledge. This is an important distinction because these more general attributes may have a pronounced impact in the use of instructional materials that provide many more processing options (e.g., opportunities to approach instruction in a non-linear fashion) than traditional instructional materials (Bendixen & Hartley, 2003).

Students' metacognitive knowledge will significantly mediate success in most learning environments (Schraw, 1998). Metacognition literally means thinking about thinking. Metacognition has been subdivided into knowledge and regulation of cognition (Jacobs & Paris, 1987). Knowledge of cognition refers to a learner's understanding of his or her own thought processes (Schraw, 1998). Knowledge of cognition includes declarative (about), procedural (how), and conditional (when) knowledge (Jacobs & Paris, 1987; Schraw, 1998). The use of a strategy is dependent on the student's awareness of the strategy (declarative), understanding of how the strategy works (procedural) and knowing when to use the strategy (conditional). Regulation of cognition is understood as those behaviors that demonstrate control of, and/or utility with, one's knowledge of cognition. Examples would include a student's ability to monitor, evaluate and set goals for their learning (Schraw, 1998). Students who lack skills such as monitoring for understanding and goal setting tend to struggle in many learning environments (Zimmerman & Bandura, 1994).

In hypermedia environments, users are faced with a constant barrage of decisions to be made. A simple example would be the "hyperlinked" term that the user will face in most hypermedia documents. The decision whether or not to follow the link should be related to the learners ability to monitor their understanding. Poor monitors will likely benefit less from the convenience of hyperlinked terms and may actually be placed at a disadvantage, as the distraction will consume valuable cognitive resources. These concerns have led to a number of studies that attempted to incorporate learning strategy instruction into hypermedia instructional materials (Hartley, 2001; Puntambekar & duBoulay, 1997). However, research to support the notion that metacognitive knowledge will positively impact achievement in a hypermedia environment is not yet available, although it is likely more important in contexts where the learners have more control over the sequence of instruction and the use of comprehension aids (Bendixen & Hartley, 2003).

In addition to metacognitive knowledge, another important learner attribute is the learner's epistemological beliefs. Epistemology refers to the study of the nature of knowledge. The epistemological beliefs held by students may have important influences on thinking and problem solving. Schommer (1990) proposed five independent epistemological dimensions corresponding to beliefs about knowledge. Each dimension is based on a continuum from naive to mature. The following lists the naive end of the continuum for each dimension:

1. certain knowledge (i.e., absolute knowledge exists and will eventually be known),

2. simple knowledge (i.e., knowledge consists of discrete facts),

3. omniscient authority (i.e., authorities have access to otherwise inaccessible knowledge),

4. quick learning (i.e., learning occurs in a quick or not-at-all fashion), and

5. fixed ability (i.e., ability to acquire knowledge is innate).

Research conducted with traditional instructional materials indicates that certain epistemological beliefs correlate with achievement. For example, Schommer, Crouse, and Rhodes (1992) reported that beliefs in simple knowledge negatively affected complex problem solving. Schoenfeld (1983) investigated some of the consequences of a belief in quick learning. He reported that even experienced students who were asked to solve math problems gave up after five to ten minutes on the assumption that if they failed to solve the problem during this time, the problem could not be solved. Jacobson and Spiro (1995) found that students with naive epistemological beliefs struggled with the nonlinearity of the hypertext environments. Bendixen and Hartley (2003) found that students with higher scores (i.e., more naive) on the factors omniscient authority and innate ability recalled less of the content presented in a simple hypermedia tutorial then their peers. The same study found that a belief in quick learning positively impacted achievement with the tutorial--a finding inconsistent with the Schoenfeld study (1983) but likely due to the differences in task (mathematical problem solving versus a brief hypermedia tutorial) and outcome measures (the hypermedia tutorial study focused on recall).

Current Study

Research to this point on the impact of metacognitive knowledge and epistemological beliefs in hypermedia, has focused largely on outcomes such as recall and has produced some findings that are inconsistent with research with more traditional instructional materials. What has not been investigated is how these learner characteristics impact their approach to hypermedia. The present research focuses on the direct impact of metacognitive knowledge and epistemological beliefs on the use of comprehension aids in a hypermedia tutorial. Metacognitive knowledge, as exemplified by students who are more actively engaged in materials, should impact the use of comprehension aids. The current study looks at two proposed facets of metacognition, regulation of cognition, and knowledge of cognition. Regulation of cognition refers to higher-level activities such as monitoring for understanding and goal setting. Knowledge of cognition refers to the knowledge one has about their cognition (e.g., learning strategies and memory limitations). In addition, the current study looks at the relationship between several epistemological beliefs and the use of a hypermedia tutorial. Naive epistemological beliefs, such as a belief in quick learning or simple knowledge, would likely result in the decreased use of comprehension aids. For example, if learning is quick and knowledge is simple, there is little reason to believe additional tools such as advanced organizers would be necessary.

Two hypotheses will be tested. The first hypothesis predicts that there will be a positive correlation between the use of comprehension aids and metacognitive knowledge. The second hypothesis is that there will be a negative correlation between the use of comprehension aids and naive epistemological beliefs.

METHOD

Participants

One hundred one students (78 females, 23 males) from a large southwestern university participated in this experiment to fulfill a class requirement.

Materials

Materials included a packet containing a 32-item Epistemological Beliefs Inventory (EBI), a 52-item Metacognitive Awareness Inventory (MAI) and a brief demographic variable worksheet.

Epistemological beliefs were measured using the EBI developed by Schraw, Bendixen and Dunkle (2002). This inventory is based on Schommer's (1990) five dimensions of epistemological beliefs which include (a) certain knowledge, (b) simple knowledge, (c) omniscient authority, (d) fixed ability, and (e) quick learning. All items were written using a 5-point Likert-type scale. See Table 1 for sample items.

Students' metacognitive knowledge was measured using the Metacognitive Awareness Inventory (MAI) developed by Schraw and Dennison, (1994) in which items are classified into two categories of metacognition (i.e., knowledge of cognition and regulation of cognition). All items were written using a 5-point Likert-type scale. See Table 1 for sample items.

Hypermedia Tutorial

The topic of the tutorial was the bacteria, E. coli. The topic was chosen due the general interest that existed at the time. The tutorial addressed basic biology topics such as the classification, reproduction, and anatomy of the bacteria. Also included were tips on avoiding E. coli contamination and news articles concerning recent outbreaks.

The tutorial screen was partitioned into four areas (Figure 1). The top 10% of the screen, the banner, contained links to an advanced organizer, objectives, glossary, site map (map representing the organization of the content) and self check questions. The banner was always visible. The other 90% of the screen was divided into three columns, the menu, content and supplementary information. The menu contained links to the four major topics in the tutorial. The content contained the text of the tutorial, which included links to relevant terms, images and other text pages. The supplementary information section displayed definitions and images when selected from the content section. For example, if while reading about E. coli, the user was confused by the term bacteriophage they could click on the term and a definition would appear in the supplementary information section.

[FIGURE 1 OMITTED]

While navigating through the tutorial, the students' use of the banner items, links to the glossary and links to other pages was recorded by a central computer. The use of each of the banner items was recorded as either zero (did not use) or one (did use). The participant's progression through the tutorial was scored in terms of deviation from linearity. The standard mode of progression through the content would be to access the pages in the order of the listed hyperlinks (the equivalent of reading a book straight through without skipping ahead or looking back). Hypermedia tutorials provide quick and easy access to related items in the tutorial through hyperlinks. Thus, a discussion of a recent E. coli outbreak could contain links to an introductory passage about E. coli. The selection of this type of link (or other links which took them to a page other than the next page in sequence) was noted as a "nonlinear" move. The number of nonlinear moves was recorded for each participant. Examples of participant's progression through the tutorials can be seen in Figures 2 (linear) and 3 (nonlinear).

[FIGURE 2 OMITTED]

Procedure

This study was a part of a larger study concerned with learner beliefs and hypermedia instruction. As a part of the larger study, students also completed, in a different session, a tutorial whose design was identical and the content was centered on the topic Yugoslavia. To account for any differences related to practice effects, some of the participants completed the E. coli during their first session while others completed the Yugoslavia tutorial. In the subsequent session, the content was switched. During the initial sessions, students completed the EBI and MAI, while the reading comprehension and syllogistic reasoning tests were completed during the second session.

[FIGURE 3 OMITTED]

Groups of five to 20 participants completed the study in a computer lab. Each group was given a brief demonstration of the tutorial. The demonstration included showing the participants each of the comprehension aids and how to navigate through the content. The participants were told that they should study the material as they would for a class and that a test over the objectives would follow. The participants were given 30 minutes to study the tutorial.

RESULTS

Means and standard deviations for each of the factors in the EBI and MAI can be found in Table 2. Correlations between the use of comprehension aids and factors in the MAI and EBI can be found in Table 3. Seven factors (2 MAI and 5 EBI) were compared to the use of six comprehension aids resulting in 42 possible relationships. Of the 42 combinations, seven were statistically significant (alpha = .05, one-tailed). No significant correlations were found between the use of comprehension aids and the users knowledge of cognition as measured by the MAI. Only the number of non-linear moves was found to be related to the regulation of cognition factor measured by the MAI (r[=.sup.+]0.20, n=101).

In terms of epistemological beliefs (note that higher EBI scores indicate less sophisticated epistemological beliefs), a strong positive correlation was found between a belief in innate ability and the use of the self-check questions (r=[+.sup.0].27, n=101). Positive correlations were also found between a belief in quick learning and the use of the advanced organizer (r=[+.sup.0].18, n=101), the glossary (r=[+.sup.0].19, n=101), the site map (r=[+.sup.0].16, n=101), and the self-check questions (r=[+.sup.0].16, n=101). This direction is the opposite of what was predicted. One negative correlation (predicted direction) was found between the Omniscient authority factor and the number of non-linear moves (r=[-.sup.0].17, n=101). No significant correlations were found with the other two factors in the EBI--belief in simple knowledge or certain knowledge.

In summary, the first hypothesis, that there would be a positive correlation between the use of comprehension aids and metacognitive skills was partially supported. The regulation of cognition factor was positively correlated with the use of nonsequential text pages but unrelated to the use of the other comprehension aids. The knowledge of cognition factor was not related to the use of any comprehension aids.

Contrary to the second hypothesis, that there would be a negative correlation between the use of comprehension aids and naive epistemological beliefs, a positive correlation was found between the use of several of the comprehension aids and quick learning. In other words, students who tended to believe that learning is quick were more likely to use the advanced organizer, glossary, site-map, and self-check questions. A positive correlation was also found between the belief in innate ability and use of the self-check questions. Beliefs in simple and certain knowledge were not related to the use of any comprehension aids.

DISCUSSION

Previous research would lead us to believe that the beliefs measured by the MAI and EBI would have a substantial impact on how learners approach these tutorials. The study provides mixed results in terms of the relationships between epistemological beliefs and metacognitive awareness and the use of a hypermedia tutorial.

One possible explanation for the mixed results is the indirect nature of the dependent variable, use of comprehension aids. The use or disuse of a comprehension aid does not by itself indicate an inefficient approach to learning. It may be that for some, just the opposite is true. For example, for many the use of self-check questions may improve comprehension, but for the good self-regulator, the self-check questions may not be necessary.

Another possible reason for the mixed results is that the task may not have encouraged the kind of deeper processing that is more commonly associated with increased metacognitive awareness and more sophisticated epistemological beliefs. Students were asked to study the tutorials as they would for a class and were given thirty minutes with which to study. A more authentic and longer task may have produced different results. A study of longer duration would also afford the opportunity to move beyond the dichotomous variable of tool use (yes or no). A study of longer duration could also begin to look at the frequency of comprehension aid use and when it was accessed.

Epistemological Beliefs and Tool Use

The increased use of the comprehension aids by students holding a belief in quick learning is interesting for a couple of reasons. One reason is that the comprehension aids may be viewed as the quick way to get the necessary information. The more reasoned and thoughtful approach may have been to concentrate on reading the text. What is not known, and warrants further investigation, is whether the use of these comprehension aids was in lieu of reading the text or if they were used (appropriately) as adjuncts.

The strongest correlation found in the study was between the belief in innate ability and the use of the self-check questions. The participants who held a belief that ability is innate were much more likely to use the self-check questions. Again, it would be helpful to know if time spent utilizing the self-check questions was at the expense of the reading, but one can imagine how some may feel that using (and even memorizing) the self-check questions could increase the likelihood of performing well on the posttest, especially when the participant believes that ability is fixed.

The relationship between the omniscient authority factor and the use of nonlinear moves was the only epistemological belief correlation that was statistically significant in the expected direction (negative). The preferred mode of progressing through the tutorial content was linear. Some participants however, chose to use links provided in the content pages and elsewhere (e.g., site map and the table of contents) that took them to pages that, although related, weren't listed as the next page. Hypermedia may be the ideal tool for the learner who eschews the idea of authority--at least in terms of the sequence of content. The students who feel more comfortable traversing the content in a more non-linear manner will find the less-controlled nature of most hypermedia materials appealing.

Metacognitive Awareness and the Use of Comprehension Aids

The apparent lack of significant correlations between the participants' knowledge and regulation of cognition is best interpreted in light of the task. While one can easily recognize the importance of regulatory skills such as goal setting on outcomes such as course grades or even the likelihood of earning a diploma, the same skills may not be as immediately rewarding when using a short hypermedia tutorial. The one exception to the lack of significant correlations was the strong, positive relationship between the participants' regulation of cognition and the number of nonlinear moves. Those with superior regulatory skills are better able to approach the content in a way that has been demonstrated to be confusing for some (Niederhauser, Reynolds, Salmen, & Skolmoski, 2000). Niederhauser et al. (2000) found that while the flexibility of hyperlinked text may be appealing, the additional cognitive load incurred can result in confusion.

CONCLUSIONS

To address the need for a better understanding of the use of comprehension aids in light of learner beliefs, further research should gather online data about the rationale that is used by the learner when deciding whether or not to use a comprehension tool. For example, while some students may use a tool frequently, depending upon their current understanding, the use may or may not be an effective use of their time. Presumably, learners referencing the objectives prior to reading are making better use of their time than the student who stops to read the objectives for the first time in the middle of the text.

This descriptive study represents an important beginning in the analysis of how different learners approach hypermedia instruction. Studies such as this combined with qualitative investigations of learning with hypermedia (e.g., Hill & Hannifin, 1997) are crucial to making informed recommendations about the use of hypermedia in schools. While many are quick to adopt new technologies in the service of learning, we need to constantly evaluate how these new tools impact a variety of learners.

Table 1

Factors and Sample Items from the EBI and MAI

Instrument  Factor                   Item

EBI         Certain knowledge        "What is true today will be true
                                     tomorrow."

EBI         Simple knowledge         "Too many theories just complicate
                                     things."

EBI         Omniscient authority     "People should always obey the
                                     law."

EBI         Fixed ability            "How well you do in school depends
                                     on how smart you are."

EBI         Quick learning           "Working on a problem with no
                                     quick solution is a waste of time."

MAI         Regulation of cognition  "I organize my time to best
                                     accomplish my goals."

MAI         Knowledge of cognition   "I understand my intellectual
                                     strengths and weaknesses."

Table 2

Means and Standard Deviations for factors in the EBI and MAI

Variable                  M    SD   Minimum  Maximum

Simple knowledge         3.10  .51   1.50     4.50
Certain Knowledge        2.32  .51   1.10     3.70
Omniscient authority     3.16  .70   1.00     5.00
Quick learning           1.92  .55   1.00     3.80
Fixed ability            2.69  .65   1.43     4.71
Regulation of cognition  3.42  .42   2.56     4.66
Knowledge of cognition   3.67  .39   2.61     4.78

Table 3

Correlations Between Learner Beliefs (EBI and MAI) and the Use of
Comprehension Aids

                      Advanced   Objectives  Glossary  Site
                      Organizer                        map

Omniscient authority    0.09       0.00        0.08     -0.03

Innate ability         -0.02       0.13        0.03     -0.01

Simple knowledge        0.04       0.05        0.14      0.07

Quick learning          0.18 *     0.09        0.19 *    0.16 *

Certain knowledge       0.06      -0.02       -0.01     -0.02

Regulationo of
cognition              -0.01       0.01       -0.09     -0.04

Knowledge of
cognition              -0.14      -0.03        0.01     -0.11

                      Self-check  NonLinear
                                  Moves

Omniscient authority   -0.08       -0.17 *

Innate ability          0.27 **     0.05

Simple knowledge        0.04       -0.07

Quick learning          0.16 *     -0.01

Certain knowledge       0.05       -0.06

Regulationo of
cognition               0.06        0.20 *

Knowledge of
cognition              -0.09       -0.08

n=101

* p<.05, one-tailed
** p<.01, one-tailed

References

Bendixen, L.D., & Hartley, K. (2003). Successful learning with hypermedia: The role of epistemological beliefs and metacognitive awareness. Journal of Educational Computing Research, 28, 15-30.

Dillon, A., & Gabbard, R. (1998). Hypermedia as an educational technology: A review of the quantitative research literature on learner comprehension, control, and style. Review of Educational Research, 68, 322-349.

Hartley, K. (2001). Learning strategies and hypermedia instruction. Journal of Educational Multimedia and Hypermedia, 10, 285-305.

Hartley, K., & Bendixen, L.D. (2001). Educational research in the Internet age: Examining the role of individual characteristics. Educational Researcher, 30(9), 22-26.

Hill, J.R., & Hannafin M.J. (1997). Cognitive strategies and learning from the World Wide Web. Educational Technology research and Development, 45(4), 37-64.

Jacobs, J.E., & Paris, S.G. (1987). Children's metacognition about reading: Issues in definition, measurement, and instruction. Educational Psychologist, 22, 255-278.

Jacobson, M.J., & Spiro, R.J. (1995). Hypertext learning environments, cognitive flexibility, and the transfer of complex knowledge: An empirical investigation. Journal of Educational Computing Research, 12, 301-333.

Kenny, R.F. (1993). The effectiveness of instructional orienting activities in computer-based instruction. Proceedings of selected research and development presentations at the convention of the Association for Educational Communications and Technology. 1-20.

Mayer, R. E. (2001). Multi-media learning. New York: Cambridge University Press.

Moore, D. W., & Readance, J. E. (1984). A quantitative and qualitative review of graphic organizer research. Journal of Educational Research. 78(1), 11-17.

Niederhauser, D.S., Reynolds, R.E., Salmen, D.J., & Skolmoski, P. (2000). The influence of cognitive load on learning from hypertext. Journal of Educational Computing Research, 23, 237-255.

Puntambekar, S., & duBoulay, B. (1997). Design and development of MIST: A system to help students develop metacognition. Journal of Educational Computing Research, 16, 1-35.

Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26, 113-125.

Schraw, G., Bendixen, L.D., & Dunkle, M.E. (2002). Development and validation of the Epistemic Belief Inventory (EBI). In B. K, Hofer & P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 261-275). Mahwah, NJ: Lawrence Erlbaum.

Schraw, G., & Dennison, R.S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 143-154.

Schoenfeld, A. (1983). Beyond the purely cognitive: Belief systems, social cognitions, and metacognitions as driving forces in intellectual performance. Cognitive Science, 7, 329-363.

Schommer, M., (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82, 498-504.

Schommer, M., Crouse, A., & Rhodes, N. (1992). Epistemological beliefs and mathematical text comprehension: Believing it is simple does not make it so. Journal of Educational Psychology, 82, 435-443.

Zimmerman, B.J., & Bandura, A. (1994). Impact of self-regulatory influences on writing course attainment. American Educational Research Journal, 31, 845-862.

KENDALL HARTLEY AND LISA D. BENDIXEN University of Nevada, Las Vegas USA [e-mail] [e-mail]