A preliminary investigation of advance organizers for a complex educational website.

The study employed a 2 (university campus) X 3 (treatment groups) X 2 (time) factorial design with random assignment to examine the use of multimedia advance organizers before an activity that involved locating and synthesizing information at the complex educational website, A Teachers' Guide to the Holocaust (http://fcit.usf.edu/holocaust). Most participants were undergraduate Education majors between the ages of 18 and 21 at two large, state universities. The results indicated no significant differences between advance organizer treatment groups and a control group with no advance organizers. Rationale, method, results, and suggestions for future studies are provided.

**********

Due in part to students and teachers having faster connections to the Internet, there now exists an abundance of web-based instructional resources. One advantage to using these hypermedia resources is the amount of navigational freedom and learner control they provide. For example, whereas students reading a traditional textbook are expected to read through the information in a way that is designated by the author; the Web and other hypermedia environments allow students to choose which information they would like to access and when they would like to access it (Becker & Dwyer, 1994). The types of knowledge representation hypermedia provides may also be closer to schema-based memory structures than linear representations (Jonassen & Grabinger, 1990).

On the other hand, the complexity of the hypermedia links and frequency with which they appear throughout web-based documents can be overwhelming and distracting. As Romiszowski (1990) cautioned:

       There is, however, some doubt as to whether all these process-
       oriented aspects of hypertext systems are necessarily "a good
       thing" in all manner of learning situations. The research on
       learner control of the learning process is, to say the least,
       mixed. There is much evidence to suggest that learners, when free
       to select their own strategies, do not always select wisely. (p.
       322)

When users become disoriented in hypermedia environments, they can lose mental power needed for learning (Jonassen, 1989; Kenny, 1993). Instructional designers are therefore presented with the problem of how to minimize the cognitive limitations of hypermedia, while not eliminating the apparent advantages of nonlinearity.

Organizational or structural aids such as overview diagrams and advance organizers can help outline the structure of hypermedia information (Tripp & Roby, 1990). Large hypermedia documents can be made more accessible to learners by presenting preview material that either relates to familiar content or helps build mental scaffolding on which to add new information. This way, not only does a learner have more cognitive power available to learn new material, but the learning experience becomes more personalized and meaningful. "Content that is unfamiliar or organized in an unfamiliar fashion will be learned poorly unless the individual is provided with or develops concepts or organizing principles that aid the acquisition process" (Clark & Bean, 1982). "Traditionally, a device for orienting students to content has been the advance organizer" (Tripp & Roby, p. 121).

McManus (2000) worked with a sample of 119 students in a Computing Tools for Educators course at a large southwestern university. Participants were divided into six groups (2 advance organizer X 3 levels of nonlinearity) to answer the question, "Is there a significant interaction between levels of non-linearity and the presence of an advance organizer in a web-based hypermedia learning environment" (p. 240)? His results, showed a near significant interaction between the levels of nonlinearity and the presence of an advance organizer (p=.052). Participants who worked in high nonlinear environments performed better when presented with an advance organizer than those in who worked in high nonlinear environments who were not presented with an advance organizer.

ADVANCE ORGANIZERS

Throughout the rather large and often conflicting body of research on advance organizers, Ausubel's (1978) logical definition has held ground. In his Subsumption Theory, Ausubel suggested that learners subsume new data under general, more inclusive concepts and principles. Ausubel's theory is generally consistent with Schema Theory; both theories acknowledge that new information can be retained to the extent that it can be related to an existing cognitive structure (Clark & Bean, 1982). Advance organizers are a practical extension of Subsumption Theory. An advance organizer is "introductory material at a higher level of abstraction, generality, and inclusiveness than the learning passage itself" (Ausubel, 1978, p.252). The purpose of an advance organizer is to provide context rather than content--to provide conceptual scaffolding rather than specific detail from a body of to-be-learned information. Ausubel (1978) describes two types of advance organizers: (a) Expository advance organizers are used when learning material is completely unfamiliar--they help provide inclusive subsumers; (b) Comparative organizers are used when the learning material is more familiar--they provide ideational scaffolding as well as pointing out explicitly the principal similarities and differences between new and previously learned ideas.

Assimilation Encoding Theory predicts, similar to Subsumption Theory and Schema Theory, that learners actively integrate new information with existing knowledge. Assimilation Encoding theory includes three factors: (a) whether information is received into working memory; (b) whether anchoring knowledge is available in the long term memory; (c) whether anchoring knowledge has been transferred to long term memory. If these factors are present--according to theory--successful encoding should result in transfer of new cognitive structure to long term memory (Mayer, 1979a).

Mayer (1979b) reviewed 27 published studies containing an advance organizer group and a control group, and found that there was a small but consistent advantage for the advance organizer group on tests of learning and retention. He found that this advantage was more likely when certain conditions were met. These conditions included: (a) when the materials were unfamiliar (meaning that the material should not elicit any general or subsuming context from the learner); and (b) when the learners were less experienced.

Mayer (1979a) examined the effects of advance organizers on subjects with different attribute levels. That is students with lower ability levels were predicted to have fewer meaningful context with which to anchor new information and thus more likely to have a need for advance organizers than those with higher ability. One hundred seventy-six college freshmen with no prior computer programming knowledge read a text concerning a simple computer programming language. Some of the participants were presented with an advance organizer in the form of a model of the computer with an explanation of each program statement within the context of the model. Mayer found that a treatment (advance organizer) x interaction effect was found mainly for participants who scored below average on SAT mathematics (Mayer).

Mayer (1979a) tested another premise of Assimilation encoding theory concerning text organization--that if text is organized in a logical manner, advance organizers are less likely to be effective. He tested this idea with 56 nonprogrammer college students who read a 24 frame text on computer programming that was either in logical or random order. The advance organizer was a short text that described a model of the computer in familiar terms. Results were consistent with the theory.

Mayer (1979a) clarified, however, that these studies were limited in that few types of instructional materials were used (e.g., technical in nature). Mayer also explained how current research suggests math and science as subjects areas appear to be most influenced by advance organizers. The current study was designed in part to test Mayer's conclusions described here by examining the effects of advance organizers in a new context--namely a complex, historical website.

Ausubel's original research (and many studies thereafter) investigated the use of textual organizers on verbal learning. Mayer (1979b) proposed that an advance organizer could be composed of a short set of verbal or visual information. In Mayer's (1997) Generative Theory of Multimedia Learning, multimedia may affect the degree with which learners process information through two separate processing systems: a verbal system for processing verbal knowledge and a visual system for processing visual knowledge. Mayer (2001) described how learners can arrive at understanding through actively building meaningful connections between verbal and visual representations. Furthermore, there is considerable evidence to support the potential power in combining media forms in instructional material (Herron, Hanley, & Cole, 1995; Kiewra, Mayer, & DuBois, 1997; Mayer 2001; Ritchie & Gimenez, 1995; Robinson, Katayama & DuBois, 1998).

Some researchers have put forward that different combinations verbal and visual organizers can facilitate learning in varying degrees (Luiten, Ames, & Ackerson, 1980; Stone, 1984). Other researchers have shown that specifically advance organizers with combined verbal and visual elements may be more useful than advance organizers that are verbal or visual alone (Chun & Plass, 1996; Mayer 1997; Mayer, 2001; Plass, Chun, Mayer & Leutner, 1998). Although results from advance organizer efficacy tests (using a variety of media and content) have not always been consistent, they have been generally positive (Luiten, Ames, & Ackerson; Stone; Kenny, 1993; Mayer, 1979a, 1979b, 1997; Plass, Chun, Mayer, & Leutner).

In summary, there may be an advantage to using of advance organizers prior to working with nonlinear material given certain conditions are met--especially for learners who have lower levels of domain knowledge or experience, such as preservice teachers. It may also be beneficial to some students to use advance organizers that include both verbal and visual components.

CONTEXT

Based on research supporting the benefits and importance of Holocaust education, several states, including California (Assembly Bill 3216), New Jersey (State legislative Bill 2780), and Florida (Holocaust Education Bill 660) have adopted legislation requiring Holocaust Education in public schools. A Teacher's Guide to the Holocaust was designed and developed for all inservice and preservice teachers in Florida as an answer to the 1994 Florida Mandate for Holocaust Education. The website includes over 7,000 multimedia files including virtual reality tours, photographs, video clips of survivor testimonies, extensive photo galleries, and thought provoking student-activities. Funding for the project has been derived from state and federal grants. The designers and developers obtained expert validation from several sources, including the United States Holocaust Memorial Museum, the Florida Commissioner's Task Force on Holocaust Education, the Florida Holocaust Museum, and the Holocaust Memorial Resource and Education Center of Central Florida.

Florida legislation mandating Holocaust education was designed to educate students on the history of the Holocaust as a singular event, but it is also designed "for the purpose of encouraging tolerance of diversity in a pluralistic society" (The Holocaust Education Bill, SB 660). To be effective, teachers need to be provided opportunities to develop the knowledge, attitudes, and skills that are required to initiate and facilitate dialogue on culturally sensitive topics such as race and racism (Howard & Denning del Rosario, 2000). Due to the nature of the website content used for this study (i.e., Holocaust Education), it was deemed relevant to conduct a preliminary examination of participants' potential change in not only knowledge but also attitudes.

MEDIA AND ATTITUDE CHANGE

Any message put forth with the intent of affecting attitude change is deemed a persuasive message according to Eagly and Chaiken (1998). Based on previous research in media and attitude change, a series of guidelines have been developed for designing mediated (persuasive) messages that change attitudes (Bednar & Levie, 1993; Simonson 1979, 1980; Simonson, Aergerter, Berry, Klook, & Stone, 1987; Simonson & Maushak, 1996). Two guidelines that are appropriate to the current study are (Simonson & Maushak, 1996):

       Guideline 1: Learners are persuaded, and react favorably, when
       mediated situations include the discovery of useful new
       information about a topic.
       Guideline 5: Learners who experience purposeful emotional
       involvement or arousal during media-rich instructional situations
       are likely to change their attitudes in the direction advocated
       in the situation. (p. 1010)

Previous research has indicated that advance organizers can be used to help contextualize a given message and increase the likelihood that learners will discover new and meaningful information. If successfully designed and implemented, an advance organizer for emotionally charged material such as the Holocaust, could: (a) facilitate the discovery of useful new information, and (b) facilitate purposeful emotional involvement with the material at hand by directing students toward relevant information within the website. As such, this study was designed to assess any impact (direct or indirect) advance organizers may have on the knowledge and attitudes of learners, relative to the website A Teacher's Guide to the Holocaust.

This study employed a 2 (university campus) X 3 (treatment groups) X 2 (time) factorial design with random assignment to investigate the use of two kinds of advance organizers before an activity drawing on the website, A Teacher's Guide to the Holocaust. The outcome measures were participants' knowledge of topics drawn from Guidelines for Teaching about the Holocaust (2001) produced by the United States Holocaust Memorial Museum (USHMM), and participants' attitudes towards traditionally marginalized groups.

METHOD

Participants

Participants were selected from students enrolled in an Introduction to Computers course in Colleges of Education at two public universities in Florida: Campus one was a large urban university and major research institution; campus two was a smaller state university about 220 miles north of campus one. The Introduction to Computers course is a requirement for preservice teachers. The curriculum is similar on both campuses.

The first campus (campus 1) provided 7 sections of the course with approximately 25 students in each section (n=179). The second campus (campus 2) provided 5 sections with around 15 students in each section (n=71). Most of the participants were white, female, Elementary and Secondary Education majors between the ages of 18 and 21.

Instrumentation

Several instruments were designed for this research: (a) a multiple-choice test of knowledge related to the Holocaust, (b) an attitude scale designed to measure bias towards traditionally marginalized groups, and (c) a multicultural affinity scale.

The instruments were validated by the following experts: two instructional technology professors; four instructional technology doctoral students (including the researcher); one research and measurement professor; two research and measurement doctoral students; and the project manager of the website A Teacher's Guide to the Holocaust--a graphics and web design specialist. Each instrument was reviewed, evaluated, and revised as a result of a series of meetings between the researchers and individual members of the group; meetings of the entire group; and examination of field test results. This process took place throughout the design, development, and evaluation phases of the instruments and the website. Each of the measures used in this study was designed with a different structure and purpose; therefore, each measure will be described individually. Knowledge Test

The domains on the Holocaust knowledge test are broad, addressing well-documented topics such as anti-Semitism, the aftermath of WWI, and the Nazi rise to power. A test blueprint was developed though an analysis of the content included in the Teacher's Guide. Design of the website content was informed by suggested topic areas included in the Guidelines for Teaching about the Holocaust (2001) produced by the United States Holocaust Memorial Museum (USHMM). From this content, a number of key ideas were distilled. These ideas were first refined into statements, and subsequently converted into test questions. The knowledge test items were assembled into a single-answer, multiple-choice test format and reviewed by content experts. The review suggested that the topical areas identified in the Guidelines were adequately represented.

The final version of the knowledge test consisted of 44 items, each of which presented four response options. The instrument was first administered in the spring of 2000 with preservice teachers at a large state university (Calandra, Fitzpatrick, & Barron, 2002). Examination of individual item psychometrics, expert panel meetings, and feedback from respondents were used to guide minor revisions to the knowledge test, and it was administered three subsequent times during the following three consecutive semesters. Internal consistency reliability (Cronbach's alpha) for the knowledge test ranged from .77 to .78 for the pretest and was .68 for the posttest during field-testing. Attitude Scales

Two attitude scales based on existing validated instruments were developed, evaluated, and revised. Internal consistency reliability (Cronbach's alpha) for the attitude scales ranged from .80 to .88.

Multicultural affinity. The first scale employed 21 items indicative of attitudes toward traditionally marginalized groups in general (e.g., "Minorities have contributed much to the cultural life of America") and toward specific groups (e.g., "I am usually comfortable around homosexuals"). The specific groups identified on the affinity scale were balanced among the six traditionally marginalized groups. Responses to each item were obtained on a 5-point Likert scale (1 = Strongly Disagree to 5 = Strongly Agree).

Bias toward traditionally marginalized groups. The second scale, developed to measure preservice teachers' bias toward traditionally marginalized groups, employed a semantic differential format. Responses to each pair of 18 bipolar adjectives (e.g., Honest--Dishonest) were obtained on a 5-point scale. As target concepts for the bipolar adjectives, each participant was presented with two of six traditionally marginalized groups (e.g., African Americans, Hispanic Americans, Jewish Americans, Asian Americans, Persons with Mental Handicaps, and Homosexuals). Multiple versions of the instrument were constructed from all pairs of these six groups in a randomized, counterbalanced design. This resulted in 30 versions of the bias instrument.

To estimate respondents' perception of traditionally marginalized groups, an index of bias was calculated. This index was obtained by calculating a deviation score from a neutral response to each item (e.g., deviation from 3.0 on the 5-point semantic differential). As deviations in either direction may be considered bias, the absolute value of the deviation score was used. The deviation scores were averaged across the two sets of 18 bipolar adjectives to obtain a mean absolute deviation from neutrality (ABSDEV) score for each respondent.

THE ADVANCE ORGANIZERS

The advance organizers used in this study were designed based on Mayer's (1979a) characteristics for the construction of an advance organizer and Ausubel's (1978) logical definition for an expository advance organizer. The advance organizers were (a) composed of either a short set of verbal, or verbal and visual information, (b) presented prior to participants' accessing the Teacher's Guide, (c) created using no specific content from the Teacher's Guide, (d) presented chronologically, reflecting the structure of the website, and meant to orient users to the logical sequencing of the material, (e) produced at a higher level of abstraction, generality, and inclusiveness than the website; and, (f) designed to influence the learners' encoding process by presenting key (subsuming) concepts within the website to provide learners an assimilative context.

The investigators created two advance organizers: One advance organizer was text-only, and the other included text and graphics. The text-only advance organizers consisted of key components of the Teacher's Guide to the Holocaust, worded at a higher level of abstraction, generality, and inclusiveness than the actual material in the section. The sentences were developed to help participants generate logical relationships between details presented in the website (e.g., The statement "During the economically devastating years after the First World War, National Socialists (Nazis) grew from a small political group to the most powerful party in Germany," provides participants with a way to connect historical data on the rise of the Nazi party.)

The text+graphic advance organizers reflected the same textual information provided in the text-only treatment together with images representing events described in the abstractions. The text+graphic advance organizers were designed in accordance with three of Mayer's (2001) principles of multimedia learning: (a) Spatial Contiguity Principle--the text and graphics were presented within close proximity of one another; (b) Temporal Contiguity Principle--the text and graphics were presented on the screen at the same time, and (c) Coherence Principle--pictures and words were designed to be meaningful and were presented in an integrated fashion. The text+graphic advance organizers were also field-tested and compared by content and design experts to ensure congruence in meaning. The images were used for their persuasive content and for their ability to reinforce the meaning of the text.

The same panel of experts who worked on the instruments for this study was involved in the design, development, and evaluation of the advance organizers. The advance organizers were field tested with graduate and undergraduate university students (n=36). Finally, they were piloted in the summer of 2001 with a group of 35 preservice teachers. Students were presented the advance organizer and an evaluation sheet. The evaluation sheet posed four questions about the design and effectiveness of the advance organizer. Space was also provided for additional comments. Students generally found the directions to be clear, understood the information being presented, were not confused by the graphics, and found the presentation held their attention. Some suggestions for improvement, however, included: increasing font size; specifying more precisely the method of navigation (e.g., mouse click, next buttons, space bar, etc.); and including captions for the graphics. Some changes were made to the advance organizers accordingly.

The text+graphic organizer was further reviewed by graphical design and Holocaust experts currently working on the Teacher's Guide for proper layout, usability, and suitablity of pictures and text. Appropriate changes were made to the advance organizers as a result of expert panel review, field-testing, and the resulting analyses.

PROCEDURES

At mid semester, participants were administered the pretest instruments (both knowledge and attitudes) to complete during their midterm examination class period. Instructors for each section of the course read a document that advised participants that their surveys would be confidential and their identities would remain anonymous. Participants were asked to work on the instruments individually, and were carefully monitored. They were all given extra credit for satisfactory completion.

Approximately two weeks after the pretests were completed; participants completed an in-lab activity using the website, A Teacher's Guide to the Holocaust. An activity sheet was distributed to participants that consisted of a series of 18 questions dealing with facts and issues surrounding the Holocaust. The same expert panel responsible for the advance organizers and measurement instruments created the activity questions. The activity was also field-tested; carefully examined for validity; and tested for interrater reliability by three different raters. The first five questions were in multiple-choice format, intended to ease participants into the activity. The remaining 11 were more open-ended, constructed response questions. Two treatment groups and the control group all worked on the Holocaust activity sheet. Activity sheets were used to measure participants' ability to navigate the site and retrieve information, and to gauge participation. Those participants who inadequately completed the activity sheet were dropped from the analysis.

First, the number of working computers was counted. This number was divided by three. Small pieces of paper were created for each of the three subgroups. Each group of papers was labeled with either the numbers 1, 2, or 3, representing the two treatment and one control group(s). The numbers were placed in a hat and shuffled. Numbers were randomly drawn from the hat and placed on the computers' keyboards. Computers for the two treatment groups (numbers 1 and 2) had a PowerPoint opened on their desktop that resembled the opening page of the Timeline section of the Web site A Teacher's Guide to the Holocaust. The first slide of the PowerPoint informed students that they could access the web site to find information for their activity only after previewing an either text-only or text+graphic advance organizer. The opening slide of the PowerPoint also informed students that they would be presented information on the Holocaust in chronological order that could help them with their activity. The last slide of the Power-Point provided a direct link to the opening page of the Timeline section of the Web site A Teacher's Guide to the Holocaust.

Computers for the control group (number three) had the opening page of the Timeline section of the web site A Teacher's Guide to the Holocaust opened on their desktop. As participants entered the room, they sat at computers labeled 1, 2, or 3. Proctors (lab instructors) distributed the Activity and informed students that they had the duration of their lab period (up to three hours) to complete the activity and that they should work individually.

To control extraneous variables due to environment, student interaction, computer hardware, and computer connectivity, the questions were answered in a controlled laboratory environment that was carefully monitored by one of the researchers and at least one other instructor. Participants were required to work on their activity sheets individually, and limit their questions to those concerning technical support only.

Two weeks later, the same knowledge and attitude measures were administered to all groups as a posttest as a delayed measure of knowledge retention. The study was executed over a total of four weeks.

DATA ANALYSIS

A [chi square] test of independence was used to compare completers versus non-completers on several demographic variables. No statistically significant differences were evidenced on campus one or campus two. In addition, an independent means t-test was conducted to explore potential differences in Holocaust knowledge and attitude pretest scores. Again, no statistically significant difference was found for either campus between completers and non-completers. Subsequently, 26 participants were dropped from the sample because they did not complete the activity sheet or posttest.

The independent variable Treatment Group had three levels: Control Group, Text-Only Advance Organizers, and Text+Graphic Advance Organizers. The independent variable Time had two levels: Pretest and Posttest. The level of significance was set at .05. Each participant was randomly assigned to a treatment group on his or her campus. The independence of observations assumption was defensible in this study. Samples from both campuses were checked for normality of population distribution by observing both skewness and kurtosis levels, and for homogeneity of variance using Levene's test. No major violations of either assumption were observed. It should be noted that observations with missing values were not included in the analysis.

To eventually merge the data from both campuses, a series of factorial ANOVAs were run. The 2 (campuses) X 3 (treatment groups) X 2 (time) ANOVAs were analyzed for all knowledge and attitude scores as well as demographic variables. Any interaction effect between the variable Campus and any other independent variable would be cause to prevent merging of the data across campuses due to the representative imbalance between the two groups of students. In only one instance was a statistically significant difference found. The factorial ANOVA run for knowledge test scores produced a significant interaction effect between the variables Campus and Treatment Group, F(4,209)=3.07, p=.02. This indicated that the effects of the treatment on change in knowledge test scores were different for the two campuses. Due to the potential confounding effect of this occurrence, data from the two campuses were analyzed separately. It should be noted that no major differences were observed in either participant demographics or treatment conditions between the two campuses. The researchers, one of whom was present for the treatment at both locations, carefully controlled these conditions including student interactions, logistics, amount of time allotted for treatment, participant compensation, computer specifications, and network connection speed.

RESULTS

Knowledge Test

Campus one. ANOVA results demonstrated that change in Holocaust knowledge test scores was not significantly different between advance organizer treatment groups on campus one: F(2,151)=1.33, p=.26. The internal consistency reliability (Cronbach's alpha) for the knowledge pretest at campus 1 was .72, and the knowledge posttest was .78.

Knowledge test scores for participants at this campus generally improved from pretest to posttest, as evidenced by a significant main effect for Time (pretest--posttest): F(1,151)=20.18, p<.0001. Glass and Smith's (1981) approach to finding an effect size was used in this study. The mean differences of the control group were subtracted from those of the two treatment groups and then divided by the standard deviation of the control group differences. The control group's standard deviation was used because it is not affected by the treatment. The knowledge test score effect size for the TAO group at campus one was 0.00. The effect size for the TGAO group was -0.24.

Campus two. ANOVA results showed change in Holocaust knowledge test scores was not significantly different between treatment groups on campus 2: F(2,60)=0.48, p=.62. The internal consistency reliability (Cronbach's alpha) for the knowledge pretest at campus two was .70, and the knowledge posttest was .78. Knowledge test scores from campus two improved from pretest to posttest to a lesser extent than from campus one.

Activity Sheet

Participants' activity sheet scores were also compared between groups in order to observe any immediate effects on reading comprehension that may have resulted from the presence of an advance organizer. A one-way ANOVA was performed on the activity scores: 3 Treatment groups x Activity Scores. The level of significance was set at .05. Differences in activity scores among the groups were not significant. The internal consistency reliability (Cronbach's alpha) for the Holocaust activity sheet at campus one was .52. The internal consistency reliability (Cronbach's alpha) for the Holocaust activity sheet at campus two was .46.

ATTITUDE SCALES

Affinity Scale

Campus one. ANOVA results indicate change in affinity towards diversity was not significantly different between treatment groups on campus 1: F(2,146)=0.74, p=.48. The internal consistency reliability (Cronbach's alpha) for the affinity scale pretest at campus one was .83. The internal consistency reliability (Cronbach's alpha) for the affinity scale posttest was .86. Affinity scale scores from this campus changed very little from pretest to posttest.

Campus two. ANOVA results indicate change in affinity towards diversity was not significantly different between treatment groups on campus 2: F(2,53)=0.12, p=.88. The internal consistency reliability (Cronbach's alpha) for the affinity scale pretest at campus 2 was .88; the affinity scale posttest was .90. Affinity scale scores from the campus 2 sample changed only slightly from pretest to posttest.

Bias Scale

Campus one. ANOVA results show change in perceptions of traditionally marginalized groups was not significantly different between treatment groups on campus 1: F(2,138) =0.62, p=.54. The internal consistency reliability (Cronbach's alpha) for each set of 18 bipolar adjectives on the perceptions scales at campus 1 ranged from .74 to .79.

Campus two. ANOVA results show change in perceptions of traditionally marginalized groups was not significantly different between treatment groups on campus 2: F(2,51) =1.43, p=.24. The internal consistency reliability (Cronbach's alpha) for each set of 18 bipolar adjectives on the perceptions scales at campus 2 ranged from .60 to .73. There was a significant main effect for Time, F(1,52) =31.23, p<.0001. This indicated significant change in ABSDEV (absolute deviation from a neutral response) scores between pretest and posttest on campus 2.

DISCUSSION

Advance Organizer Design

The results of this study showed that the advance organizer groups did not perform significantly better than the control group. Mayer (1979a) constructed a checklist for examining effective organizers that encouraged designers to ask: (a) whether the organizers included some or all of the logical relationships in the to-be learned material (similar to an "expository organizer"), (b) whether the organizer provided the means for relating unfamiliar material to familiar, existing knowledge (similar to a "comparative organizer"), (c) whether the organizer was easy for the learner to use, (d) whether the learner would fail to normally use an assimilative set (of their own) for this material--due to inexperience, for example. He proposed that if the organizers meet any one, of these criteria, they merit further investigation (Mayer).

The advance organizers designed for the current study were constructed to serve an assimilative context (help provide an anchoring knowledge). Those participants who were exposed to advance organizers were encouraged to click through them to facilitate their success on the activity. The advance organizers were also field tested for ease of use and clarity of instructions. Finally, as measured by pretests, the learners did not appear to have a firm grasp on key concepts related to the Holocaust. According to Assimilation Encoding Theory, these three factors should have contributed to the advance organizers' success.

Instruments

It was the purpose of this investigation to measure participants' knowledge of key concepts about the Holocaust using a series of multiple choice questions pulled from those concepts. R. E. Mayer (personal communication, April 20, 2001) has suggested that measures in any future studies within the current context should tap problem-solving transfer, as advance organizers are meant to cultivate meaningful learning. To continue this line of investigation, new cognitive instruments need to be designed to measure problem solving transfer (meaningful learning) as well as measures of retention. That is, the instrument needs to tap assimilation and (encoding) integration of information. It must also be noted that Assimilation Encoding Theory predicts "breadth of transfer" rather than amount of transfer as a learning outcome (Mayer, 1979a, p.375). Perhaps a more delayed posttest could thusly be employed.

Media and Attitudes

Regarding media and attitude change, it was assumed that a greater length of time and/or frequency of interventions would have been required to properly measure mediated change in social attitudes--as most likely was the case at campus one (Sanchez, 1997; Vendley, 1998). Extending the treatment over a semester or longer may further increase the degree of change in attitudes overall. It has also been suggested that the greater the fidelity (realistic effect) of the media with which the intervention is presented, the greater the chance for a desired change in attitudes (Simonson & Maushak, 1996). Again only a small incidental effect, if any, was expected as a result of the advance organizers--or for the entire intervention (advance organizer with or without web activity).

Finally, a significant change in attitudes was noted for all groups at campus two, indicating that the exploration of the Teacher's Guide to the Holocaust may have had a significant effect on participants' attitudes. This result was especially surprising in that very little if any change in participants' attitudes about traditionally marginalized groups was expected to occur as a result of a one-time treatment with a relatively short duration (1-2 hours). A significant change overall in knowledge test scores were also evidenced on campus one, but not on campus two.

It is difficult to determine the cause of these observed differences between campuses. Lab conditions for the study at both campuses were closely monitored by the researcher and found to be quite similar. The time frames within which each campus received their treatments were the same within one week. Demographic data for the two campuses were also similar. The current study did not measure for individual differences such as learner preferences. This factor, unaccounted for in the current study, may have had an effect on the outcomes between campuses.

In addition, multimedia research suggests that learner styles are a significant factor in how they are influenced by multimedia. Although random assignment of participants to treatment groups was used to control for these two factors; it would be interesting to see how multimedia advance organizers can influence different kinds of learners (e.g., "verbalizers" and "visualizers") within the current context.

As suggested by Story (1998), advance organizers have been the subject of a large body of research and command attention of instructional designers and teachers alike. Their effectiveness in the facilitation of learning within the contexts and limitations of this study, however, still needs further research and clarification. The current study was meant to examine a potentially effective approach to help (specifically preservice) teachers with low levels of subject matter knowledge use a large-scale web-based resource. Methods and results from this work are being applied to future endeavors. This data can also be made available in further detail to colleagues across academic disciplines who share similar interests. For more detailed descriptions of the data, or to request copies of the instruments or advance organizers please contact [e-mail].

References

Ausubel, D.P. (1978). In defense of advanced organizers: A reply to the critics. Review of Educational Research, 48(2), 251-57.

Becker, D.A., & Dwyer, D.A. (1994). Using hypermedia to provide learner control. Journal of Educational Multimedia and Hypermedia, 3(2), 155-172.

Bednar, A. & Levie, H. (1993). Attitude-change principles. In M. Fleming & H. Levie (Eds.) Instructional message design, (pp. 283-304). Englewood Cliffs, NJ: Educational Technology Publications.

Calandra, B., Fitzpatrick, J., & Barron, A. E. (2002). A Holocaust website: Effects on preservice teachers' factual knowledge and attitudes toward traditionally marginalized groups. Journal of Technology and Teacher Education, 10(1), 75-93.

Chun, D.M., & Plass, J.L. (1996). Facilitating reading comprehension with multimedia. System, 24(4).

Clark, C. H., & Bean, T. (1982). Improving advance organizer research: Persistent problems and future directions. Reading World, 22, 2-10.

Eagly, A.H., & Chaiken, S. (1998). Attitude structure and function. In D.T. Gilbert, S.T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology (4th ed., Vol. 1, pp.269-322). New York: Mcgraw-Hill.

A Teacher's Guide to the Holocaust (2002). Retrieved 2002, from the Florida Center for Instructional Technology Web site: http://fcit.coedu.usf.edu/holocaust

Glass, G.V., McGraw, B., & Smith, M.L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

Herron, C.A., Hanley, J.E.B., & Cole, S.P. (1995). A comparison study of two advance organizers for introducing beginning foreign language students to video. The Modern Language Journal, 79, 387-95.

Howard, T. C., & Denning del Rosario, C. (2000). Talking race in teacher education: The need for racial dialogue in teacher education programs. Action in Teacher Education, 21(4), 127-137.

Jonassen, D.H. (1989). Hypertext/hypermedia. Englewood Cliffs, NJ: Educational Technology Publications.

Jonassen, D.H., & Grabinger R.S. (1990). Problems and issues in designing hypertext/hypermedia for learning. In Jonassen, D. & Mandl, H. (Eds.). Designing hypermedia for learning. Berlin: Springer Verlag.

Kenny, R.F. (1993). The effectiveness of instructional design orienting activities in computer-based instruction. Paper presented at the meeting of the Association for Educational Communications and Technology, New Orleans, LA. (ERIC Document Reproduction Service No. ED362172).

Kiewra, K.A., Mayer, R.E., & DuBois, N.F. (1997). Effects of advance organizers and repeated presentations on students' learning. The Journal of Experimental Education, 65, 147-59.

Luiten, J., Ames, W., & Ackerson, G. (1980). A meta-analysis of the effects of advance organizers on learning and retention. American Educational Research Journal, 17(2), 211-218.

Mayer, R.E. (1979a). Can advance organizers influence meaningful learning? Review of Educational Research, 4(2), 371-383.

Mayer, R.E. (1979b). Twenty years of research on advance organizers: Assimilation theory is still the best predictor of effects. Instructional Science, 8, 133-167.

Mayer, R.E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1-19.

Mayer, R.E. (2001). Multimedia Learning. New York: Cambridge University Press.

McManus, T.F. (2000). Individualizing instruction in a web-based hypermedia learning environment: Nonlinearity, advance organizers, and self-regulated learners. Journal of Interactive Learning Research, 11 (3), 219-251.

Plass, J.L., Chun, D., Mayer, R.E., & Leutner, D. (1998). Supporting verbal and visual learning preferences in a second language multimedia learning environment. Journal of Educational Psychology, 90(1), 25-36.

Ritchie, D.C., & Gimenez, F. (1995). Effectiveness of graphic organizers in computer-based instruction with dominant Spanish-speaking and dominant English-speaking students. Journal of Research on Computing in Education, 28, 221-33.

Robinson, D.H., Katayama, A.D., & DuBois, N.F. (1998). Interactive effects of graphic organizers and delayed review on concept application. The Journal of Experimental Education, 67(1), 17-31.

Romiszowski, A.J. (1990). The hypertext/hypermedia solution--but what is exactly the problem? In D.H. Jonassen & H. Mandl, (Eds.), Designing hypermedia for learning, (pp. 321-354). Heidelberg, Germany: Springer.

Sanchez, T.R. (1997). The social studies teacher's lament: How powerful is the textbook in dealing with knowledge of ethnic diversity and attitude change? Urban Education, 32(1), 63-80.

Simonson, M. (1979). Designing instruction for attitudinal outcomes. Journal of Instructional Development, 2(3), 15-19.

Simonson, M. (1980). Instructional media, attitude formation, and change: A critical review of the literature. In M. Simonson (Ed.), Proceedings of Selected Research Papers, (pp. 473-526), Denver, CO. (ERIC Document Reproduction Service NO. ED194073)

Simonson, M., Aergerter, R., Berry, T., Kloock, T., & Stone, R. (1987). Four studies dealing with mediated persuasive messages, attitude, and learning styles. Educational Communications and Technology Journal, 35(1), 31-41.

Simonson, M., & Maushak, N. (1996). Instructional technology and attitude change. In D. Jonassen (Ed.), Handbook of research for educational communications and technology, (pp. 984-1016). New York: Macmillan.

Stone, C.L. (1984). A Meta-analysis of advance organizer studies. Journal of Experimental Education, 51(4), 194-199.

Story, C.M. (1998). What instructional designers need to know about advance organizers. International Journal of Instructional Media, 25(3), 253-61.

Tripp, S.D., & Roby, W. (1990). Orientation and disorientation in a hypertext lexicon. Journal of Computer-Based Instruction, 17(4), 120-124.

Teaching resources: Guidelines for teaching about the Holocaust. (2001). Retrieved 2002 from the United States Holocaust Memorial Museum Web site: http://www.ushmm.org/education/foreducators

Vendley, G. (1998). Effect of a multi-ethnic, multicultural program on student participants. NASPA Journal, 35(3), 234-244.

BRENDAN CALANDRA

Georgia State University 

USA

[e-mail]

ANN E. BARRON

University of South Florida 

USA

[e-mail]