Framework for Metacognitive Mapping to Design Metadata for Intelligent Hypermedia Presentations.

This article presents a conceptual model and content framework to aid in the linkage of cognitive variables, to the metacognitive attributes of those variables, and related metadata for the design of the presentation media. Instructional designers seeking to provide mediated instruction to target specific learning strategies, in specific target markets, or for specific individuals, may use the framework to help align the design elements comprising the instructional presentation, with the metacognitive learning styles of the target population, with the cognitive variables governing the subject matter presentation. The framework was designed to aid in the addition of automated intelligence to presentations and the variables, attributes and elements were selected and organized for this purpose. The research foundation integrates traditional cognitive science and learning theory with current thinking in these areas from the artificial intelligence perspective.

Instructional design professionals have long excelled in the application of proven media design principles to the education or training task at hand. The basic media design principles, and their underlying instructional strategies, have evolved over time and through many generations of media. Hypermedia and client/server communications have been the latest in a long series of technical innovations, and researchers have successfully developed design principles to implement advanced interactivity and networking into their mediated instruction.

The latest of the technical innovations applies an area of artificial intelligence, which deals with the gathering of user information. This information may range from demographics, to test scores, to user-specific options. The commercial interests, which have pioneered this technology, are using it to develop personalized online shopping experiences, and to improve Internet-based direct marketing campaigns. Instructional designers can use this same technology to personalize and customize learning experiences--including the content presented and the interactions with that content. This would be only the initial stage, and the long term potential of software programs able to intelligently interact with users, and to simultaneously perform content customization with large numbers of users opens new worlds for the next generation of multimedia and hypermedia design.

Instructional design frameworks are needed to help guide the development of the projects. This will include the integration of the theoretical, conceptual, and practical aspects of the subject matter. The next generation of technical innovation will likely add a layer of intelligence to media design, and open possibilities for mapping this intelligence to the metacognitive attributes of users. A framework is needed to guide the theoretical and practical application of this intelligence and to help align it with the learning objectives and learning styles specific to a user or target population. Such a framework is the subject of this paper.

FRAMEWORK BACKGROUND AND THEORETICAL BASIS

This article develops a framework to help hypermedia designers address content personalization and customization. This is accomplished through structures which align the metadata which represents the media design variables with the metacognitive attributes, which determine the receptivity of the learner to specific learning situations and environments. The framework begins with the underlying cognitive variables, which determine the basic category of the learning and intellectual processes to which the media and instruction will be addressed. A higher level of abstraction leads to metacognitive attributes, which address strategies for knowledge acquisition, understanding, and learning. Metadata design elements are drawn from these metacognitive attributes. The framework provides a basis for the addition of intelligence through the linkage of user-specified options to underlying design elements, and subsequently to learning strategies.

A media designer employing the framework would first assess the basic cognitive variable or learning to be addressed. Next, the designer would choose a metacognitive attribute, which best communicates the likely learning strategy for the content, and best matches the understanding processes of the target market or individual. Finally, the selected metadata design elements would address the specifics of the presentation design and provide the basis for the linkages to the metacognitive attributes and cognitive variables. An intelligence gathering mechanism may be employed to monitor user-specific options and then link these to the underlying elements, attributes, and variables. This would result in a knowledge base specific to a user or population.

Media Personalization

The personalization of media and instructional environments has been a persistent goal of educators for many generations. It has also been an elusive goal. There have been difficulties developing the appropriate software, difficulties categorizing the nearly limitless possibilities for media presentation, and a general absence of frameworks specifically designed to help the media designer map the sequences of their presentations with the user's cognitive profile. This absence makes it difficult, if not impossible, to link specific media elements to established instructional principles and desired learning outcomes. In such activities, the classroom teacher has far excelled through his or her ability to recognize individual learning styles and the types of media presentations, which work most effectively with a particular learner.

As with all learning, the instructional experience is more beneficial if aligned with the learning style of the user. Content customization and personalization can create enhanced possibilities for knowledge dissemination and acquisition (Chambreuil, Chambreuil, & Cherkaoui, 1994; Caglayan & Harrison, 1997). A typical application would be an automated media selection based on a comparison between the content of the media presented and the learning profile of a particular user (Maule, 1991; Kearsley, 1993; Balabanovic & Shoham, 1997).

Eventually, an instructional design framework integrating learning and design principles may enable user learning preferences to be captured and applied to dynamically coordinate the creative structure of the content and presentation. The collected information may be applied to help determine the user's learning style, strategy, preference, or need. While customization in hypermedia displays by way of branching and linkages have been a mainstay for many years, a framework, which aligns the user's cognitive processes for knowledge acquisition, with metacognitive strategies for learning and understanding, with the metadata for the design elements of the presentation has been generally absent.

An initial implementation may derive learning patterns from the users. Linkage and branching options are the most apparent measurement data (Botafogo, Rivlin, & Shneiderman 1992; Johnson, 1995). An educational simulation may yield a wealth of data. The output would include a series of learning options based on the experiences and actions of the students. Thus, at the cognitive and metacognitive levels, frameworks aiding learning environment customization and personalization would help make the online experience more enjoyable for users while also deriving knowledge about the user's actions to help identify problems and issues to be resolved (Woolf, 1996).

Media Management for Individual Differences

In a media design context, the term "knowledge management" may be defined as the layer of abstraction above information management, which addresses the correlation between the user and the media. Abstraction is often a key to the design strategy since different media representations will trigger different perceptions in different users. The perceptions lead to internal metaphors and subjective analysis based on the user's experience with that knowledge (Jacobson, 1995). The user thereby derives abstractions through selective interpretation. A means to categorize these interpretations is foremost by the base cognitive learning variable, and secondarily by the corresponding learning strategy or metacognitive attribute. In the context of a metadata design framework, knowledge management would consider filtered user information which might be used to create individualized learning experiences.

In a learning exercise a user typically contends with new information by adding ideas to their existing repertoire of ideas, by distinguishing among the variables in the expanded repertoire, and by subsequently restructuring their personal knowledge and interpretation (Linn. 1996). This abstraction, and the underlying analysis, may thereby guide media design and development. Personal interpretation leads to the corresponding cognitive variables and metacognitive attributes. The success of the media and content presentations, whether graphical or textual, will depend not only on whether the information meets the user's needs, but on the degree to which the representations fulfill the user's information acquisition requirements and are appropriate to the user's task (Petre, 1995; Dillenbourg , 1996).

In the commercial sector, information products are often linked to knowledge variables to help determine the specific interests of individuals (Davenport, 1997). From the user's patterns of interest, over time and in the collective, probable patterns of action may be derived. The user profiles may represent a variety of variables, including demographics, purchases, inquiry processes, and navigational choices. In a learning environment, the designer might use such profiles to ascertain the learning strategy of a user in a specific situation. From this knowledge, the designer might learn about specific human-machine interactions, and the technicalities of interactions between the presentation and the learner (Maule, 1998). New technical capabilities, using AI innovations in areas such as intelligent agents, will aid designers in their efforts to dynamically gather user data (Kearsley, 1993; Gaskin, 1997). From this data the instructional media designer can draw cognitive profiles, derive metacognitive attribut es and strategies, and provide new opportunities for content personalization in software-based educational learning environments.

CORRELATING METADATA ELEMENTS AND METACOGNITIVE ATTRIBUTES

Cognitive maps have long been used in knowledge engineering for expert systems and for automated expert assistance in matters related to managerial communications (Schwartz, 1992; Te'eni, Schwartz & Boland, 1992). In developing the resources, the media/information designers collect and structure the materials to reflect both their understanding of the users, and their anticipations of the user's possible interests, needs, or ability to comprehend the content (Marshall & Shipman, 1995; McGraw, 1994). Cumulatively, and over time, a monitoring of the interactions will provide data for media design. The developers become more knowledgeable about the target population. Increasingly knowledgeable users become more deeply involved in directing information development and presentation.

Over time, cognitive scientists gain an ever deeper understanding of the needs of users, and media designers gain an ever deeper understanding of cognition and the mental variables encountered in their media presentations (Stacy, 1995). A framework to help assess the likely direction of the human-machine exchanges helps structure and supports this process. This may include categorization of the variables and the probable types of metacognitive processes triggered during the interactions. The following model, framework, and discussion provide a research basis for this assessment. Applications of the framework can help in the alignment of metadata design elements with user-specific metacognitive attributes and cognitive learning variables, to advance research in the field of media development for personalized and individualized content presentations.

Cognitive science addresses mental constructs as advanced by the disciplines that study the human mind, including cognitive psychology, epistemology, linguistics, computer science, AI, mathematics, and neuropsychology (Reber, 1985). In cognitive science, comprehension is often characterized as the construction of a mental model that represents objects and semantic relationships (Thuring, Hannemann, & Haake, 1995). Metacognition considers the mindful engagement of the user in a task and addresses the knowledge and control the user has over his cognitive processes (Lopez, 1997). In simple terms, metacognition herein addresses a learning strategy or strategy for understanding.

Metacognition deals with awareness, observation, reflection, and analysis. The cognitive map is often presented as the best available means for structuring cognitive variables and metacognitive attributes for those variables. The map analogy is useful since psychologists believe that users intuitively seek to create a mental analog of a real map as they traverse electronic information. Cognitive theorists even believe that the cognitive map is fundamental to human mental processing. Users form the map by chaining experiences, mentally drawing spatial relationships, and subsequently drawing inferences, preferences, and judgments (Reber, 1985).

Recent studies of Web site design have found that users prefer to see an overview or map of an information resource to help them make judgments about content and navigation (Abels, White, & Hahn, 1997). Some even suggest that humans build and use these mental models to solve problems and understand situations (Stacy & MacMillian, 1995). Mental events and representations are extended into mental constructs of consciousness--such as schemas, ideas, strategies, memories, ideas, beliefs, intentions, and other decision-making criteria. A goal of the media designer would be to assist users in the construction of their mental models--strengthening factors that support the design processes, and weakening those that impede it (Thuring, Hannemann, & Haake, 1995).

Some of the most readily measurable design elements would include the user's responses, activities, reactions, movements, processes, and operations. To this can be added more conceptual or strategic and metacognitive classifications--including patterns of behavior, which might indicate likely intrinsic states and intervening cognitive variables, strategies for dealing with ambiguities, matters of context, recognition-to-recall progressions, recognition skills, relationships in hierarchies of media, structural modeling, and cognitive strategies for exploration (Jacobson, 1995). A framework encapsulating these variables may serve as a guide for educational multimedia software designers seeking to use automated intelligence to produce personalized media.

Metadata Design Elements and Applications

Ideally, a media designer would be able to design programs which can recognize the users' learning processes, establish media selections based on the identified cognitive needs of the user for information, and employ the metacognitive strategies that the learner is likely to need to achieve an understanding. The framework developed herein is designed to provide the basis for the structuring of these processes and may guide media development by helping to synchronize the cognitive maps of the users with the content maps of the services. This line of thinking assumes that the foundation variables and media structures follow some basic constructs.

First, the appropriate cognitive theories may be tested through an analysis of the structure of the software and the user's interactions with that structure (the content presentation). At this point, the user is actively engaged in the program and the design agenda is focused on structural matters, including program sequencing, information linkages, visual designs, and interactive processes. The instructional design strategy would involve a comparison of the cognitive map of the user with the elements influencing the design of the presentation. For example, the measured variables may include linkages, selections, browsing choices, and even the user's navigation strategy. The chosen paths, and thereby the design elements, may be tracked back to the cognitive variables and the metacognitive attributes.

An important facet of the customization and personalization process would involve the service interface and the overall environmental setting of the online experience. It is commonly known that users react differently to different mixes of aesthetic and sensory input, for example, audio, visual, and textual elements. Some users absorb more knowledge through the audio channel, while others through reading or watching. The visual layout and design of the media are an important facet. Industrial applications have often targeted sensory data as a means to facilitate interpretation, planning, and other cognitive resources involved in tasks and actions (Chaib-draa, 1995).

The ability to control media options is important since it may reduce the cognitive "overhead," or the additional and unnecessary effort and conceptualization which limits capacities for human information processing (Thuring, Hannemann, & Haake, 1995). Schemas and representations can all be isolated and tested against the user's perceptions of those events. The manner in which the designer uses this knowledge to interpret user preferences, and to modify program development to better represent the needs of the users, will determine the effectiveness of the media delivery. The result is that the media may become more subjective, entirely functioning through these compiled views of the world (Boden, 1977). Learning experiences may use background knowledge to construct explanations of experiences, which may be compiled into rules to anticipate similar situations (Langley & Simon, 1995).

Hypermedia Metadata for Instructional Design

One of the strengths of hypermedia is that it enables designers to present content in different contexts (Marshall & Shipman, 1995). Environmental variables may be categorized by the user's interactive and emotional preferences. These may be documented through the user's choices and the corresponding metacognitive attributes which are triggered. The various interfaces presented throughout the online experience may thereby offer important measurement data, ranging from user preferences for information access, to the degree and manner of support for user interaction (Maule, 1992; Neilsen, 1993).

Interface designs may also be evaluated by whether they address intrinsic or extrinsic elements. Examples of the former would include cognitive factors addressing the aesthetics of the presentation, including such elements as color, arrangement, warmth, and the overall feel of the environment. Extrinsic elements may consider the linkages, branches, input/output elements, and the degree to which structural variables correlate with the cognitive style of the user. Structural elements may address the nodes, their object properties, and media/object interrelationships (Johnson, 1995).

Through the alignment of media behaviors with the cognitive styles of the users, a media designer may move ever closer to customized experiences. A classification framework will help the media designer better identify the prominent learning variables, and once identified, to more tightly control those variables. The control of the user's explorations through an information space can help structure the presentation to more precisely convey the desired experiences and to support tasks requiring deep understanding and learning (Thuring, Hannemann, & Haake, 1995). This also means new research opportunities for the analysis of media and for the testing of design elements. An appropriately targeted research framework can be a means to delineate relationships between media and structural linkages across an environment.

FRAMEWORK DESIGN AND IMPLEMENTATION

The framework herein builds from cognitive variables, to metacognitive attributes, to metadata design elements. The application of the framework leads to a software design perspective somewhat different than traditional educational software. The traditional approach to educational media has been to examine the mechanisms through which users learn and to then develop media to fit those processes. This approach has led to some excellent examples of well-planned and exquisitely executed media. However, this strategy requires that the designer select an approach or two and then build the media. This may not be entirely effective given the many different techniques through which people learn.

Even if more than one cognitive approach has been advanced through the media designs, the option for the learner to select among cognitive learning approaches, or to have the software select a cognitive profile, which best matches the student's learning style, is extremely difficult to implement and is not generally provided. The framework herein is designed to provide a basis for the cognitive mapping of instruction to the media preferences of individual users and provide a reference for the mapping. The expectation is that future media designers will employ automated intelligence to map an individual's cognitive traits to their strategies for learning and understanding.

The software and instructional design would thereby be addressed through the framework. The framework would be applied in the design of the instructional narrative, instructional simulations, interactive sequences, evaluation exercises, test questions, and the linkages between the concepts and the evaluations. This will accomplish three primary objectives.

* First, the metacognitive references in the framework will enable a better tracking and alignment of learning strategies with the media design elements.

* Second, the application will lead to a new and more robust set of media design principles for developers of instructional media.

* Third, it will build a base of data for dynamic personalization and customization of instructional experiences around the cognitive profiles of individual users.

The end result would be an environment or simulation, which supports dynamic media for variable learning environments. In an instructional setting, the user will be able to explore and learn at his or her own pace and access media and support tailored to his or her individual cognitive and metacognitive needs. Formative and summative testing may establish the legitimacy of the design approaches, cognitive mapping, intelligent guidance, and supporting design elements. Information resulting from this testing may be tracked back to the metadata design elements, metacognitive attributes, and cognitive variables. Each may be assessed and modified in light of the concept-to-test linkages and knowledge acquisition exhibited by the users.

Framework and Model

In addition to direct factual learning and memorization, learning can occur through exploration, manipulation and experimentation. These latter areas are "active learning" in which the user constructs personalized interpretations and individualized knowledge. The framework herein is designed to support both types of learning, but to be more specifically optimized for active learning. To accomplish this, the framework is developed around accepted approaches to human understanding and based in established research in cognition and metacognition.

Cognition is one of the major concerns of the learning sciences. Some consider the learning sciences as addressing educational research from the dual perspectives of human cognition and computing technologies. One of the major applications is in the study of cognition--which is addressed through structures and processes of learning and teaching by which organized knowledge, skills, and understanding are acquired. Conveying this information in an applied framework which researchers can then build upon, and which media designers can reference for their instructional designs, is advanced through the model in Figure 1 and the framework in Figure 2.

Cognitive Variables

The foundation cognitive variables for the framework are drawn from the literature on intelligence. The Stanford-Binet and Wechsler Intelligence Scales provided the basic categorizations. The rationale for this approach is that standardized terminology for cognitive measurement is needed. These two scales are widely accepted and thereby provide a strong foundation for the measurement of general cognition. While this is an abbreviated approach, the parallel is that intelligence scales are based in media, which is generally paper and pencil. Hypermedia programs are similarly based in media, albeit computer media. Both can work equally well from the basic classes of cognition. The cognitive base acts as a foundation upon which testable content can be developed.

The Stanford-Binet Intelligence Scale classification system groups intelligence into four primary categories: reasoning, abstraction, memory, and visio-spatial understanding. Wechsler uses a slightly different scheme, with fewer major classifications and more variables. These variables are: information, comprehension, similarities, vocabulary, arrangement, completion, design, assembly, memory, arithmetic, coding, mazes, and symbols. The intent of the framework herein is to derive design components from these cognitive/metacognitive categories. The Stanford-Binet classifications have been adopted since the categorizations appear to convey a sufficient number of cognitive variables, and the classifications appear sufficient for media design purposes. The subcomponents of Stanford-Binet and Wechsler are very similar and convey a level of detail sufficient to support the base cognitive variables and to provide a structure from which the metacognitive learning strategies may be derived.

There is one exception. Wechsler lists "information" and general subject matter questions as a key criteria in evaluating for intelligence. Stanford-Binet does not provide such a broad measure. The "information" categorization is simply too coarse for a media developer seeking to derive metacognitive attributes for mapping to media elements. At the other extreme, Stanford-Binet's four primary categories are too fine for a first level of media categorization. So, the framework herein provides a compromise solution. At a first level, it uses three cognitive components of "information." The basic cognitive variables are basic knowledge, conceptual knowledge and applied knowledge. Into these three categorizations the Stanford-Binet classification schemes of reasoning, abstraction, memory, and visio-spatial understanding are applied. These serve as the primary variables for the general mapping of the metacognitive attributes.

Metacognitive Attributes

Stanford-Binet uses reasoning as a primary categorization. Reasoning is broken into several areas. Verbal reasoning stresses vocabulary, comprehension and relationships. Wechsler similarly identifies these variables as vocabulary, comprehension and similarities. The framework herein applies the strategic aspect of these variables as basic metacognitive attributes to be addressed through the narrative and supporting media.

The vocabulary aspect of verbal reasoning is found in terminology and in the arrangement of words throughout the narrative. In Stanford-Binet, vocabulary addresses the names of pictures and objects. The assessment requires that the user give the meaning of words. Wechsler expands this to include words of increasing difficulty. In an application for computerized media, one approach would be to highlight important terms and tally the number of occurrences of those terms throughout the presentation. These tallies may then be correlated against user responses in the evaluation sections. Typically, instructional media will build in complexity of concept and narrative throughout the program, thereby addressing the Wechsler criteria of increasing difficulty. The evaluation components would measure the difficulty of terms and concepts, and the number of repeat occurrences of those terms and concepts in the program (prior to evaluation).

The Stanford-Binet comprehension subcomponent of verbal reasoning tests practical and social judgement. Wechsler adds an assessment and critical evaluation emphasis by requiring the user to explain why certain courses of action are appropriate. In an application, comprehension could be correlated through conceptual knowledge or judgement, and reinforced through applied knowledge or a practical application. Two separate categories in the framework provide for a specific focus in conceptual judgement and practical knowledge. The metadata for the design elements and evaluation processes address the basic knowledge, conceptual knowledge, and applied knowledge. Within the conceptual and applied categorizations are the options for critical assessment, explanation, and description. The relationship aspect of Stanford-Binet verbal reasoning addresses similarities and differences. Wechsler defines this as the ability to tell why things are alike or different. The relationship cognitive variable may be addressed withi n the context of similarities and differences between subject matter components.

The Stanford-Binet Intelligence Scale classifies abstract and visual reasoning to include pattern analysis and matrices. Wechsler addresses these areas through assessment questions concerning item arrangement, completion, design, assembly, coding, mazes, and symbols. Pattern analysis refers to the ability to identify, match, or copy like patterns or shapes. This may include the arrangement or placement of pictures or objects in correct order, reproduction of a series of designs, or symbolic representation, and the ability to distinguish between abstract shapes in rows. Matrices are concerned with the manner in which the learner selects an item to complete a set, finds a missing part or component of a picture, or assembles parts into a whole. Derived variables include coding where the user matches symbols with numbers according to a given key and spatial cues wherein the learner is tested for visio-spatial acuteness through maze and puzzle exercises. In an application, the general pattern or puzzle format may be employed through graphic illustrations in the subject presentation and as an evaluation technique in formative and summative assessments.

Short-Term Memory is presented in Stanford-Binet as primarily focusing on object recall. This consists of repeating a series of items, reproducing objects in order, repeating sentences as heard, repeating a series of designs, and identifying like objects and shapes from memory of a previous example. An application may assess for the attribute by building from an instructional strategy designed to discover the learner's ability to comprehend and recall specific and measurable media components. These may range from terms, to phrases and concepts, to illustrations and animated examples. In addition, the evaluation of short-term memory may measure the number of occurrences of tested media throughout the program, including the combinations of occurrences across formats (text, illustration, animation, audio, etc.). Once the recall and patterns have been established, individual differences may lead to further insight on metacognitive strategies for learning and understanding.

Metadata Design Elements

The classification and categorization of the design elements, including the themes and objectives for the framework, were compiled from several sources. The specifications from these sources serve as the data. The design elements are derived from this data. The generic categorizations, or information about the data, are herein addressed as the "metadata" for the media design. The individual styles/strategies for the media design metadata are referred to as "elements." The data was compiled from several recognized sources with a history in subject area specializations related to science. These include national standards organizations, prominent science groups, and testing and accreditation bodies. Some of the sources reviewed in deriving the basic data, and in compiling and interpreting it to produce the metadata design elements, and the underlying metacognitive attributes, include:

* Science for All Americans, Benchmarks for Science Literacy 

* National Science Education Standards 

* National Assessment Governing Board (NAGB)

* National Assessment of Educational Progress (NAEP)

* Eisenhower National Clearinghouse for Mathematics and Science Education

* Third International Mathematics and Science Study (TIMSS)

* American Association for the Advancement of Science (AAAS)

* National Science Teachers Association (NSTA)

* National Center for Improving Science Education

* National Academy of Sciences

* Harcourt Brace Educational Measurement

* Association for the Advancement of Computing in Education (AACE)

* U.S. Department of Education, Office of Educational Research and Improvement

Data from these sources was additionally analyzed for positions on learning objectives, metacognitive strategies, subject level content, subject matter presentation formats, and testing and evaluation criteria. The compilation process led to the basic categorizations represented through the framework. The intent was to derive categories that would be usable by instructional design professionals for purposes of media design, development and assessment. In the framework, these basic/generic classifications and definitions are herein represented under the design "metadata" classification. In an application, the individual elements may be used as a guide and reference for the development of instructional sequences and components.

Concept-Test Linkages and Evaluation Criteria

Assessment and validation are key facets of the framework. The framework assumes a direct link between each concept presented and the evaluation of that concept. The metacognitive attributes and metadata design elements form the foundation for the evaluation components. These are presented within the framework's cognitive classifications of Basic, Conceptual, and Applied knowledge. In the design process, variables within each of these categories would be selected and evaluated--looking first to the cognitive variable, then to one of the metacognitive attributes, then to one or more of the metadata design elements.

Ideally, the concept-evaluation linkage will measure differences between the formative and summative questions. In a typical application, the formative questions might stress basic knowledge, and summative questions seek to ascertain more in the areas of conceptual or applied knowledge. Still, this assumes ample variety in the types of questions asked and the types of answers elicited. The conceptual and applied questions of the summative evaluation will determine the level of retention of the basic knowledge, the degree of internalization of that knowledge, and the student's ability to apply the concepts. Variations in the presentation-assessment patterns may themselves be considered metacognitive attributes. The supporting metadata design elements may thereby be evaluated against the media, instructional design, and research framework.

The evaluation components are thereby tied to the metadata design elements, which are in turn tied to the metacognitive attributes. Both may be assessed in light of the test results. This form of analysis enables a more precise evaluation of instructional media and over a much wider range of instructional design variables. In a practical application, a knowledge component may be analyzed by the user's recall of that component, or the user's ability to apply that knowledge in different situations. Applications may be categorized by the key metacognitive attributes and associated metadata design elements. A direct recall of basic knowledge would consist of a question/answer format in a presentation display closely approximating the original instruction. A complex evaluation criteria would assess compound or derivative abilities through questions derived from a simple extension or combination of the knowledge components.

The assessment of conceptual knowledge may begin with the metacognitive learning strategies for generalization, interpretation, abstraction, and so forth. The metadata design elements of a conceptual representation through sequencing would lead to questions, which assess the student's ability to integrate subject and media components to arrive at a conceptual understanding. A strength of the framework in this process is the level of detail provided in evaluating each of the media components involved in an instructional presentation. The instructional components are linked to the metadata design elements, which are linked to the metacognitive attributes, that in turn are linked to the main cognitive variables.

Specific elements to be evaluated may range from the style of narrative, to graphic/test combinations, to tallies of key word repetitions or concept reinforcements, to implementation styles of multimedia elements, to interactivity and the manner in which concepts are linked, to the impact of various wording styles such as direct or inferred narrative. Evaluation of these elements, and the relation of these elements to the instruction and evaluation results and processes, will help in the generation of user profiles. These profiles will provide insight on the learning strategies of each user. The metacognitive strategies, and their corresponding metadata design elements, would then be available for reference and future research and development. Future research may test the framework by manipulating specific design elements, and thereby the underlying metacognitive attributes, to determine the degree of correlation. The framework might also be applied in specific applications, with specific subject matter or user groups, or in targeted instructional settings.

CONCLUSION

This article provided a framework to link cognitive variables, to the metacognitive attributes of those variables, and to metadata for hypermedia presentation design. Media and instructional designers seeking to provide computer-based instruction targeted to specific learning strategies, in a specific target market or for a specific individual, may use the framework to help align the design elements comprising the instructional presentation, with the metacognitive learning styles of the target population, with the intelligence routines helping the learner through the instruction.

Acknowledegments

This research is based upon work supported by the U.S. Department of Education, Office of Educational Research and Improvement, under an RFP for the development of technologies to support learning, cognitive and social development. Any opinions, findings, conclusions or recommendations expressed in this publication are those of the author and do not necessarily reflect the views or policies of the Department of Education

References

Abels, E., White, M., & Hahn, K. (1997). Identifying user-based criteria for Web pages. Internet Research, 7(4), 252-262.

Balabanovic, M., & Shoham, Y. (1997). Content-based, collaborative recommendation. Communications of the ACM. 40(3), 66-72.

Boden, M. (1977). Artificial intelligence and natural man. New York: Academic.

Botafogo, R., Rivlin, E., & Shneiderman, B. (1992). Structural analysis of hypertexts: Identifying hierarchies and useful metrics. ACM Transactions on Information Systems, 10(2), 142-180.

Caglayan, A., & Harrison, C. (1997). Agent sourcebook. New York: Wiley Computer Publishing.

Chaib-draa, B. (1995). Industrial applications of distributed Al. Communications of the ACM, 38(11), 49-53.

Chambreuil, A., Chambreuil, M., & Cherkaoui, C. (1994). Individualization within a multi-agent computer-assisted learning-to-read environment. Journal of Artificia1 Intelligence in Education, 5(4), 557-590

Davenport, T. (1997, November 15). The knowledge biz, CIO Enterprise, 32-34.

Dillenbourg, P. (1996). Some technical implications of distributed cognition on the design of interactive learning environments. Journal of Artificial Intelligence in Education, 7(2), 161-179.

Gaskin, J. (1997, December 8). New agents help Web merchants. Interactive Week, 5.

Jacobson, F. (1995). From Dewey to Mosaic: Considerations in interface design for children. Internet Research, 5(2), 67-73.

Johnson, S. (1995). Control for hypertext construction. Communications of the ACM, 38(8), 87.

Kearsley, G. (1993). Intelligent agents and instructional systems: Implications of a new paradigm. Journal of Artificial Intelligence in Education, 4(4), 295-304.

Langley, P., & Simon, H. (1995). Applications of machine learning and rule induction. Communications of the ACM, 38(11), 55-64.

Linn, M. (1996). Key to the information highway. Communications of the ACM, 39(4), 34-35.

Lopez, A. (1997). Analogy, logic programming, and metacognition. Journal of Artificial Intelligence in Education, 8(1), 77-88.

Marshall, C., & Shipman, F. (1995). Spatial hypertext: Designing for change. Communications of the ACM, 38(8), 88-97.

Maule, R. (1991). A review of hypermedia research. Journal of Hypermedia and Multimedia Studies, 1(2), 15-17.

Maule, R. (1992). Online multimedia for education. Journal of Educational Multimedia and Hypermedia, 1(2), 169-177.

Maule, R. (1998). Cognitive maps, Al agents and personalized virtual environments in Internet learning experiences. Internet Research, 8(4), 347-358.

McGraw, K. (1994). Performance support systems: Integrating AI, hypermedia, and CBT to enhance user performance. Journal of Artificial Intelligence in Education, 5(1), 3-26.

Neisen, J. (1993). Usability Engineering. Boston: Academic Press.

Petre, M. (1995). Why looking isn't always seeing: Readership skills and graphical programming. Communications of the ACM, 38(8), 33-44.

Reber, A. (1985). Dictionary of psychology. London: Penguin.

Schwartz, D.G. (1992). Cognitive mapping as a knowledge engineering tool. In G. Biswas, (Ed.), Applications of artificial intelligence X: Knowledge-based systems, SPIE Vol. 1707.

Stacy, W. (1995). Cognition and software development. Communications of the ACM, 38(6), 31.

Stacy, W., & MacMillan, J. (1995). Cognition bias in software engineering. Communications of the ACM, 38(6), 57-63.

Te'eni, D., Schwartz, D.G., & Boland, R.J. (1992). Cognitive maps for communication: A specification of functionality and usability. 4th Symposium of Human Factors in information Systems.

Thuring, M., Hannemann, J., & Haake, J. (1995). Hypermedia and cognition: Designing for comprehension. Communications of the ACM, 38(8), 57-66.

                      Subject
                      Comprehension
Cognitive Variables   Basic Knowledge
Metacognitive         - Strategies to
Attributes            understand directly-
                      stated details or
                      relationships.
                      - Individual strategies for
                      knowledge, discovery,
                      retention, and recall.
Design                - Placement of testable
Metadata Elements     content in presentation,
                      including examples and
                      formative questions.
                      - Presentation variables
                      including item
                      repetitions, media mix,
                      illustrations, examples
                      and linkages.
Concept-Test Linkage  Learning of textual facts.
                      Role information,
                      questions direct from
                      content and examples.
Evaluation            Direct: Questions
Criteria              stressing rote
                      memorization of factual
                      material direct from text.
                      Multifaceted: Questions
                      requiring simple
                      connections of 2 or more
                      factual statements to
                      arrive at a conclusion,
                      Mediated: Narrative
                      supplemented with
                      media variables that
                      convey a required and
                      supplemental facet of
                      the required information,
                      Subject
                      Comprehension
Cognitive Variables   Conceptual Knowledge
Metacognitive         - Strategies for abstract
Attributes            learning, generalization,
                      - Inference and
                      interpretation strategies
                      to comprehend implicit
                      relationships.
                      - Comprehension
                      beyond directly stated
                      relationships;
                      connections beyond
                      presented information.
Design                - Sequencing of media
Metadata Elements     elements to convey a
                      desired idea or concept.
                      - Narrative presentation
                      with conceptual media
                      representation,
                      - Introduction of
                      additional sensory data,
                      inferences,
Concept-Test Linkage  Relaled information
                      questions requiring
                      simple abstraction or
                      generalization from
                      primary subject
                      presentation.
Evaluation            Integrated: Questions
Criteria              requiring compilation or
                      integration of subject
                      and media components,
                      Inferential: Questions
                      requiring extrapolation of
                      conceptual themes from
                      primary subject matter,
                      or conceptual inference
                      from one or more media
                      components.
                      Predictive: Predict
                      events based on
                      conceptual information,
                      Critical: Analyze and
                      evaluate explicit and
                      implicit information and
                      relationships.
                      Subject
                      Comprehension
Cognitive Variables   Applied Knowledge
Metacognitive         - Strategies for under-
Attributes            standing the subject's
                      purpose and the direct
                      application of the learned
                      concept.
                      - Inferential ability to
                      extrapolate meaningful
                      context and apply in
                      similar and dissimilar
                      applications.
                      - Strategies to explore
                      knowledge through real-
                      world connections or real-
                      life experiences.
Design                Subject synchronization
Metadata Elements     from content to examples
                      for:
                      - Direct application of
                      learned principles in
                      similar scenarios.
                      - Indirect application of
                      learned principles in
                      dissimilar scenarios
                      requiring simple deriva-
                      tion, in text or evaluation,
                      narrative or media.
Concept-Test Linkage  Understand and explain
                      why certain subject
                      variables are appropriate,
                      certain courses of action
                      are appropriate, or certain
                      real-world situations exist.
Evaluation            Direct Applied: Practical
Criteria              application of learned
                      content and concept in
                      scenario similar to
                      presentation example.
                      Indirect Applied:
                      Application in dissimilar
                      but related scenario.
                      Applied Conceptual:
                      Content applied with sup-
                      port from media,
                      examples.
                      Predictive Applied: Predict
                      events based on applied
                      Information, Interpreted
                      information, and personal
                      conclusions.
                      Critical Applied: Ability to
                      develop, determine, or
                      describe strategies for
                      assessment of subject
                      content in given situations.