E-learning library with local indexing and adaptive navigation support for web-based learning.

Learning with existing web-based resources has become popular and important, for example, in cases where there are diverse learning resources dealing with the same learning topic. However, many resources do not have a clear description of their characteristics, which makes it difficult for learners to select appropriate resources. This article approaches this issue to provide a digital library for web-based learning called e-Learning Library, which includes a learning resource repository, local indexing support, and adaptive navigation support. The aim of the e-Learning Library is to promote learners' learning with the use of diverse resources involving a certain topic. Also reported on are the experimental uses of the local indexing support and adaptive navigation support modules, the results of which indicate that the system is useful.

**********

Over the past several years, an increasing number of hypermedia-based resources on the Web have become available that are designed from an educational point of view. Learning with these existing resources has become more important, particularly as the realization of the potential of lifelong and distance learning (Kashihara & Toyoda, 1998).

On the Web, there are currently a number of learning resources on any topic, each of which is designed by different authors. Some of them are suitable for augmenting domain concepts/knowledge about the topic, and are also suitable for gaining a deeper understanding of the topic, or applying their knowledge. By using these resources properly, learners can learn about a topic from diverse points of view, which is an excellent approach to web-based learning. However, finding learning resources on the Web has become increasingly difficult because most existing resources do not have a clear description of what kind of learners should use the resource, what kind of learning goal can be achieved by using the resource, and so forth (Hasegawa, Kashihara, & Toyoda, 2000).

The main issue addressed here is how to help learners select instructive web-based learning resources to promote the learning of any topic from diverse points of view. The authors' approach is to provide a digital library for web-based learning called e-Learning Library, which includes a learning resource repository, local indexing support, and adaptive navigation support. The learning resource repository is a collection of a great number of learning resources reorganized by indexes called resource indexes, which list the resources' characteristics so that learners can know what and how they can learn from each resource. Local indexing is an approach to building the repository with resource indexes from instructors' points of view, which is different from the indexes supplied by the authorities concerned. Local indexing support enables instructors to index existing learning resources in a simple and practical way. Adaptive navigation support guides learners to the resources to be learned next from the reposi tory in accordance with learners' knowledge states or needs.

In the following sections, thee key ideas towards realization of the e-Learning Likrary are proposed. First the way to reorganize the learning resource repository with the resource indexes is described. Next, a support module of local indexing is explained. Then, the adaptive navigation support with the repository is presented. Experimental uses of the local indexing support and adaptive navigation support modules, the results of which indicate that e-Learning Library is useful is also reported.

E-LEARNING LIBRARY

Learning with Existing Resources on the Web

In this article, a learning resource means hyperdocuments, which describe a learning topic within a web site. The learning resource generally provides learners with hyperspace that consists of a number of web pages and their links. Learners can explore the hyperspace to learn domain concepts/knowledge (Conklin, 1988). For most topics, there are a number of learning resources available, which could facilitate diverse learning phases such as augmenting and applying domain concepts/knowledge.

As shown in Figure 1, web-based learning is viewed as learning a topic in three phases and as the transition among the learning phases. Following Bloom's taxonomy, the learning phases are classified: accretion, understanding, and stabilization (Bloom, 1956). Each phase is also explained as follows:

* Accretion is the phase in which domain concepts/knowledge are augmented;

* Understanding is the phase in which known concepts/knowledge are understood; and

* Stabilization is the phase in which known concepts/knowledge are stabilized.

The transition among these phases is expected to occur according to completion or impasse of learning in a phase, and to take place from the accretion phase to the stabilization phase or in the opposite direction. Learners' knowledge is ultimately expected to stabilize.

Framework for e-Learning Library

In learning a topic, learners would select a learning resource according to the state of their knowledge about the topic. When learners use search engines or collection sites of learning resources to search for resources with a keyword representing the topic, however, they may get a large number of search results, many of which do not have a clear description of their contents that would let learners know which learning phase could be facilitated by learning them. One way to help the learners select resources more appropriate to them is to provide a digital library. There currently exist many digital libraries that collect a large number of multimedia resources on the Web. Researchers setting up these libraries are mostly focused on the tools for indexing, retrieval, and visualization, which are necessary tools for constructing a digital library for web-based learning (Goh & Leggett, 2000; Shneiderman, Feldman, Rose, & Grau, 2000). The authors accordingly propose an e-Learning Library, which consists of three main modules as shown in Figure 2, which are a learning resource repository, local indexing support, and adaptive navigation support. In the following sections, these key modules that will be part of the e-Learning Library will be discussed.

LEARNING RESOURCE REPOSITORY

Resource Indexes and Reorganization 

Before discussing how to reorganize web-based learning resources to build up the learning resource repository, let us first consider resource indexes representing the resources' characteristics. At present, many collection sites gathering URLs of web-based learning resources exist. These sites use resource indexes, which mainly represent learning topics/subjects such as physics, mathematics, and so forth to classify the learning resources. Using these indexes, they can select learning resources from a "what to learn" point of view (Hasegawa et al., 2000). However, such indexes are not enough in selecting resources since the learners would usually think of not only "what to learn" but also "how to learn" especially in which learning phase they can learn.

The authors have consequently provided resource indexes that consist of How To Learn (HTL) indexes in addition to conventional What To Learn (WTL) indexes as shown in Table 1, and propose a way to reorganize learning resources. In helping learners select resources that are appropriate for the transition between learning phases, as shown in Figure 1, the learning phase is considered the most important feature in the HTL indexes. In helping learners continue learning in a phase, secondary HTL indexes are needed to differentiate among resources that could facilitate the phase. In web-based learning resources dealing with the same topic, there exists various media types for representing the contents and various interactive or real-time environments. Such media types and communication channels would influence how one learns in a learning phase, so they are also regarded as HTL indexes.

Figure 3 shows how to reorganize learning resources with WTL and HTL indexes to build up the learning resource repository. First, the resources are classified with WTL indexes so that learners can see from a "what to learn" point of view. Next, they are classified with learning phase indexes so that learners can select from a number of resources covering the same topic. Some resources may have two or three learning phase indexes. Finally, media type and communication channel indexes are attached to each resource as its attributes so that learners can select from several resources that could facilitate the same phase.

Basic and Local Repositories 

An important issue of reorganization is how to apply WTL and HTL indexes to web-based learning resources. The resources should be generally indexed with indexes supplied by the authorities concerned, which public learners can share. However, not all indexes will be meaningful to all learners. Although WTL, media type, and communication channel indexes can be shared, it is quite difficult to share learning phase indexes, since the setting of the learning phase depends on the learner's knowledge level.

Based on the considerations just discussed, a framework for developing the learning resource repository was proposed. First a basic learning resource repository whose resources are classified with WTL, media type, and communication channel indexes was implemented. The learning resources were gathered from registrations by designers of the resources and from some resource collection sites on the Web. In the framework, instructors then take their learners into account to apply the learning phase indexes to the basic repository, building a local learning resource repository.

LOCAL INDEXING

What is Local Indexing?

To apply learning phase indexes, a practical approach is proposed that individual instructors assess the learning phase indexes from their points of view. This approach is called local indexing. In local indexing, each instructor can index learning resources on the assumption that his/her learners use them. Different instructors may accordingly index a resource with different indexes. The local repository with the learning phase indexes helps the learners select more instructive resources according to their learning contexts.

In indexing learning resources, instructors may have difficulty in assessing the learning phase indexes, since the indexing often requires careful reading of the contents of resources and there are a great number of resources for many topics. This article proposes a support for local indexing, which allows instructors to assess learning phase indexes in a simple and consistent way.

Local Indexing Support

How learning resources are composed would generally depend on the learning phase assumed by designers of the resources (Kalyuga, Chandler, & Sweller, 1997). Assuming the accretion phase, in which a learner wants to augment new domain concepts/knowledge, for example, a designer might construct the contents of a resource with diagrams to facilitate the learner's knowledge accretion. This suggests that the structure/function of resources would provide information necessary for assessing the learning phase indexes.

To make the correspondence between the structure/function and the learning phase clear, the focus is on the cognitive load that a learning resource imposes during the learning process. Exploring in hyperspace involves cognitive load, which would enhance learning (Chan, Lin, Lin, & Kuo, 1993; Kashihara, Kinshuk, Oppermann, Rashev, & Simm, 1997). Following Eklund's discussion, the cognitive load was classified as the selection, contents-structuring, knowledge-structuring, or reflection load (Eklund, 1995). These loads can also be subdivided, as shown in Table 2. The designers usually compose learning resources so that specific loads can be provided and specific learning phases can be facilitated. In other words, what kind of loads should be provided depends on the learning phase assumed in the learning resources (Hasegawa, Kashihara, & Toyoda, 2001).

The learning phase indexes can be assessed from the structure/function of resources through the cognitive load imposed. To make this indexing possible, the correspondence among the cognitive load, the structure/function, and the learning phase is made clear. The correspondence between the structure/function and the cognitive load is the one that rearranges diverse references between the structure/function and cognitive load presented in related work (Thuering, Hannemann, & Haake, 1995; Wilson & Jonassen, 1989). Considerations of the structure/function of learning resources are divided into 16 items, which are classified into seven groups according to the cognitive load that each item enables the resource to impose on learners, as shown in Figure 4. These items of structure/function are presented to instructors in the form of a checklist, by which they can assess the learning phase in a simple and consistent way. The correspondence between the cognitive load and the learning phase comes from the cognitive mode l of the learning process proposed by Weinstein & Mayer (1986). Figure 4 also shows the correspondence between each learning phase and the cognitive load to be imposed.

Procedure and Interface

Figure 4 shows an example of local indexing. First, the local indexing support module requires an instructor to read the resource roughly and to mark the checklist. If the resource provides some informative diagrams, for example, the instructor should mark the item of "Informative diagram is provided" in the checklist. In Figure 4, this is indicated by the Mark column. Next, the module calculates the proportion of the marked items to all items that are related to each learning phase, and regards it as the score of the phase. The module then regards those phases whose scores are higher than threshold [alpha] as resource indexes. The threshold [alpha] is heuristically set at 0.6, although the value of [alpha] should be ascertained from experiments. In this example, the accretion phase is regarded as a resource index.

Figure 5 shows the interface of the module that incorporates the local indexing support described above. The module has been implemented with a Common Gateway Interface (CGI). The right window displays a learning resource for which an instructor tries to do the indexing. The left window displays the checklist. Using this module, the instructor reorganizes the resources with learning phase indexes in the local resource repository.

ADAPTIVE NAVIGATION SUPPORT

Learning Resource Navigation

The main aim of this support is to promote learning of a specific topic with diverse learning resources so that learners' knowledge can be stabilized. To accomplish this, the support attempts to facilitate the transition among learning phases and to change media types and communication channels to promote learning in one phase. If a learner reaches an impasse in the understanding phase, for example, he/she is encouraged to return to the accretion phase to resolve it. If he/she completes the understanding phase, on the other hand, he/she is encouraged to move to the next phase, that is, the stabilization phase. He/she is alternatively encouraged to continue learning in the same phase with different resources that have different media types and communication channels.

In the navigation support, two knowledge states are considered: impasses and completion of learning a resource. Learners are asked which knowledge state they reach after learning the resource. If necessary, they can also demand a change of media types and communication channels. The support uses the information given by learners to make a list of resources to be learned next. The resources are put in the order of priority. The list provides them with a guide in selecting the next appropriate learning resource to suit their knowledge states and needs.

Procedure

Next, how to prioritize recommendations of learning resources to be learned next is illustrated.

1. Ordering with knowledge states. When learners reach an impasse in a learning phase, resources that could facilitate the previous phase are recommended first, so that they can resolve the impasse. In contrast, the next phases are not given priority. Resources that have the same media types and communication channels are also recommended, since learners may become confused with a change of media types and communication channels in addition to the change of the phase. Consequently, the learning resource that has the previous phase, the same media types, and communication channels as resource indexes is given the highest priority. In case learners' knowledge state is one of completion of learning, on the other hand, the learning resource that has the next phase, the same media types, and communication channels as resource indexes is recommended with the highest priority.

2. Ordering with learners' needs. Some learners may demand a change of media types and communication channels for the resource to be learned next. Regardless of learners' knowledge states, in this case, the same learning phase, different media types, and communication channels as indexes are given priority. In this case, different learning phases as indexes are not given priority according to learners' knowledge states. In the case where learners' knowledge state is at impasse, for example, the next phases are not given priority. Learning resources that have the same phase, different media types, and communication channels as resource indexes are consequently recommended with the highest priority. However, the way of ordering with knowledge states is preferentially executed if learners reiterate the same phase.

Calculation for Recommendation

Figure 6 shows an example of ordering five learning resources. First, a learner is asked which knowledge state he/she has reached after learning the current resource. If necessary, he/she can also demand a change of media types and communication channels. In this example, the learner inputs impasse as his/her knowledge states in learning a resource, which is in the understanding phase, with text only as HTL indexes. Next, the five resources are ordered with their recommendation scores, which are calculated for every resource. In this case, the accretion phase as an index is given priority so as to resolve his/her impasse and it receives a score of 10 points. The stabilization phase is not given priority and is scored minus 10 points. In addition, the text only as media type indexes is given priority and scored one point, while the other media types and communication channels are not given priority and are scored minus one point for each of them. Therefore, the recommendation scores for the five resources are calculated as shown in the right side of Figure 6. The larger the recommendation score is, the higher the priority of the recommendation. The resource that has the accretion phase and text only as HTL indexes is consequently recommended with the highest priority.

Figure 7 shows the interface of the module for adaptive navigation support. This module, implemented with a CGI, consists of two windows. The left window enables learners to input their needs and knowledge states in learning the current resource. It also shows a history of resources used, which encourages the learners to reflect on their learning processes. The right window displays a list that puts resources to be learned next in order of priority for recommendation.

EXPERIMENTAL USE

Experiment on Local Indexing Support

The main purpose of this experiment was to ascertain the usefulness of the local indexing support. Comparing each learning phase index that the system assessed with the checklist marked by each subject (System-indexing for short) and the index that each subject identified after careful reading of the contents of the resource (Subject-indexing for short), the usefulness of the local indexing support was ascertained. The correspondence of System-indexing to Subject-indexing means that the local indexing module can assess those learning phase indexes that instructors want to apply to learning resources in a consistent way.

Table 3 shows two learning resources used in this experiment. Both are resources on the topic of "Global Warming." The subjects were 10 graduate and undergraduate students in the department of engineering. Because the topic was well known to the subjects, the results of a pretest indicated that they had sufficient domain knowledge and they could be regarded as instructors. The procedure was as follows:

1. Reading one resource shown in Table 3 roughly, the subjects were first required to mark the checklist items with the local indexing module. From this step, three data of System-indexing were gathered, which indicated whether each learning phase was indexed or not.

2. They were next required to read the resource carefully, and to evaluate to what degree each learning phase could be facilitated on a scale of one to five. In case the scale is four or five, the phase was regarded as the resource index. From this step, three data of Subject-indexing were gathered.

3. They were also required to carry out the same procedure for another resource. We accordingly got six data of the correspondence of System-indexing to Subject-indexing from each subject.

Table 4 shows the results that totaled System-indexing and Subject-indexing in each learning phase. The vertical axis indicates whether the system regarded each learning phase of each resource as a resource index or not (System-Index or System-NoIndex for short). The horizontal axis indicates whether the subjects regarded each learning phase of each resource as a resource index or not (Subjects-Index or Subjects-NoIndex for short). The values shown in the table indicate the number of cases that fulfilled both indexing states. Fisher's exact probability test using the data in Table 4 was performed and a significant relevancy was found between System-indexing and Subjects-indexing, which had a positive correlation.

Experiment on Adaptive Navigation Support

The adaptive navigation support was also tested to determine whether or not the way of calculating the recommendation order was useful. The order of priority for recommendations generated with the adaptive navigation support was compared to the order in which subjects placed learning resources to be learned next after reading them carefully. Table 5 shows the learning resources used in the experiment, which are resources about "Global Warming." The subjects were 12 graduate and undergraduate students in the department of engineering. In spite of a well-known topic, the results of pretesting indicated that they did not necessarily have sufficient domain knowledge. The procedure was as follows:

1. The subject was asked to learn the resource A in Table 5 and then to input his/her knowledge state after learning. If he/she wanted to change media types and communication channels, he/she could also input it as his/her need;

2. The subject was asked to read the remaining resources carefully and place them in the best order for his/her knowledge state and need.

Table 6 shows the results of this experiment. The vertical axis is the order in which the system placed the resources (System-decided order for short), and the horizontal axis is the order in which subjects placed them (Subjects-decided order for short). The smaller the number of the order is, the higher the priority for recommendation is. Each value in the table means the number of cases that fulfilled System-decided and Subjects-decided orders.

To further examine an approximate tendency shown in Table 6, the order of priority was divided into High and Low. As shown in Table 7, the High order includes the first and second places of both System-decided and

Subjects-decided orders, and the Low order includes the third and fourth places. Fisher's exact probability test was performed on the data in Table 7 and a significant relevancy between System-decided and Subjects-decided order, which had a positive correlation was found. This indicates that System-decided order agreed with Subjects-decided order approximately.

DISCUSSIONS

Local Indexing Support

Although the results of the experiment on local indexing support were limited to the two resources shown in Table 3, the researchers believe that the local indexing module could assess learning phase indexes, which instructors would apply to learning resources. Such simple and consistent indexing support is necessary for instructors when indexing a large number of web-based learning resources. However, it is also important to improve the accuracy of the indexing. In doing so, the following need to be determined:

* Whether the checklist items as shown in Figure 4 are sufficient;

* Whether the subjects were able to mark the checklist accurately; and

* Whether the threshold of each phase is appropriate.

To address the first problem, additional checklist items need to be inserted. However, it had to be done carefully, because having too many items would reduce the simplicity of the indexing even though it might improve the accuracy of the indexing.

As for the second problem, the subjects could not mark the checklist in most cases of the System-NoIndex/Subjects-Index in Table 4, because they were not able to find some learning support function that the resources had provided. To resolve this problem, it is important to develop an additional support module, for example, which gathers some summary information about the structure/function of learning resources before marking the checklist. Such a support module would enable the instructor to index without viewing all the pages of learning resources in a more simple and accurate way.

As for the third problem, for example, five of seven cases of the System-Index! Subjects-NoIndex were accretion phase cases, as shown in Table 4, which means the threshold of the accretion phase ([+ or -]=0.6) was so small that the module cannot index the resource accurately. When the threshold value regarding only the accretion phase was changed to 0.7, as shown in Table 8, the results were improved to some degree. So, it may be necessary to set the threshold differently depending on each phase's characteristics.

Adaptive Navigation Support

The experiment on the adaptive navigation support showed that it was useful for learners to learn a certain topic with diverse learning resources. However, it does not work well for learners who cannot input their knowledge states and needs by themselves, which is important information for the adaptation. One way to resolve this is that instructors could help such learners input the necessary information. In addition, some learners may input the wrong knowledge state. However, this is not a serious problem from a whole learning process point of view, since inputting completion as a knowledge state despite incompleteness of learning would cause a serious impasse in the next learning phase, for example. Alternatively, inputting impasse despite a learner's completeness of learning as a knowledge state would cause a complete learning in the previous phase without difficulty.

Finally, the adaptive navigation support compared with related work on courseware generation on the Web is discussed (Vassileva, 1997). Courseware is generally generated to facilitate the learning of a series of topics and relationships between these topics. Each topic included in the courseware accordingly needs to be designed as a learning resource from a specific point of view. In related work on courseware generation, the same designer prepares each learning resource for each topic on the Web. However, it is hard to make courseware from existing web-based resources since they are usually designed from different points of view. On the other hand, the focus is on properly using diverse resources with the same topic, not with related topics, to promote learning it from diverse points of view.

CONCLUSION

In this article, an e-Learning Library was described with three main modules, which makes use of diverse learning resources involving a certain topic to promote web-based learning. First the learning resource repository was discussed. It is a collection module of a great number of web-based learning resources, which are reorganized by WTL and HTL indexes. Using these indexes, learners can select learning resources from a "what to learn" and "how to learn" points of view.

Next a form of local indexing support was proposed. It is a construction module of the learning resource repository where each instructor can index existing learning resources from his/her own point of view to build a local resource repository. The local repository with the learning phase indexes helps his/her learners select more instructive resources according to their learning contexts.

In addition, an adaptive navigation support was proposed. It is an extended retrieval module of the learning resource repository, which guides learners to some resources to be learned next in accordance with their knowledge states and needs. This support enables learners to promote learning of a specific topic with diverse learning resources so that learners' knowledge can be stabilized.

Furthermore, experimental uses of the local indexing support and adaptive navigation support modules were reported. From the results of the experiment on local indexing support, the authors believe that the local indexing module can assess learning phase indexes, which instructors would apply to learning resources. They have also made sure the adaptive navigation supports would be useful for learners to learn a certain topic with diverse learning resources.

In the future, it will be necessary to evaluate not only each module but also the whole e-Learning Library in more detail. Especially, we would like to apply e-Learning Library to various web-based learning resources with many instructors and learners. In addition, we should improve the reliability and accuracy of the local indexing support and adaptive navigation support modules. We would also like to develop a more practical system and make it public as a web-based learning portal site.

Table 1

Resource Index

WTL Index    Academic Year,
           Subject, Learning
                 Topic

           Learning Phase
           Media Type
HTL Index
           Communication
           Channel

WTL Index  Academic Year, Subject, Learning Topic

           Accretion, Understanding, Stabilization
           Texts only, Graphics, Animations,
HTL Index  SoundsSimulations, E-Mail, BBS, Chat, Others
           High Immediacy, High Interactivity
           Questions and Answers

Table 2

Kinds of Cognitive Load

                       Cognitive Load


Selection              - Selecting necessary information in a
                         page.
                       - Selecting next pages.
Contents-Structuring   - Structuring information in a page.
                       - Integrating between pages.
Knowledge-Structuring  - Integrating information into related
                         knowledge.
                       - Applying knowledge.
Reflection             - Evaluating learning process.

Table 3

The Learning Resource for Experiment Use

            Resource A

Title       Eco-Life Guide - The
            Issue of Global Warming
URL         http://www.eic.or.jp/
            ecolife/t001.html

Author      National Institute for
            Environmental Studies
Page Count  22 (except for CGI page)
Link Count  43
Phase       Understanding, Stabilization

            Resource B

Title       Kyoto-Earth's Homepage-
            Environment/Global Warming-
URL         http://www.pref.kyoto.jp/intro/
            21cent/kankyo/globe_prob/
            earthwarm/index.html
Author      Kyoto Perfecture

Page Count  14
Link Count  55
Phase       Accretion, Understanding

Table 4

Result of Experiment for Local Indexing Support

                Subjects-NoIndex      Subjects-Index

System-Index            7                   10
System-NoIndex         35                    8
                                  [phi] =0.40 (p=0.0042)

Table 5

The Learning Resource for Experiment

          Resource                         HTL Index

A Think about global warming               Phase : Accretion
  http://www.nature-n.com/                 Media : Graphics
  g_wrm/index-j.htm

B Eco-Life Guide/The Issue                 Phase : Understanding,
  of Global Warming                        Stabilization
  http://www.eic.or.jp/ecolife/t001.html   Media : Graphics, Others
                                           Communication : Q & A

C Kyoto-Earth's Homepage - Global Warming  Phase : Accretion,
  http://www.pref.kyoto.jp/intro/21cent/   Understanding
  kankyo/h3.html                           Media : Graphics, E-Mail

D Global Warming                           Phase : Accretion,
  http://www.wnn.or.jp/wnn-eco/cop3/       Understanding,
  index.html                               Stabilization
                                           Media : Graphics, Others
                                           Communication : Q & A,
                                           High Interactivity

E Tackling to the global                   Phase : Accretion
  environmental problems                   Media : Graphics
  http://www.epcc.pref.osaka.jp/apec/
  jpn/earth/index.htm

Table 6

Results of the Experiment

System-decided
order                 Subjects-decided order

                1            2         3         4
4               2            1         4         5
3               1            3         3         5
2               3            3         4         2
1               6            5         1         0

Table 7

Results of the Experiment

                          Subjects-decided  Subjects-decided
                             high order        low order

System-decidedlow order           7               17
System-decidedhigh order         17                7
                                              [phi] =0.42
                                              (p=0.00867)

Table 8

Change of Threshold Values

                Subjects-NoIndex  Subjects-Index

System-Index            5               10
System-NoIndex         37                8

(Change of Threshold Values [alpha]=0.7 with Accretion Phase)

[phi] =0.46 (p=0.00077)

Figure 4

Example of indexing

      Structure/Function
      Checklist Items                              Learning
Mark  (16items)                 Cognitive Loads      Phase

 v    - Nodes are easy to       Selecting          Accretion
        read.                   necessary         Score 7/10
      - Important information   information
        is emphasized.          in a page.
 v    - Quantity of
        information in each
        node is appropriate.

 v    - The navigation between  Selecting
        the node is easy.       learning page.
 v    - Index is easy to back
        anywhere.
 v    - Site structure is
        simple.

      - Important information   Structuring
        is emphasized.          information
 v    - Quantity of             in a page.
        information in each
        node is appropriate.
 v    - Informative diagram is
        provided.
      - Text and diagram are
        integrated.

 v    - Node layout is stable   Integrating      Understanding
        through all nodes.      between           Score 7/13
 v    - The navigation between  pages.
        the node is easy.
 v    - Index of contents is
        provided.
      - Site Structure is
        appropriate.

 v    - Informative diagram is  Integrating
        provided.               information
 v    - Many comprehensible     into related
        explanations are        knowledge.
        provided with
        examples.
      - Interactie environment
        is provided.
      - Site structure is
        networked.
      - Learning history is
        available.

      - Problem exercises is    Applying         Stabilization
        provided.               knowledge.         Score 2/9
      - Interactive
        environment is
        provided.

      - Problem exercises is    Evaluating
        provided.               learning
      - Learning history is     process.
        available.

Acknowledgements

This research is supported in part by International Communication Foundation, and in part by The Telecommunications Advancement Foundation.

References

Bloom, B.S. (1956). Taxonomy of educational objectives. New York: David McKay.

Chan, T.W., Lin, C.C., Lin, S.J., & Kuo, H.C. (1993). OCTR: A model of learning stages. Proceedings of the Sixth World Conference on Artificial Intelligence in Education '93, (pp. 257-264).

Conklin, J. (1988). Hypertext: An introduction and survey. IEEE Computer 20(9), 17-41.

Eklund, J. (1995). Cognitive models for structuring hypermedia and implications for learning from the world-wide web. Proceedings of the first Australian Worldwideweb Conference [Online]. Available: http://www.scu.edu.au/sponsored/ausweb/ausweb95/papers/hypertext/eklu nd/ index.html 

Goh, D., & Leggett, J. (2000). Patron-augmented digital libraries. Proceedings of the Fifth ACM Conference on Digital Libraries 2000, (pp. 153-163).

Hasegawa, S., Kashihara, A., & Toyoda, J. (2000). An adaptive navigation support with reorganized learning resources for web-based learning. Proceedings of the international conference on computers in education, (pp. 917-925).

Hasegawa, S., Kashihara, A., & Toyoda, J. (2001). A local indexing for web-based learning resources. Proceedings of the World Conference on Educational Multimedia, Hypermedia and Telecommunications, (pp. 687-692).

Kalyuga, S., Chandler, P., & Sweller, J. (1997). Levels of expertise and user-adapted formats of instructional presentations: A cognitive load approach. Proceedings of User Modeling '97, (pp. 261-272).

Kashihara, A., Kinshuk, Oppermann, R., Rashev, R., & Simm, H, (1997). An exploration space control as intelligent assistance in enabling systems. Proceedings of the International Conference on Computers in Education '97, (pp. 114-121).

Kashihara, A., & Toyoda, J. (1998). Report on experiences of telelearning in Japan. Informatik Forum, 12, (1), 39-44.

Shneiderman, B., Feldman, D., Rose, A., & Grau, X.F. (2000). Visualizing digital library search results with categorical and hierarchical axes. Proceedings of the Fifth ACM Conference on Digital Libraries 2000, (pp. 57-66).

Thuering, M., Hannemann, J., & Haake, J.M. (1995). Hypermedia and cognition: Designing for comprehension. Communication of the ACM, 38(8), 57-66.

Vassileva, J. (1997). Dynamic course generation on the WWW. Proceedings of the Eighth World Conference Artifical Intelligence in Education, (pp. 498-505).

Wilson, B.G., & Jonassen, D.H. (1989). Hypertext and instructional design: Some preliminary guidelines. Performance Improvement Quarterly, 2(3), 34-49.

Weinstein, C.E., & Mayer, R. (1986). The teaching of learning strategies. In M.C. Wittrock (Ed.), Handbook of research on teaching (3rd ed.), pp. 315-325. New York: Macmillan.