Eye-tracking adaptable e-learning and content authoring support.

In this paper we describe our ongoing research project called AdeLE, a framework for adaptive e-learning utilising both eye tracking and content tracking technology. Possible areas of application are described, such as using the information about the position of the eye for providing additional context specific information to the learner. We report more in detail about current research challenges where we observe users' learning behaviour in real time by monitoring characteristics such as objects and areas of interest, time spent on objects, frequency of visits, and sequences in which content is consumed This research is focused on analysing eye-movement patterns during learning and linking these patterns with cognitive processes. The concept of the appropriate authoring tool is outlined as one of the challenges for the future work.

Povzetek: V clanku je opisan raziskovalni projekt AdeLE, katerega cilj je postaviti okvirno recitev za adaptivno ucno okolje, ki temelji tako na tehnologiji sledenja pogledu in spremljanja ucnih vsebin.

Keywords: adaptive e-learning, real time eye tracking, learning research, cognition, authoring

1 Introduction

Past research and projects at FH JOANNEUM, Department of Information Design, have been among others in the area of adaptive hypermedia systems [SCALEX] and application of eye tracking for web usability evaluation in the Web Usability Center. Gained competences and experiences from the previous research in the area of adaptivity and personalisation, as well as user centered applications, created the motivation to merge adaptive hypermedia systems and eye-tracking technology with the goal of making learning more efficient and effective. The AdeLE prototype offers a solution framework in the direction of eye tracking based real-time adaptive hypermedia systems.

What is AdeLE? The presented research of the AdeLE (Adaptive e-Learning with Eye tracking) project is focused on a new generation of adaptable knowledge transfer in e-learning environments [AdeLE]. This new and innovative approach strives to capture dynamically user behaviour based on a real-time eye-tracking system (see also [Pivec et. al 2004], [Pivec et al. 2005a]). We apply eye-tracking for a more profound learning research and improvement of cognitive processes understanding to be able to support adaptive teaching and learning in a technology-based e-learning environment in the future.

In our research we are concentrating on how information from eye-movements could be used to support the learning process. In most e-learning environments information is mainly provided by means of written text. Thus, reading this information is essential for learning. By means of real-time tracking of the user behaviour, unseen sections of content units provided to the learner are identified by the system and the system might intervene in an appropriate way. Of course, this method can not reflect information about pre-knowledge. Based on the information about content sections skipped by the learner, adaptable and context specific assessment tests can be compiled to check the learner's knowledge about these particular concepts.

The aim of this paper is to present the research endeavours of the AdeLE project. The paper is based on two conference publications: (i) Look Into My Eyes And I Will Tell You How To Learn, by [Pripfl et al. 2005], and (ii) Adaptable Features and needed Content Authoring Support, by [Pivec et al. 2005b]. The paper gives inside in the eye-tracking technology and outlines application scenarios of adaptable e-learning by means of eye tracking, as well as ongoing research and further developments in the field of adaptive authoring.

2 Adaptable Learning Based on Eye-Tracking

Real-time Eye-tracking (User Tracking)

Defining a reliable set of parameters is one of the emerging research issues in the AdeLE project. Eye movements, scanning patterns and pupil diameter are indicators of thought and mental processing involved during visual information extraction [Rayner 1998], [Kahneman and Beatty 1966]. Thus, real-time information of the precise position of gaze and of pupil diameter could be used for supporting and guiding learners through their learning journey.

How does eye-tracking work? Very roughly, eye movements can be divided into two components: fixations, i.e. periods of time with relatively stable eye movements, where visual information is processed, and saccades, which are defined as rapid eye movements that bring a new part of the visual scene into focus. However, more important indicators can be gained by analysing both components together with other derived parameters. Gaze duration (i.e. time spent on an object) and fixations are not indicative of attention per se, because one can also pay attention to objects, which do not lie in the centre of the focused region. Nevertheless, by considering other indicators, such as saccadic velocity, blink velocity and rate as well as eyelid's degree of openness, a better and more meaningful approximation can be gained. Saccadic velocity, for example, is said to decrease with increasing tiredness and to increase with increasing task difficulty [Fritz et al. 1992]. Further, blink rate, decreasing blink velocity and decreasing degree of openness may be indicators for increasing tiredness [Galley 2001]. Thus, if tiredness is identified, it should be possible through adaptive e-learning mechanisms to suggest optimised strategies such as the best time to take a break.

At the present there basically exist two types of eye-tracking systems on the market: remote systems and head-mounted systems. Remote systems are characterised by the fact that one or more cameras record the eye of the participant and trace the gaze in a scene through imaging algorithms. The cameras are positioned in front of the participant. One of the advantages of these systems is given by the fact that the camera can be integrated into the monitor, and therefore remains basically invisible (i.e. a relatively non-intrusive monitoring is possible). Head-mounted systems are characterised by a special device that the participant has to wear on the head like a helmet. More characteristics of both systems along with advantages and disadvantages related to the requirements of the AdeLE project are outlined in detail in [Pivec et al. 2004].

The AdeLE project team decided to utilize the remote eye-tracking system "Tobii 1750" with the infrared diode lamps and camera are integrated into a 17" TFT monitor. This system is easy to operate and all tracking processes run automatically. Thus, it can be used for all forms of eye-tracking studies with stimuli that can be presented on a monitor, such as Web sites, slide shows, videos and text documents. With real-time eye tracking user gaze data are gathered. The evaluation of the users' eye gaze data gives information about what the user is doing, e.g. learning or reading, looking at the pictures and illustrations or eventually struggling with the system's navigation.

Eye Movement Parameters and Their Influence on Adaptation

In our research we are concentrating on how information from eye-movements could be used to support the learning process. The study focuses on finding eye-movement patterns which distinguish between skimming through, reading of and learning facts from written text. Based on various research findings different models of eye-movement control during reading have been developed. Their application is in general not feasible for the contents of a real life e-learning environment. Guided by the research literature but also considering the practical usage of our system, we designed an eye-tracking study in which subjects have to deal with texts of three difficulty levels under four different conditions (1. skimming through text, 2. single reading of text, 3. learning the content of the text and 4. searching for a specific information within the text). The study was carried out with 40 test persons. The gathered data of this study are applied for the definition of eye-movement parameters which can reliably distinguish between the four conditions listed above. In a further step these parameters will be taken to identify user behaviour within an e-learning environment in real-time (e.g. if the user really learns a text or just reads it).

By merging eye-tracking technology with proper content presentation the goal of the research is to identify, evaluate and develop methods of adaptive instruction for personalised e-learning. From real-time eye tracking data six different user behaviour parameters are reported to the AdeLE prototype: (i) learning, (ii) reading, (iii) skimming through text, (iv) searching in text, (v) observing a picture or reading a text and (vi) looking on the navigational elements. User parameters can be in the range between 0 and 1 expressing the probability of a certain user behaviour. Reported user parameters trigger further various reactions of the system in terms of adaptation of the content and additional information offered to the user. Application of SCORM run time environment enables dynamic content sequencing, which calculates the next steps and provides personalised content for the individual user (see Garcia-Barrios et al. 2004 for detailed description of the AdeLE framework architecture). For example in an e-learning course about the handling of dangerous chemicals our system will react as follows: The user is faced with a text he should learn and know for understanding the following chapters--the system detects that the user is just skimming through this text--the system consults the user if he just wants to get quick information about the lecture content or structure, or if he already knows the details of the content--according the answers the system interacts in one of the following ways: it suggests to show just the abstracts of each chapter or to go on without any changes; it provides some content specific questions or suggests to repeat the lecture.

Real time user observation can be applied also for the enhancement of a user profile that has influence on adequately personalised course content presentation for each individual learner. For example, the prototype can distinguish between (a) learning style of the user (e.g. text based, acoustic) and (b) cognitive style i.e. holist or analyst, meaning that to the analyst the entire content is presented consequently in contrast to the presentation of the content to the holist, where an overview of chapters and subchapters along with summaries is optionally offered.

Possible Application Scenarios

Currently, the research efforts of the AdeLE team concentrate on three issues, which are discussed in the following sections. The first issue is to develop methods to extract individual learning strategies from the learner's gaze behaviour and adapt against the identified learning style. Comprehensive reviews of cognitive psychology research indicate that people exhibit significant individual differences in how they learn [Schmeck 1988], [Glaser 1984], [Honey 1986]. A simple example being individuals who have a strong visual memory but weaker verbal processing will find text based material harder to process than individuals who have stronger verbal skills. In the traditional classroom environment a teacher has the chance to adapt or explain material to suit individual's needs. In e-learning environments where a teacher is frequently not present, pedagogical material is nowadays more uniformly presented. In an e-learning environment information about the learner's gaze behaviour would be a great opportunity to optimise material to an individual's needs. For example, if somebody prefers text and ignores pictures the amount of pictures presented could be reduced, and vice versa.

The second issue addresses the usage of information about the specific content accessed by the user (specific words, paragraphs, areas of pictures, tables, and the like) to provide additional context specific information. For example, an animated picture could accompany textual information, whereas the integration of the picture proceeds in relation to the words or paragraphs accessed by the user, as illustrated in the following example. In an e-learning course concerned with Alexander the Great's Conquest of Persia, a map of Alexander's advance in the region is shown parallel to the text. The map content is updated in correspondence to the paragraph currently read by the learner. When the second paragraph about Granikos is being read, the map shows in animated form the journey of Alexander from Macedonia to Granikos. When the reader has advanced to the fourth and fifth paragraph about Alexander's journey to Gangamela, the map is automatically updated with a corresponding illustration containing the passage from Issos to Gaugamela. Further research questions related to this topic are "Does such an eye-triggered animation really help a student to learn?" or "How should such an animation be integrated into the text to support cognitive processes?" among others.

The third research challenge is based on developing and testing appropriate intervention strategies when the learner is found to have problems. The e-learning environment might react in an appropriate way when a learner is not focused on a relevant part of the computer screen, or is focused completely outside the task area for a certain period of time, or the eye gaze is sufficiently quick for a given period of time. Just to give one example, in case of knowledge acquisition problems for a particular content section more detailed content or background information can be provided to the learner.

3 Adaptable Content Authoring Tool Concept

To provide content structured to support adaptable and personalised content presentation, adequate authoring tools for content authors have to be provided. The basic requirements of such a tool are as follows: to make the authoring a structured process; to keep this process as simple as possible and not to require from the authors to think explicitly about adaptivity and various content presentations.

Within the further development of the AdeLE framework a template based authoring tool that supports stepwise authoring and separation of presentation layers and content chunks will be provided. The structure of the authoring tool is depicted on Figure 1. Based on Templates the author is guided to provide different chunks of content appropriate for different knowledge levels of the learner (e.g. novice, advancer, expert: note that the levels can be broken down in various manners). Templates enable stepwise and structured authoring. The information and knowledge chunks are then semi-automatically tagged with meta-data and saved in the Content Repository. The authoring tool has to support the re-use of the content i.e. the application of the content chunks from various repositories. The function of the Adoption Engine is to dynamically create personalised content from the available information chunks and different presentation layers. The presentation layers can be of different kinds e.g. role specific (learner, teacher, author, etc.), goal or context specific (introduction to the topic, basic definitions, application scenarios, etc.), respective to the output device. The creation of a user specific presentation is also influenced by the user model and the user state.

[FIGURE 1 OMITTED]

4 Conclusions

The AdeLE framework with the assets of extreme adaptation and personalisation to each individual user on various levels (e.g. macro level in terms of general adaptations of the course and micro level, where each page can be different, considering also the pace and momentary user performance) is the first step to innovative human centered technology enhanced learning and knowledge management solutions.

The ultimate goal of our approach is to interpret various users' parameters in form of input data for an adaptable e-learning system that assists users to improve their learning behaviour thus achieving better learning results. In the context of user behaviour interpretation, it is very important not to rely exclusively on eye tracking data, but to supplement it also with constant user feedback. It is possible to suggest optimised strategies such as the best time to take a break, the best time for repeating specific learning content considering the forgetting curve [Davis and Palladino 2002] or suggesting better sequencing of the learning objects. However, the user will always retain the final decision on whether to accept or reject the system's suggestions.

The AdeLE framework can be integrated into different applications e.g. content management systems, e-learning environments, knowledge management systems etc., thus providing new highly user sensitive personalised adaptive solutions. The proposed authoring tool will support appropriate adaptive content authoring.

Evidently, the price of an advanced eye-tracking system plays a decisive role in the application possibilities of the AdeLE solution approach. Nevertheless, existing systems show that the eye-tracking device can be integrated into a standard monitor. Due to the continuing trend of rapid technical progress, we expect that in the next few years it will be possible to build a low-cost but high-quality eye-tracking system based on standard hardware components, which will be suitable for real-time analysis of eye-tracking information as described in this paper. This will make it possible to provide applications related to attentive workplaces for broad populations.

Received: May 28, 2005

Acknowledgements

The AdeLE project is partially funded by the Austrian ministries BMVIT and BMBWK, through the FHplus impulse programme. The support of the following institutions and individuals is gratefully acknowledged: Department of Information Design, Graz University of Applied Sciences (FH JOANNEUM); Institute for Information Systems and Computer Media (IICM), Faculty of Computer Science at Graz University of Technology; especially Karl Stocker and Hermann Maurer.

References

[AdeLE] AdeLE "Adaptive e-Learning with Eye Tracking" project page http://adele.fh-joanneum.at, accessed on 26.01.2005

[Davis and Palladino 2002] Davis, S. F. & Palladino, J. J.: Psychology (3rd ed.), Prentice-Hall: Upper Saddle River, NJ, 2002.

[Fritz et al. 1992] Fritz, A., Galley, N., Groetzner, Ch.: Zum Zusammenhang von Leistung, Aktivierung und Motivation bei Kindern mit unterschiedlichen Hirnfunktionsstorungen, Zeitschrift fur Neuropsychologie, 1(1), pp. 79-92, 1992.

[Galley 2001] Galley, N.: "Physiologische Grundlagen, Messmethoden und Indikatorfunktion der okulomotorischen Aktivitat"; In Frank Rosler (ed.): Enzyklopadie der Psychologie, 4, Grundlagen und Methoden der Psychophysiologie, 2001, 237-315.

[Glaser 1984] Glaser, R.: Education and Thinking: The Role of Knowledge, American Psychologist, 39, pp. 93-104, 1984.

[Honey 1986] Honey, P.: The Manual of Learning Styles, Peter Honey: Maidenhead, Berks, 1986.

[Kahneman and Beatty 1966] Kahneman, D. & Beatty, J.: Pupil diameter and load on memory, Science, 154, pp. 1583-1585, 1966.

[Pivec et al. 2004] Pivec M., Preis M.A., Garcia V., Guetl C., Mueller H., Trummer C., Moedritscher F.: Adaptive Knowledge Transfer in E-Learning Settings on the Basis of Eye Tracking and Dynamic Background Library, in Proceedings of EDEN 2004 Annual Conference, Budapest, Hungary.

[Pivec et al 2005a] Pivec, M., Pripfl, J., Gutl, C., Garcia-Barrios, V.M., Modritscher, F. & Trummer, C.: AdeLE First Prototype: Experiences Made. In Proceedings of I-KNOW'05, 2005.

[Pivec et al. 2005b]] Pivec, M., Pripfl, J., & Trummer, C.: Adaptable Features and needed Content Authoring Support. In Proceedings of the World Conference on E-Learning in Corporate, Government, Healthcare, & Higher Education (E-Learn 2005), October 24-28, Vancouver, Canada.

[Pripfl et al. 2005] Pripfl, J., Pivec, M., Trummer, C. & Umgeher, M.: Look into my eyes and I will tell you how to learn. Proceedings of the European Distance and E-Learning Network 2005 Annual Conference, June 21-23, Helsinki, Finland.

[Rayner 1998] Rayner, K.: Eye movements in reading and information processing: 20 years of research, Psychol Bull, 124, pp. 372-422, 1998.

[SCALEX] SCALEX home page http://www.scalex.info/, accessed on 20. 08. 2005

[Schmeck 1988] Schmeck, R.R.: Learning Strategies and Learning Styles, New York, Plenum Press, 1988.

Maja Pivec, Christian Trummer and Juergen Pripfl Department of Information Design, Alte Poststrasse 152, A-8020 Graz, Austria, Europe University of Applied Sciences FH JOANNEUM, Graz, Austria