Learning C with Adam.

Studies have shown that when learning occurs in an environment that uses animated pedagogical agents and personalized instruction, the learner learns the material more deeply and can recall it easier when compared to learning without an agent. Thus, an effective learning system creates personalized contexts for each learner. The "one size fits all" concept is not very effective across a large number of diverse learners. An adaptive instruction approach is more suitable for reaching large diverse groups of learners in electronic learning environments. Adam is an adaptive instruction system that uses animated agents as teachers. It is a web-based system that imparts knowledge using different teaching styles to make understanding concepts easier and personal. An experiment was conducted in which Adam was used to teach C programming concepts in an introductory level C course. The results of this experiment are reported here.

**********

Gilbert & Han (2000) used the term instructional model to refer to the ratio of teachers to learners. In a traditional classroom environment, there is a one-to-many instructional model with one teacher and many learners. In a tutoring environment, the instructional model is one-to-one. While the one-to-one relationship provides a very personal and comfortable learning experience, the knowledge is restricted to one teacher. This is also a restriction of the one-to-many relationship. Arthur was designed to reduce the effect of this restriction (Gilbert & Han, 1999). Arthur is an adaptive instruction system, which provides a many-to-one instructional model in an effort to accommodate an individual learners' learning style. In Arthur, instruction created by multiple teachers for the same lessons/concepts is made available to learners using a single user interface. Delivering a many-to-one instructional model. Instruction is selected by the system so that the best lessons, or teachers, are presented to each individual learner. Adam is an extension of Arthur that implements the many-to-one instructional model using animated pedagogical agents as teachers. This article will discuss the infrastructure, architecture, and results of an experiment conducted using Adam.

CONCEPTUAL DESIGN

Adaptive instruction refers to the varying of pedagogy to accommodate an individual's learning style. For example, if a learner doesn't understand a concept using pedagogical method A, then the instructor should change the pedagogical method to something other than A. In general, each learner has a preferred method of learning, which is their learning style (Dunn & Dunn, 1978). When instruction is presented in such a way that it matches an individual's learning style, it can enhance the learner's learning experience. This is the fundamental motivation for providing adaptive instruction using the many-to-one instruction model as seen in Arthur and Adam.

The conceptual design for Adam begins with a course, which is composed of many lessons. Each lesson is further divided into one or more concepts. These concepts form the smallest units of instruction and are referred to as instructional units. Each instructional unit has two parts, instruction and assessment, Figure 1. The instruction contains objectives and pedagogy. The assessment aims to evaluate each learner with respect to the objectives in the instructional unit. The instruction is delivered by an animated pedagogical agent (Johnson, Rickel, & Lester, 2000).

[FIGURE 1 OMITTED]

Animated pedagogical agents are animated characters with life-like characteristics that facilitate learning in computer based learning environments (Johnson, Shaw, & Ganeshan, 1999). Andre, Rist, and Muller (1998) and Lester, FitzGerald, & Stone (1997) suggest that animated agents increase learner motivation and attention. Their presence can also reduce the learner's perception of the difficulty level of the material (Laurel, 1990). The agents in Adam are implemented using a web browser plug-in. This plug-in was developed by Haptek (2003). The plug-in can perform text to speech in English, German, and Spanish. It also provides voice manipulation and face animation. The face animation controls allow the developer to manipulate the agent's emotions in real time. This nonverbal feedback coupled with the ethnicity of the agents affect the learner's perception of the interface.

Ethnicity plays a critical role in the social interaction between the learner and the agent (Nass, Isbister, & Lee, 2000). Identification with the agent by the learner provides significant and consistent effects on the learner's behaviors and attitudes. When the learner perceives the agent as similar to himself or herself, the agent is perceived as more socially attractive and trustworthy (Nass et al.). The presence of the agent also provides increased arousal, which is "consistent with the best preparation methods for exams in school." (Rickenberg & Reeves, 2000). Learners also exhibit better retention and problem solving transfer with spoken text when compared to onscreen text (Moreno, Mayer, Spires, & Lester, 2001). In Adam, the learner selects an animated pedagogical agent as a teacher. Each agent has a specific personality (different physical appearance, voice, ethnicity, etc.). The agent selected presents instruction to learners from an instruction repository of course concepts. The concepts are created by different teachers using their own pedagogical style as seen in the classroom. In Figure 2, there are three teachers (Lopez, Brown, Smith) and four lessons/concepts. Each row in Figure 2 represents a course module taught by the teacher on the left. All three teachers teach the same course using their own pedagogical style: Explanations first--Auditory, Examples first--Visual and Constructivist. The columns are lessons/concepts that are taught as part of the course. The individual squares are instructional units. It is the instructional unit that is stored in the instruction repository and presented to the learner.

The instructional unit is presented using web-based multimedia. The nature and effects of multimedia presentations on learning have been investigated and researched. Mayer and his colleagues have discovered (Doolittle, 2001):

1. Learners learn better with both words and pictures than just words.

2. Closer spatial congruity of corresponding words and pictures results in better learning.

3. Closer temporal congruity of corresponding words and pictures results in better learning.

[FIGURE 2 OMITTED]

4. Pictures and narration (spoken words) produce better learning results than pictures and online text.

When a learner starts a session with Adam, the multimedia presentation begins. The agent delivers instruction by speaking and manipulating web pages and/or objects. This instruction is personalized for each learner by using a collection of different teaching styles in the form of instructional units, Figure 2. This leads to the physical design and architecture of Adam.

PHYSICAL DESIGN AND ARCHITECTURE

Overview of Adam

Adam uses a many-to-one instructional model to perform adaptive instruction much like the Arthur system. Adam stores instructional content in a database repository without the need of special application system requirements. It is a web-based environment that uses inexpensive web servers and a system capable of running MySQL. Expanding to hundreds of classes simply requires the scaling of server hardware and networking.

Adam Subsystem 

The physical architecture of the Adam environment begins with a web server and a database. The web server is also called the agent server. The agent server will host all of the instructional units (IU). Each IU is maintained as a web-deliverable screenplay. Metadata such as the course unit's name, description, and author's contact information will be stored in the database. This data is used by the delivery systems.

Adam's instructional units are delivered in a couple of formats. The first format uses animated pedagogical agents as teachers (Johnson et. al., 2000). The agents act as teachers during the instructional screenplay. In the instructional screenplay, the agents deliver content by speaking to the learner and turning slides in the content frame, Figure 3. When the learner starts a screenplay, the agent will deliver instruction by speaking and manipulating web pages and/or objects in the content frame, this is to the right of the agent in Figure 3. The agents used in this environment are implemented using the Haptek Player (Haptek, 2003). The order in which the instructional units are presented to the learner is determined by the delivery system.

Delivery System

There are two delivery systems depicted in Figure 4. Delivery systems perform instruction method selection. When a learner's instructional method needs to be adapted, the delivery system performs this operation. Delivery systems select instructional units from the instruction repository on the agent server and displays them in the learner's web browser. As illustrated, there can be several delivery systems using the same instruction repository. One delivery system methodology uses case-based reasoning. In case-based reasoning, the learners' assessments are used as cases. The cases are compared to each other to determine the next instructional method. A second delivery system uses random instruction method selection. In this delivery system, when instruction method selection is required, an instructional unit is selected randomly from the instruction repository. The user interface sits on top of the delivery systems.

[FIGURE 3 OMITTED]

[FIGURE 4 OMITTED]

User Interface

The user interface can be divided into two sections, navigation and content. The initial, or welcome page of the Adam interface, authenticates the user using a user name and password. After the user is authenticated by the system, the animated pedagogical agent appears. Figure 3 provides an illustration of this user interface. On the left hand side of the user interface there is a navigation frame. The navigation frame consists of a control panel and an animated pedagogical agent. The navigation frame is a control panel very similar to that of a remote control. The control panel allows the learner to start and stop a session, submit a question, logout of the session, change the teacher's persona and to review previously seen instructional units. When the learner starts a session, the agent will begin to deliver instruction by speaking and manipulating web pages and/or objects in the content frame. The content frame is where the visible instruction appears, that is, equations, images, illustrations, and so forth.

IMPLEMENTATION

Instruction Creation

The process of instruction creation is extremely tedious and time consuming. Accordingly, improvements to the instruction creation process are required. The output of the instruction creation process is an instructional screenplay that features an animated pedagogical agent as the teacher. The creation process begins with a real teacher. In the current implementation, teachers are video taped in a tutoring environment. The materials and all other manipulatives used during the tutoring session are collected when the session ends. The audio of the tutoring session is transcribed to text and used as input to a Text-to-Speech (TTS) engine. The facial manipulations of the teacher are transferred to the animated pedagogical agent. The manipulatives collected at the end of the session are transcribed into web-deliverable objects for example, HTML pages. Each tutoring session focuses on a specific concept. Additionally, teachers are asked to create an assessment in the form of a quiz for each concept. The concepts and assessments are then combined into an instructional unit. The following four components: audio, facial expressions, manipulatives and assessment are synchronized along a time line to create an instructional screenplay.

Instructional Screenplays

The audio of the screenplay spoken by the agent is created using the Microsoft text to speech engine and the Lernout and Hauspie Text-to-Speech engine. The agents are assigned specific voices selected from these engines. The facial expressions are introduced using the Haptek Player. These components are synchronized together creating the instructional screenplay. This screenplay is then stored in the instruction repository on the agent server.

Instruction Repository

Adam's model for incorporating distributed curriculum is maintained within the instruction repository. Each course is composed of several course modules. A course module is a single teacher/educator's delivery of an entire course or lesson. Each course module is divided further into concepts, as seen on a syllabus. Each concept has an associated web address or URL, which is stored as part of the metadata in the database. This URL is used by the delivery system to deliver concepts to learners. Adam knows where to get the content and retrieves concepts when requested through a delivery system. The delivery systems control the process of adapting instruction.

Adapting Instruction

The delivery systems facilitate the adaptability of the instruction. Adaptive instruction is necessary when a learner requires corrective instruction. Corrective instruction occurs when a learner performs poorly on an evaluation. In a teaching environment, the teacher shares information with the learners and gives them an evaluation to understand the learners' grasp of the material. This evaluation has several forms: short answer questions; multiple-choice questions, True or False questions, and so forth. Evaluation questions relate to the subject matter being taught by the teacher. If the learners perform adequately according to the teacher's standard, which is referred to as a threshold, the teacher assumes that the learners understand the material and moves on to the next logical concept. If the learners do not meet or exceed the threshold, the teacher goes back and tries corrective instruction. The corrective instruction refers to instruction that is provided in an effort to increase the learners' understanding of the material that was explained. Usually the concept is repeated to the learners utilizing a different teaching style or explanation. This method of providing corrective instruction using different instruction styles is adaptive instruction.

Adam provides adaptive instruction through an instruction repository that references several instructional units. Each instructional unit corresponds to a concept. The learner is given instructional units one at a time in the order designated by the teacher. In a traditional teaching environment, the learner is presented with new material at the teacher's pace. Upon the completion of each lesson the learner is given an evaluation in the form of a quiz. This means that although the instruction method of one teacher may be different from another, the learner will be assessed using the same quiz. If the learner performs adequately (e.g., above the threshold established by the teacher), then the learner advances to the next logical lesson/concept. Adam utilizes a similar concept to facilitate learner advancement.

More specifically, Adam presents instruction to the learner using instructional units; however, the learner controls the pace of the instructional presentation (e.g., the learner proceeds at his or her own pace). Upon completion of a concept, a quiz is administered. This quiz is standard for each concept. It is not dependent on the teaching style or course module. Therefore, each learner takes the same quiz for a particular concept. A performance threshold of 80%, which comes from mastery learning (Bloom, 1976), is set in Adam for most concepts. If the learner scores below 80%, then adaptive instruction is necessary. Another instructional unit for the same concept from a different course module is assigned to the learner and the learner is required to take the quiz again. Once the learner successfully meets or exceeds the quiz threshold, the next logical concept is presented to the learner using the same instructional method or course module. In Adam, when corrective instruction is required, a randomly selected teaching style is used to present the concept to the user.

Case-based reasoning was used to facilitate adaptive instruction in the Arthur system but it is not implemented in Adam, (Gilbert & Han, 1999). Adam uses a random instruction method selection algorithm. When adaptive instruction occurs, the instructional units are randomly selected such that the last instructional unit is excluded from the selection. For example, assume a learner takes Concept 1 using Brown's instructional unit. The learner scores 50% on the concept quiz. This will cause the delivery system to perform adaptive instruction. Therefore, the learner will be forced to repeat Concept 1, using Lopez or Smith's instructional unit for Concept 1. The selection of Lopez's or Smith's instructional unit is done at random, excluding Brown's Concept 1. Adaptive instruction occurs throughout a learner's journey of the Adam system. This results in a learner map. The learner map represents the instructional units selected by Adam for the learner. The learner map can be used as input into a delivery system to perform instruction method selection or it can be used to identify similarities between learners. The conceptual and physical design, architecture, and implementation of Adam have been discussed; the experiment design and analysis are next.

EXPERIMENT AND ANALYSIS

Introduction

The goal of conducting the research experiment with Adam was to evaluate the effectiveness and usability of the interface. The usability of the interface was investigated using a questionnaire to determine user satisfaction. Furthermore, this study was conducted to determine the effectiveness of using animated pedagogical agents in the Adam interface. Adam requires each learner to perform at a mastery level (Bloom, 1976; Woolfolk, 1998), which is the threshold required to advance from concept to concept. If learners complete a course using Adam, then those learners have performed at a mastery level on all of the concepts covered in the course. The experiment used Adam, which incorporates animated pedagogical agents, and a no-agent version of the same course information. The details of the experiment are discussed in the sections that follow.

Method

Participants and procedures. Students in an introductory C programming course offered by the Computer Science and Software Engineering Department at Auburn University served as the participants. There were 302 participants: 244 male and 58 female. The students were undergraduate engineering majors from all disciplines. Two versions of the course were provided for the students to use. The first version was Adam, which included 12 different agent personas, and the second version used a simple Web interface without any agents (e.g. the no-agent version). Of the 302 students, 98 used Adam and 171 used the no-agent version. The assignment of course versions to participants was determined by the student's accessibility to a computer that supported the agent installation requirements. The participants were given a week to complete four C programming concepts using Adam or the no-agent version. Each concept was followed by a quiz. In Adam, if the learner was unable to score above the threshold, he or she was randomly reassigned the same concept using a different instructional unit designed by a different teacher. However, with the no-agent version, the same concept designed by the same teacher was reassigned to the learner until the learner was able to clear the threshold of the quiz. The student's interactions with Adam were stored in the database. These interactions included the number of times the student changed the persona of the animated pedagogical agent, the number of times the quiz was taken before the learner was able to successfully clear the threshold for each concept and other pertinent information.

Materials. The course given to the learners consisted of four concepts covered by three different instructional modules designed by three different teachers. Each teacher's module covered all four of the concepts. The concepts involved were as follows:

1. Introduction to arrays.

2. Arrays as parameters to functions.

3. Sorting arrays with the bubble sort algorithm.

4. Searching arrays with the linear search algorithm.

As previously mentioned, each concept had its own threshold set for its respective quiz. These individual thresholds were:

1. Arrays (Introduction) -- 80% threshold;

2. Arrays as function parameters -- 75% threshold;

3. Arrays (Sorting) -- 75% threshold and

4. Arrays (Searching) -- 70% threshold.

The students using the no-agent version could access the system from any desktop with Internet access. The students, who used Adam, had to download the Haptek Player plug-in (Haptek, 2003) and other text-to-speech software.

Experiment Results

Adam allows the learner to select the agent persona of the teacher. Analysis of the agent persona selections determined that the most popular personas were Shannon, Eve, and Monkey as shown by Table 1. Thirty six percent (36%) of the persona changes made by the Adam participants were to Shannon and Eve. It is believed that Shannon and Eve from Figure 5 were favored because of their physical attractiveness. Table 1 also shows that 14% of the persona changes favored Monkey. It is hypothesized that Monkey was favored because of his cartoon-like persona, which is similar to those seen in video games.

[FIGURE 5 OMITTED]

From the post experiment questionnaire, the following discoveries were made:

* Though no instructions were provided before the use of Adam, the learners found the interface predictable and consistent;

* The navigation frame was easy to use as it is comparable to the traditional remote control as shown in Figure 3;

* The aesthetic integrity of the interface was highly rated as a result of the capability to personalize the agent according to the learners' likes coupled with the flexibility to proceed at the learner's own pace;

* 78% of the learners had a fulfilling experience when interacting with Adam. They were interested in taking a similar online course.

When comparing Adam to the no-agent version of the course, Adam out performed the no-agent version with respect to the number of attempts required to succeed at each concept quiz. For example, the "Introduction to Arrays" concept under Adam took learners an average of 4.5 attempts to pass the concept quiz versus an average of 4.9 attempts under the no-agent version. Overall the learners who used Adam required fewer attempts to clear the quiz thresholds. However, the average number of attempts could be improved upon. It is believed that this can be achieved by varying the instruction method selection algorithm from random to some other method, that is, case-based reasoning. It is also believed that the performance could be improved by increasing the number of course modules from three to a higher number. Therefore, a dual approach of increasing the number of course modules and varying the instruction methods will likely decrease the number of repetitions per concept.

A drawback of the system was the lack of detailed quiz performance feedback. The learners found the quizzes difficult and would have appreciated more specific feedback on their performance, especially when they performed poorly. The interface as a whole proved to be an effective learning tool as the learners were able to comprehend the material provided. Further-more, the students rated the agent interface favorable.

Adam received high ratings for agent speech clarity, as 99% of the learners were able to understand the agent's speech. This was a critical finding because the agents used synthetic text-to-speech. However, the students didn't like the synthetic speech, but they were able to understand the agents.

CONCLUSIONS

When instructional messages are personalized, learners recall and learn the material deeper (Moreno & Mayer, 2000). Adam provides personalized instruction to each leaner on two levels. First, each learner is allowed to select his/her own personal teacher as an animated pedagogical agent. This sets up a learning environment selected by the learner. If the learner does not like any of the animated pedagogical agent personas, the learner can simply turn them off. The second level of personalization occurs within the delivery systems when instruction method selection is performed. A personalized instruction context is created for each learner by adapting instruction based upon the learner's performance on assessments. The personalized instruction context aims to accommodate each learner's learning style (Gilbert & Han, 1999). Combining both levels of personalization, Adam attempts to create a personalized context for each individual learner in an effort to enhance learning. Experiment results show that Adam was successful in its goal.

The success of Adam is attributed to adaptive instruction more so than the incorporation of the animated pedagogical agents. Previous studies have shown similar or better gains in learning without the agents (Gilbert & Han, 2002). Also according to Kim and Baylor (2003), agents have no specific effect on learning. However, Adam has an opportunity to make a significant impact in providing universal access to alternative forms of instruction.

Table 1 Persona Name and their Usage (%)

Persona Name  Usage (%)

Shannon          18
Eve              18
Monkey           14
Tiger            12
Jake             12
Ted E. Bear       6
Tom               5
Johnny            5
Jackson           4
Adam              4
Janet             3
Joe               1

Table 2 Average Number of Attempts Required in Both the Versions

                        Avg. No. of Attempts  Avg. No. of Attempts
Concept                 with (Adam)           with (No-Agent)

Introduction to Arrays  4.5                   4.9
Arrays as Parameter     7.0                   8.0
Arrays (Sorting)        2.7                   3.0
Arrays (Searching)      4.3                   6.5

Acknowledgement

This research is supported under the National Science Foundation grant #EIA-296169.

References

Andre, E., Rist, T., & Muller, J. (1998) WebPersona: a lifelike presentation agent for the World-Wide Web. Knowledge Based Systems II. Retrieved April 15, 2005, from http://citeseer.ist.psu.edu/andre97webpersona.html

Bloom, B.S. (1976). Human characteristics and school learning. New York: McGraw Hill.

Doolittle, P. (2001) Multimedia learning: Empirical results and practical applications. Teaching Online in Higher Education. Retrieved April 15, 2005, from http://edpsychserver.ed.vt.edu/workshops/edtech/pdf/multimedia.pdf

Dunn, K., & Dunn, R. (1978). Teaching learners through their individual learning styles: A practical approach. Reston, VA: Reston Publishing.

Gilbert, J.E., & Han, C.Y. (1999). Adapting instruction in search of "a significant difference". Journal of Network and Computing Applications, 22(3), 149-160.

Gilbert, J.E., & Han, C.Y. (2000). Researching adaptive instruction. In P. Brusilovsky, O. Stock, & C. Strapparava (Eds.), Lecture notes in computer science, Vol. 1892, Proceedings of the First International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, (pp. 409-414).

Gilbert, J.E., & Han, C.Y. (2002). Arthur: A personalized instructional system. Journal of Computing in Higher Education, 14(1). Amherst, MA: Norris Publishers.

Haptek. (2003). [Designer of plug-in]. Retrieved April 15, 2005, from http://www.haptek.com/

Johnson, W.L., Rickel, J.W., & Lester, J.C. (2000). Animated pedagogical agents: Face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education 11, 47-78.

Johnson, L., Shaw, E., & Ganeshan, R. (1999) Pedagogical agents on the Web. In Proceedings of the Third International Conference on Autonomous Agents.

Kim, G., & Baylor, A. (2003). The impact of image and voice with pedagogical agents. In Proceedings of E-Learn World Conference on E-Learning in Corporate, Government, Healthcare & Higher Education, Phoenix, Arizona. Norfolk, VA: Association for the Advancement of Computing in Education.

Laurel, B. (1990) The art of human-computer interface design. New York: Addison-Wesley.

Lester, J.C., FitzGerald, P.J., Stone, B.A. (1997, January) The pedagogical design studio: Exploiting artifact-based task models for constructivist learning. Proceedings of the International Conference on Intelligent User Interfaces, Orlando, FL.

Moreno, R., & Mayer, R.E. (2000). Engaging learners in active learning: The case for personalized multimedia messages. Journal of Educational Psychology, 92, 724-733.

Moreno, R., Mayer, R. E., Spires, H., & Lester, J. (2001). The case for social agency in computer-based teaching: Do learners learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19, 177-213.

Nass, C., Isbister, K., & Lee, E.J. (2000) Truth is beauty: Researching embodied conversational agents. Communications of the ACM.

Rickenberg, R., & Reeves, B. (2000). The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. Proceedings of CHI 2000.

Woolfolk, A.E., (1998). Educational psychology (7th ed., p. 334) Boston: Allyn & Bacon.

JUAN GILBERT, DALE-MARIE WILSON, AND PRIYANKA GUPTA

Auburn University, Auburn, AL USA

[e-mail]

[e-mail]

[e-mail]