Are language learning websites special? Towards a research agenda for discipline-specific usability.

With the intention of defining an initial research agenda for discipline-specific factors in the usability of e-learning websites, this article focuses on the example of foreign language learning. First, general notions and concepts of usability are analyzed, and the term "pedagogical usability" is proposed as a means of focusing on the close relationship between usability and pedagogical design. Then, to address the key issue of whether there are aspects of pedagogical usability that are discipline-specific, the article examines how language learning and teaching, in particular Technology Enhanced Language Learning (TELL), has approached usability when developing technology-enhanced learning materials. Three elements of a research agenda are identified: (a) pedagogical usability, (b) intercultural usability, and (c) website evaluation. In conclusion, it is suggested that, just as language learning websites may require a discipline-specific approach, other disciplines' pedagogical usability may also need to be considered in relation to the specific requirements of the discipline, and that a debate around these issues is timely.

**********

Usable systems are generally regarded as being efficient, easy to learn, effective to use, and enjoyable or engaging from the user's perspective. For several years, the authors have been investigating aspects of pedagogical usability, that is, usability as this affects educational website design and development, particularly in the context of supported open and distance learning (Shield & Kukulska-Hulme, 2003a; Muir, Shield, & Kukulska-Hulme, 2003a). As part of that research project, student responses to the Annual Courses Survey, a questionnaire administered every year to obtain student feedback about different aspects of Open University (OU) courses, were analysed (Shield & Kukulska-Hulme, 2003b). The survey includes questions about student satisfaction with learning support, such as tutorials and ease of contacting tutors, and aspects of the learning materials that are offered in various media ranging from print through audio, video, CD-ROM, DVD, and supplementary software to generic and course-related websites. It should be noted that questions included in the Annual Courses Survey do not directly address course website usability. Rather, students are asked how helpful they find course websites, how often they access these websites, and how satisfied overall they are with the information and materials located there. The authors believe, however, that students' responses to the questions posed in the survey, provide a useful perspective on how they view their experience of their course-related websites.

Completed questionnaires were received from 1582 students of languages. Analysis of their responses to questions concerning web-based materials revealed their ratings of their course websites to be the least favourable in the university. They displayed the lowest levels of satisfaction with electronic resources accessed through their course websites. Just 14.5% of those who responded chose the "very helpful" option. On the other hand, 16.6% of them considered these resources to be "not at all helpful," while only 6.8% of students from both the Business School and the Faculty of Science came to the same conclusion. Electronic resources accessed from non-OU websites fared slightly better, with 27.6% of Languages students who responded to this item choosing "very helpful" and only 10.1% selecting "not at all helpful" from the available responses. This result, however, still places Languages students among the most dissatisfied groups across the university. Only students from the School of Health and Social Welfare (10.5%) and the Faculty of Mathematics and Computing (10.7%) had a higher dissatisfaction rating in this category. In contrast, when asked to rate the quality of information provided by generic, institution-wide websites related to study skills, Languages students' level of satisfaction was much higher: 38% of those who responded to this question considered themselves to be "very satisfied," while only 3.3% professed to be "not at all" satisfied. These responses showed Languages students' satisfaction levels with generic websites to be higher than for their course websites. In the former case, though, they expressed the least satisfaction of students surveyed across eight faculties, whereas in the latter, they comprised the fourth most satisfied group.

By invitation of the academic team involved in its development, the authors carried out an evaluation of one OU Languages course website. This evaluation revealed several usability issues, ranging from navigation (e.g., insufficient differentiation between graphical links and graphics used as non-clickable backgrounds) to organisation of information (e.g., random insertion of new information, making it difficult for the user to find. For instance, information relevant to the completion of student assignments might be inserted between information about online tutorials and links to unordered authentic resources). Furthermore, comments made by OU language students who were interviewed about their course websites suggested that there may be deeper issues involved than those that can be addressed by generic usability guidelines of the sort commonly adopted by website designers and developers (Nielsen, 2000). For example, students have access to course websites, to the OU's generic Student Home website and to the OU Library website. During the course of interviewing Languages students at a day school, it became apparent that although they were aware that these were different websites, they regarded them as part of a continuous whole. One student reported difficulties in accessing the Library website for example, but, for that student, the issue was not that the Library's server was unstable. Rather, the link from the course website to the Library website "did not work," as a result of which that student stopped attempting to access the Library website completely.

These findings, and those that emerged from an analysis of responses to the Annual Courses Survey, raised the question of whether there are aspects of pedagogical usability that are discipline-specific. As Jones, Zenios, and Griffiths (2004) point out, this is not a well-understood area: "Research investigating disciplinary differences has not been fully developed to explore whether such disciplinary and subject differences affect the ways in which digital resources are conceptualized and used" (p. 1).

This article explores the issues around the usability of e-learning websites. With a view to developing an initial research agenda concerning the possible effect of discipline-specificity on such websites, it goes on to examine the perspective of a single, exemplar discipline, namely foreign language learning. First, the notions and concepts of usability are considered, and how generic website usability studies have influenced educational websites. Next, the ways in which language learning and teaching, in particular Technology Enhanced Language Learning (TELL), has approached usability, are examined. The outcomes of research with OU languages students are drawn upon and, finally, based on the authors' investigations in this field, a research agenda is proposed taking into account the discipline-specificity of pedagogical usability studies.

Refining the Notions and Concepts of Usability

The Web provides a means by which to offer learners access to up-to-date and easily-updatable course materials, activities, resources, and tutorial support. In spite of these benefits, learners may be left frustrated or disappointed by their encounters with their course-related websites when these do not address their needs or expectations. Preece (2000) remarked that website developers often assume they know users' wants and behaviours, forgetting their own high level of technical expertise. Difficulties may also arise when graphic design is prioritized over other aspects, so that a web page looks attractive but is difficult to read (Brinck, Gergle, & Wood, 2002). In other words, website design is typically the province of the technical rather than the content expert. However, increased awareness of user issues relating to website design means that website usability is an area that touches those people involved in educational website development, such as academic content providers, who would not previously have thought this to be relevant to their work.

Usability and Design: Is There a Difference?

The concept of usability was originally developed within the discipline of Human-Computer Interaction (HCI) and applied to the interaction of one user with one computer. In the early 1990s, Preece et al. (1994, p. 14) described usability as a "key concept" in HCI, concerned with making systems "easy to learn and easy to use" and distinguishable from the notion of "user experience," which encompasses a wider set of concerns such as creating systems that are satisfying, enjoyable, fun, entertaining, helpful, aesthetically pleasing, supportive of creativity, rewarding, or emotionally fulfilling. At that time, HCI researchers already recognized that to produce systems with good usability, it was necessary to understand the psychological, ergonomic, organizational, and social factors that determine how people operate, and to consider group working, integration and interaction of media, as well as the wider impacts of computer technologies. While how people learn to use a computer system was well understood, much less was known about how people learn with, as opposed to about, computers. HCI being a specialized area of knowledge, it appears that there was also a gap between what was known and the actual practice of interface design, a problem that would still be recognized today. A few years later, Preece (2000) had moved on to consider usability in the context of online communities, concluding, "... software with good usability supports rapid learning, high skill retention, low error rates and high productivity. It is consistent, controllable, and predictable, making it pleasant and effective to use" (p. 27).

To support online communities, Preece (2000) implied that there is a further dimension to consider: developers must identify software with suitable usability, such software often being web-based, or embedded in a website, "... then tailor it more closely to meet the community's needs" (p. 27). In a subsequent book on Interaction Design, Preece, Rogers and Sharp (2002) explained that "interaction design" is defined as "designing interactive products to support people in their everyday and working lives." This is also described as "finding ways of supporting people," and it is concerned with a broader range of issues, topics and paradigms than has traditionally been the scope of HCI. It entails "creating user experiences that enhance and extend the way people work, communicate, and interact" (Preece et al., p. v). Usability is again distinguished from user experience, the former encompassing effectiveness, efficiency, safety, utility, learnability, and memorability, while the latter, as mentioned, has a focus on aspects such as satisfaction and enjoyability. A difference between generic design principles and generic usability principles is also identified: "... whereas design principles tend to be used mainly for informing a design, usability principles are used mostly as the basis for evaluating prototypes and existing systems" (p. 26). In other words, design principles are general reminders about what to provide or to avoid when designing websites, while usability principles are used to assess the acceptability of interfaces. An interesting question is the extent to which these two areas of responsibility should be separated. E-learning brings usability into a shared arena, highlighting the need for technical or design experts and academic experts to work together more closely than ever before to produce usable websites. Figure 1 presents one possible interpretation of the relationship between the components the authors believe to be central to generic usability. It is recognized, however, that generic usability could be seen as contributing to interaction design rather than the other way round. For the authors, though, the contribution of interaction design to the ease with which users can learn and use a system and to the overall user experience is of central importance. Without taking interaction design into consideration during the planning and design phases, the overall usability of a system when it is evaluated is likely to be low.

[FIGURE 1 OMITTED]

BEYOND USABILITY: E-LEARNING

Understanding of the specific requirements of usability in web-based learning and e-learning is gaining momentum, but there is little published research as yet in this quite specialized area. Among researchers in educational applications of computing, Shiratuddin and Hassan (2001) described a usability-testing model they proposed that will help teachers to check the usability of their sites. Wade and Lyng (2000) noted that not all academic materials providers have expertise in designing usable sites. For this reason, they investigated the possibility of developing a web-based automated service to carry out usability evaluations for materials producers. Avouris, Dimitracopoulou, Daskalaki, and Tselios (2001) turned their attention to student assessment and evaluation systems, asking what effect system usability may have on learner performance. Laurillard (2002) addressed issues of usability from a pedagogical perspective, focusing on three aspects: (a) user interface, (b) design of learning activities, and (c) checking whether learning objectives have been met. She emphasized that "the aim is to design an interface that never intrudes on the task in hand" (p. 194). Hale and French (1999) considered the assessment of web design based on what they describe as "learning principles": (a) reducing conflict and frustration; (b) repetition of concepts using variations in technique; (c) positive reinforcement; (d) active student participation; (e) organization of knowledge; (f) learning with understanding; (g) cognitive feedback; (h) individual differences; and (j) motivation. Hall (2001) concluded that in training websites, several themes are important: site organisation, taking advantage of the hyperspace environment by building in flexibility and learner control, and use of case examples from the vast resources available on the Web. Hall also believed that collaborative learning activities should be part of such a web environment. Cronje (2001) warned against placing learning materials "in a prepackaged instructivist learning shell," typically a system that does not allow content providers to employ a range of pedagogical approaches in their e-learning materials. Such a shell, Cronje believed, may create an impoverished environment, one in which creativity and imagination remain unchallenged. Nielsen (2001) remarked that although general usability standards apply equally to e-learning, there are additional considerations, for example, the need to keep content fresh in learners' minds so that they do not forget things while trying to accommodate new concepts.

Introducing Pedagogical Usability

In the authors' own usability project at the UK Open University, the idea has been developed that there are several layers of usability, namely technical, general, academic, and context-specific (Muir et al., 2003a). Further, the authors have gone a long way toward refining their understanding of the academic and context-specific aspects (summarized in Figure 2), which include the broad context of e-learning as well as contexts defined in terms of specific disciplines and the learning activities undertaken within those disciplines. It has become apparent how the specific context of a website's use can dictate certain pedagogical design decisions. For example, knowing that Technology students tend to have much more experience with computing than Healthcare students means a site for a Technology course based on the Internet may be able to prioritise the use of cutting-edge technology, while a site for a Healthcare course probably would not.

The way the authors see it, if the learning and teaching resources supplied by an educational website are not presented and sequenced in a pedagogically-focused manner, the learner is less likely to succeed in achieving the specified learning outcomes of the course.

[FIGURE 2 OMITTED]

To take a concrete example, language learning can be seen as being not only an intellectual pursuit but also skills-based; a successful learning outcome cannot be achieved unless a learner is able to demonstrate her/his prowess in one or more of "the four skills" of reading, writing, listening, and speaking upon which language learning activities tend to concentrate. One major learning outcome of such activities usually relates to the ability to speak to others in the foreign or target language. Until recently, it has not been possible to support interactive, synchronous speaking practice of this sort, that is, learners speaking to others, online. This has been a major challenge to the pedagogical design of online language courses, many of which have addressed it either by ignoring the need for learners to practise speaking in the foreign language or by including a face-to-face component, rather negating the distance element included in the pedagogical design as a whole. A third option has been to use synchronous text-based communication tools. The assumption here is that the same effect on the acquisition of the target language will be achieved by using any real-time communication tool, regardless of channel and despite research evidence suggesting that synchronous text- and voice-based discourse are not identical (Weininger & Shield, 2003). With the advent of ubiquitous voice-over-internet applications, however, such issues can now be addressed. Language learners can talk with others in the foreign language in real time, thus gaining experience that will allow them to achieve learning outcomes concerning speaking in the target language, although it could be argued that the context in which the communication occurs (e.g. audio or video) may affect its realisation. Here, then, pedagogical usability depends on technical usability to support pedagogical design decisions. But that is not the end of the story. No matter how pedagogically effective the content of the activities may be, it is of little use if the learner is unable to locate it in a poorly organized website. Pedagogical usability, then, is also based upon principles of general usability. Technical usability is therefore the basis for the other aspects, whilst not being sufficient by itself.

Technology-Enhanced Language Learning and Usability Research

Technology-enhanced language learning (TELL) is an "umbrella" term that incorporates different approaches to the use of computers for language learning and teaching. It includes computer-aided/assisted language learning (CALL), often drill-type programs intended to improve accuracy in the target language, computer-mediated communication (CMC), and even generic computer-based production and presentation tools such as word-processing packages and computer-supported research tools such as concordancing and parsing programs. As it has become ever more ubiquitous, the Web has been used by TELL practitioners for the development of materials and activities for both CALL and CMC as well as to support learner access to authentic target language materials that may be used as part of the overall experience of engaging with the target language and culture.

APPROACHES TO TELL

TELL materials and activities have their beginnings in different language teaching philosophies ranging from approaches that focus on form and concentrate on drilling particular forms of the target language (the language being studied) through to pedagogies that draw upon constructivist philosophies and focus on fluency, collaborative, and cooperative learning. Describing the necessity to make principled choices about which technologies to employ for language learning, particularly at a distance, Doughty and Long (2003) place TELL within a theoretical framework that refers primarily to second language acquisition (SLA) theory and educational psychology. Referring to the methodological principles employed in "an embryonic theory of teaching" known as Task-Based Language Teaching (TBLT), they map these onto pedagogical procedures in the second language classroom and in TELL in a way that seems to be applicable to language learning and teaching more widely. In their work, Doughty and Long drew a useful distinction between methodological principles that are "... motivated by theory and research findings ... which show them to be either necessary for SLA ... or facilitative of it" (p. 51) and pedagogic procedures which "comprise the potentially infinite range of local options for realizing the principles at the classroom level" (p. 53).

Indeed, if methodological principles, as Doughty and Long (2003) suggested, might be "... universally desirable instructional design features...." a similar approach to e-learning website usability might well distinguish pedagogical methods from their realization in local contexts--issues that are implicit in the work of researchers such as Hemard and Cushion (2000) and Plass (1998). Hemard and Cushion, touch upon the possibility of discipline-specificity in usability studies, reporting that in their work they strive to "... establish important links between HCI and CALL as two distinct, but overlapping and interdependent, disciplines ..." (p. 103). Plass argued that: "... evaluation criteria [for multimedia software] need to be developed based on domain specific learning processes and activities and on the cognitive processes that these activities involve" (p. 35) and proposes a model for interface design that "... puts the user, the content, and the instructional activity in the center of the design process" (p.41). The key question is whether it is possible to identify "local features", specific to the methodology of a discipline's pedagogy, and, if so, does incorporating these pedagogic procedures into web-based learning materials influence their usability?

Approaches to Usability Within TELL

An examination of the literature of TELL reveals that usability issues, sometimes also referred to as evaluation criteria, have been addressed mainly in the area of standalone CALL materials, although practitioners such as Kelly (2000) have produced guidelines to help language teachers, in particular teachers of English as a Second Language, to develop "good" websites. TELL researchers and practitioners demonstrate awareness of general usability heuristics such as those proposed by Nielsen (2000, 2001), referring to aspects such as "user interface" and "HCI" (Hemard & Cushion, 2000; Allum, 2001), but references to the user experience are now beginning to appear more frequently in the literature; for example, Hemard (2003) captured the notion of "going beyond" general usability, offering a definition of usability that includes reaching "a threshold of acceptability beyond which users can begin to interact productively and voluntarily instead of simply acting and reacting" (p. 23). Indeed, investigations into aspects of TELL-based usability appear to reflect an underlying trend similar to what is described in general usability literature, namely a move from a focus on technical usability to an approach that is centred increasingly on the user experience. For example, TELL usability research has ranged from studies of how to transform "... good lesson plans, classroom or textbook activities ... into effective CALL exercises" (Allum, 2001, p. 146), to investigations of "students' views on the web as a viable environment for learning" (Felix, 2002, p. 48).

The fact that usability has been approached from a variety of standpoints within the TELL community becomes very clear in a study carried out by Levy (2002). Using a corpus drawn from a survey of CALL literature, he addresses the notions and concepts that have emerged around the term design in that field. Although the main thrust of his research is to describe "..the discourse processes and products of design in CALL ..." (p. 58), Levy also addresses some aspects of usability as these emerge in what he refers to as "design of an artefact," that is "... the building of websites, learning environments, courseware packages, exercises and authoring systems ..." (p. 61) and language learning websites, primarily aspects of HCI as these involve the end-user. Two websites, which Levy referred to as "integrated learning environments," in particular, are identified as successful, Stanford African Languages and Multimedia Applications (SALAMA) and the Virtual Language Center (VLC) because: "They each involve a coordinated and suitably linked collection of resources plus a mixture of tools and tutorial features that, properly managed, can provide learners with the information and help they need when they need it" (p. 64). The emphasis here is not on the interface, but rather on the content of the website. However, content alone does not make a website usable, as Allum (2001) pointed out: "Teacher-designed, pedagogically sound software often fails in terms of usability" (p. 146). Allum implied that generic usability principles as well as sound pedagogy must be employed in the development of usable language learning websites, arguing that software that does not meet the end-users' expectations in terms of interface design is likely to prove to be unusable. He urges that "general usability concepts" should be employed throughout the design process and commends Nielsen's generic usability guidelines to the reader.

In 2002, Hubbard (2003) carried out a survey to elicit research questions that TELL professionals (Hubbard calls them CALL professionals) would like to see answered. He not only asked respondents to outline their "favorite" research question, but also to classify their own position within TELL as primarily practitioners (teachers), researchers, or software developers. The survey, a covering letter explaining its purpose and a consent form were sent to 120 researchers who were considered to be "professionally active" in TELL. Professional activity was defined on the basis of fulfilling at least one the following four criteria: (a) membership of the committee of a professional TELL organisation such as EuroCALL, (b) membership of the editorial board of a professional TELL journal such as ReCALL, (c) recent publication in TELL journals, and (d) presentation at a TELL conference.

Hubbard's (2003) analysis of the 64 completed surveys revealed that (a) research questions fell into four main categories, design-centred issues, learner-centred issues, effectiveness issues, and research-centred issues and (b) 29 respondents considered themselves to be researchers, 22 practitioners (teachers), and 12 classified themselves as software developers while one respondent abstained from choosing a particular category. Hubbard found that the majority of questions posed by his respondents focused on "effectiveness issues" (e.g., does TELL "work"?), and this is telling. The question of its effectiveness is a major area of research for those working in TELL. There is, as yet, no definitive proof of its effectiveness, whether standalone, CMC, or web-based in format, whether drill-and-practice or authentic use of the target language in content. From the point of view of the usability researcher, the category design-centered issues, in Hubbard's survey, is probably of most interest, and is defined as: "... questions concerning the appropriate design of CALL [TELL] software, tasks or environments." (p. 149). Of the 15 responses received in this area, 10 could be said to fall into usability issues (2 content, 2 evaluation, 3 HCI, and 3 multimedia) but only two of these questions were suggested by respondents who considered themselves to be primarily software developers. Under Hubbard's definition, questions that fall into his category "learner centered issues" also address some aspects of usability. According to Hubbard these are concerned with "... learner variables, learner strategies and human-computer interaction" [authors' emphasis] (p. 149). There were 29 responses in this category. Of the four concerning "interaction," three were provided by software developers, while one was suggested by a practitioner. They included reference to: "activity design," "interactivity and intelligence," "learner-computer interaction," and "types of interaction" (in this case, this appears to mean "activity types" again).

A second aspect of Hubbard's (2003) survey was an attempt to identify whether self-classification correlated with different types of research question; here, although the number of responses received was relatively small, the difference between the interests of software developers as opposed to researchers and practitioners was quite marked (Table 1).

Software developers concentrated on HCI and interaction issues, areas that relate to usability, while those who classified themselves "researchers" or "practitioners" tended to ask questions related to pedagogy, learner variables, or language acquisition issues. Only one researcher and no practitioners asked an interaction-related question, suggesting a possibly different focus may indeed exist according to professional background. These findings are clearly not conclusive, being based on a relatively small, self-selecting sample. Hubbard (2002) noted that trends identified in his survey are empirically weak. He pointed out that the relatively limited number of questions from software developers may be the result of the low number (12 / 64) of his respondents who considered their primary role to be in this category. Furthermore, Hubbard admitted that the tentative taxonomy he developed on the basis of responses to the survey might differ were another researcher to classify them. He stated that, nevertheless they: "... may still provide some insight into the current state of mind of a large group within the field."

Felix (2002) is one of the few researchers who reported directly on learner views of the appropriateness of web-based language learning sites for independent study: "An analysis of which aspects of the web materials contributed to the perception of their usefulness showed that the most important elements were clear and logically organized content ..., clear objectives ..., meaningful feedback ..., and easy navigation through the program ..." (p. 51).

In language learning literature, then, usability appears to be a recognized area that has been somewhat explored from several different angles. Some researchers (Hemard, 2003; Felix, 2002; Allum, 2001; Plass, 1998) stress the importance of the relationship between content, user expectations, and usability principles, while the different approaches to software and website design become apparent in surveys of the literature (Levy, 2002) and CALL (TELL) professionals' own views (Hubbard, 2003).

Towards a Research Agenda

So far in this article, the authors have presented an overview of generic website usability principles and concepts and their evolution and a review of usability as it appears in TELL literature. Taking these sources together, it appears that there are elements that could form part of a research agenda, that have not been addressed in depth and that are particularly pertinent to language learning websites, the exemplar discipline. These fall into three main areas:

1. Pedagogical usability. For language learning websites, there is an important set of choices that needs to be made, and which is not really addressed in any of the sources that have been examined here: should the website use the first or target language? Linked with this question are pedagogical concerns such as: should the interface be presented in the target language? While often pedagogically desirable, would such an interface expose learners to particular usability issues?

Furthermore, given that websites can, and often do, take a multimodal approach, what presentation media should be used and how should users interact with them? Kress and van Leeuwen (2001, p. 46) pointed out that in postindustrial societies, multimodal methods of communication may be of at least equal importance to language. The ways in which language experts conceptualise user interfaces may also be specific to the culture and sub-cultures of their discipline. For example, Kukulska-Hulme (1999) and Boardman (2005) both interpreted user interfaces and user interactions from the perspective of language analysis; Boardman made the point that some of his analysis "hinges on the idea of narrative as the driving force behind the Web and how it works as a form of communication" (p. 3). These issues are closely related to the second aspect of usability that may be of particular relevance for language learning websites, intercultural usability.

2. Intercultural usability. Although intercultural aspects of website usability are the focus of research (Evers, Kukulska-Hulme, & Jones, 1999; Marcus & West Gould, 2000), there are likely to be specific issues around intercultural usability bound up with e-learning websites, particularly with language learning websites. As well as questions about which language to use, the way in which information is best presented is worthy of further investigation. Among those questions that might be addressed are included: do usability principles vary according to culture? If so, should language learning websites adhere to the usability principles of the home or the target culture? Here, the relationship between intercultural and pedagogical usability is clear, since the pedagogy of the home culture may influence the course-related website in a way that could negate the intercultural aspects of the study of the target culture. There are signs of an awakening interest in the portrayal of "self" and "other" in website design. For example, van Dijck (2000) compared two Colombian portals, one that is an adaptation of a U.S. portal to a Colombian context and one that was developed in Colombia, while Singh & Baack (2004) investigated culture-specific versus culture-nonspecific styles in US and Mexican websites. In contrast, in spite of a growing interest in intercultural aspects of website usability, there is little discussion of general e-learning or language learning websites. On the whole: "Current emphasis in modern language teaching and learning highlight interculturality and reconceptualise goals in terms of producing 'intercultural speakers' who will be capable, adaptable actors and mediators in globalized contexts" (Jordan, 2002). There are, however, a few exceptions. In their description of the Cultura project, Furstenberg, Levet, English, & Maillet (2001) noted: "Our focus will be on the pedagogy of electronic media, with particular emphasis on the ways in which the Web can be used to reveal those invisible aspects of a foreign culture, thereby giving a voice to the elusive 'silent language' and empowering students to construct their own approach to cross-cultural literacy" (p. 55). The project itself addresses issues of interculturality such as how the target culture is reflected in articles from authentic websites from the home and target cultures. It does not explicitly consider the types of usability issues that may surround these websites, although some explanation for this might lie in the "invisible aspects of a foreign culture" they refer to, and which are partially addressed by one respondent to Hubbard's (2002) survey. In her/his research question, s/he raised several issues the authors would expect to find in an approach to intercultural usability, although not all of those they have identified above: "What is the best screen lay-out of data for language acquisition? What colours, fonts, windows, multi-media inputs, and so forth, enhance most (or detract from) the cognitive process? Does the cultural background of the user lead to different reactions? Does the employment of "western-style" screen lay-out do a disservice to learners from other cultures? Are we using the optimum lay-out in the West?"

3. Website evaluation. The authors' own usability research has included face-to-face small group interviews with 22 OU Languages (French and Spanish) student volunteers about reactions to their course websites, which typically include learning resources and give access to discussion fora. Participants, who were all distance learners, had met for a face-to-face day school, thus providing the opportunity for interviews to take place in a single geographical location. Interviews, which were carried out by the project team's Research Fellow, were semi-structured (Table 2) and took no longer than 10 minutes, since volunteers could only participate during their coffee break. The content of the course websites was provided by teams of content experts, while their structure, function, and graphic design were generic and developed by a team of website developers.

Students from three courses, two French and one Spanish, were interviewed. Unfortunately, no students of German were able to participate because of prior commitments, so only two of the three languages offered by the OU were represented in this survey. Typically, groups of two or three students were approached at a time and the purpose of the study was explained to them. Key points of their responses to each question were taken down in writing rather than audio- or video-recording their semi-structured interviews. The interviewer thus did not have to ensure that technical equipment was working before commencing the interviews. The advantage of this approach lay in its simplicity and flexibility. In the context of a busy day school, this was felt to be appropriate and relatively unintrusive.

Analysis of the information provided during the interviews revealed that participants had not used their course websites regularly (Muir, Shield, & Kukulska-Hulme, 2003b). When questioned further, they rationalised this behaviour through factors including: lack of time, belief that "traditional" materials like books, audio-tapes, and video-tapes were sufficient and abundant enough to make redundant the need to access a website and misconceptions about the role of language course websites; one student, for example, suggested that there was little point in accessing a website, since it would not be able to provide the answer to relevant questions such as, "How do you pronounce...."

Reasons for these students to access their course website were primarily practical or social. They would use it to check for information about submission dates for assignments, to discover whether new course materials had been despatched to them or if any of their peers were online in their course's conference chatroom. Students also believed that their course websites could provide them with a portal to authentic target language and culture websites, including railway timetables in the target countries, cheap holiday and flight websites and tourist sites. One student reported that s/he enjoyed using the links to external articles providing different perspectives on aspects of Spanish culture such as religion.

The interpretation of the course website as a portal to authentic resources implies that students expect content experts to offer pointers to "good" authentic websites. At the same time, however, the question arises as to how learners themselves could evaluate the "worthiness" of authentic target language websites when undertaking both guided and independent browsing during the course of their studies. This appears to be closely related to intercultural and pedagogical aspects of language learning websites; if learners are to be able to carry out independent web browsing in the target language, they will need the tools to evaluate the content of what is available in terms of linguistic accuracy, reliability, bias, and up-to-dateness. For example, students of Spanish who took part in the interviews described here said they would have liked their course website to link to a political website, FARCEP (http://www.farcep.org/), the website of the Columbian guerrilla organisation, FARC.

Issues of how the Web is used by multilingual users have hardly been touched upon, and besides the ability to evaluate website content, would include the usability of search tools as well as an understanding of emerging language registers and conventions on the Web.

CONCLUSIONS

It has been shown how the general usability research literature reflects a gradual change from a focus on "ease," that is, making systems easy to learn and easy to use, towards an interest in the user experience, which encompasses a wider set of concerns such as satisfaction, enjoyment, and helpfulness, and looks for ways of supporting people. Enhancing and extending the way people work, communicate, and interact, are key principles in this evolved approach to usability. Within an online community context, principles such as consistency, control, and predictability have been highlighted, along with a need to tailor the interface more closely to meet the community's needs.

Going "beyond usability" is about looking for learning principles and seeing learners interact productively and voluntarily through an interface that does not intrude on their task. It is about taking advantage of the hyperspace environment by building in flexibility and learner control. It involves an appreciation of cognitive activity such as the need to keep content fresh in learners' minds. What is more, creativity and imagination should not be forgotten.

In terms of language learning it seems, little research directly related to the usability of language learning websites has, as yet, been carried out; what has been done has concentrated, on the whole, on the technical aspects of usability. On the other hand, more recent studies have begun to reflect a growing interest in the end-user experience of language learning artefacts such as websites with researchers such as Felix (2002) and Hemard and Cushion (2001) reporting on end-user feedback. Levy (2002) identified some of the features that go towards making a language learning website usable (suitably linked resources with learning support tools) and Hubbard's (2002) survey revealed that researchers in the area of online language learning are interested in usability-related issues such as the relationship between the technical and the pedagogical, with one participant in the survey stressing the importance of considering the end-user experience.

Finally, it would appear that there are areas of usability relating to language learning websites that are currently under-researched and that may require a discipline-specific approach. These include intercultural aspects of language learning and authentic target language websites and how these may affect the end-user's experience. Issues that may, at first sight, appear to belong to more generic usability principles may also be discipline-specific; one of Hubbard's questionnaire respondents, for instance, asked whether choice of font, colour, or page layout may in some way affect second language acquisition. While the authors are aware, at least to a certain extent, of the effect such aspects have on the end-user from the point of view of generic usability, it is less clear at present whether these have a similar effect from a discipline-specific perspective. Since the primary aim of language learning websites must be to promote the acquisition of the target language and culture, these are questions of great importance and worthy of further investigation. By raising these issues with respect to language learning, the authors also hope that this will promote wider debate about discipline-specificity in website usability and the requirements of different disciplines.

References

Allum, P. (2001). Principles applicable to the production of CALL-ware: Learning from the field of human computer interaction (HCI). ReCALL, 13(2), 146-166.

Avouris, N., Dimitracopoulou, A., Daskalaki, S., & Tselios, N. (2001). Evaluation of distance-learning environments: Impact of usability on student performance. International Journal of Educational Telecommunications, 7(4), 355-378. Retrieved March 10, 2006 from http://dl.aace.org/6469

Boardman, M. (2005). The language of websites. London: Routledge.

Brinck, T., Gergle, D., & Wood, S.D. (2002). Usability for the web: Designing web sites that work. San Francisco: Morgan Kaufmann.

Cronje, J.C. (2001). Metaphors and models in Internet-based learning, Computers and Education, 37, 241-256.

Doughty, C., & Long, M. (2003). Optimal psycholinguistic environments for distance foreign language learning. Language Learning & Technology, 7(3), 50-80. Retrieved March 10, 2006 from http://llt.msu.edu/vol7num3/doughty/default.html

Evers, V., Kukulska-Hulme, A., & Jones, A. (1999). Cross-cultural understanding of interface design: A cross-cultural analysis of icon recognition. In E.del Galdo & G. Prahbu (Eds.), Designing for Global Markets: Proceedings of the First International Workshop on Internationalisation of Products and Systems (pp. 173-182). Rochester, NY: Backhouse Press.

Felix, U. (2002). The web's potential for language learning: The student's perspective. ReCALL, 13(1), 47-58.

Furstenberg, G., Levet, S., English, K., & Maillet, K. (2001). Giving a virtual voice to the silent language of culture: The CULTURA project. Language Learning & Technology, 5(1), 55-102. Retrieved March 10, 2006, from http://llt.msu.edu/vol5num1/furstenberg/default.html

Hale, C., & French, D. (1999). Web-related assessment and evaluation. In D. French (Ed.), Internet based learning: An introduction and a framework for higher education and industry. Sterling, VA: Stylus Publications.

Hall, R.H. (2001). Web-based training site design principles: A literature review and synthesis. In B. Khan (Ed.), Web-based training. Englewood Cliffs, NJ: Educational Technology Publications.

Hemard, D. (2003). Language learning online: Designing towards user acceptability. In U.Felix (Ed.), Language learning online: Towards best practice (pp. 21-42). Lisse, The Netherlands: Swets & Zeitlinger.

Hemard, D., & Cushion, S. (2000). From access to acceptability: Exploiting the web to design a new CALL environment. Computer Assisted Language Learning, 13(2), 103-118.

Hemard, D., & Cushion, S. (2001). Evaluation of a web-based language learning environment: The importance of a user-centred design approach for CALL. ReCALL, 13(1), 15-31.

Hubbard, P. (2002). A survey of unanswered questions in computer assisted language learning. Retrieved March 10, 2006, from http://www.stanford.edu/~efs/callsurvey/index.html

Hubbard, P. (2003). A survey of unanswered questions in CALL? Computer Assisted Language Learning, 16(2-3), 141-154.

Jones, C., Zenios, M., & Griffiths, J. (2004). Academic use of digital resources: Disciplinary differences and the issue of progression. Proceedings of Networked Learning Conference 2004. Retrieved March 10, 2006, from http://www.shef.ac.uk/nlc2004/Proceedings/Symposia/Symposium9/Jones_et_al.htm

Jordan, S. (2002). Intercultural issues in foreign language learning and ethnographic approaches to study abroad. The guide to good practice for learning and teaching in languages, linguistics and area studies. Retrieved March 10, 2006, http://www.lang.ltsn.ac.uk/resources/goodpractice.aspx?resourceid=100

Kelly, C. (2000). Guidelines for designing a good website for ESL students. Internet TESL Journal, 6(3). Retrieved March 10, 2006 from http://iteslj.org/Articles/Kelly-Guidelines.html

Kress, G., & van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. New York: Oxford University Press.

Kukulska-Hulme, A. (1999). Language and communication: Essential concepts for user interface and documentation design. New York: Oxford University Press.

Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). London and New York: Routledge.

Levy, M. (2002). CALL by design: Discourse, products and processes. ReCALL, 14(1), 58-84.

Marcus, A., & West Gould, E. (2000, June). Cultural dimensions and global web user-interface design: What? So what? Now what? Proceedings of 6th Conference on Human Factors and the Web, Austin, Texas. Retrieved March 10, 2006, from http://www.amanda.com/resources/hfweb2000/hfweb00.marcus.html

Muir, A., Shield, L., & Kukulska-Hulme, A. (2003a, June). The pyramid of usability: A framework for quality course websites. Proceedings of EDEN 12th Annual Conference of the European Distance Education Network, The Quality Dialogue: Integrating Quality Cultures in Flexible, Distance and eLearning, (pp. 188-194) Rhodes, Greece.

Muir, A., Shield, L. & Kukulska-Hulme, A. (2003b). Report on the usability roadshow to region 06. Internal Open University Report.

Nielsen, J. (2000). Designing web usability: The practice of simplicity. Indianapolis, IN: New Riders Publishing.

Nielsen, J. (2001). Just how important is usability in e-learning? Elearning Post. Retrieved March 10, 2006, from http://www.elearningpost.com/features/archives/001015.asp

Plass, J. (1998). Design and evaluation of the user interface of foreign language multimedia software: A cognitive approach. Language Learning and Technology, 2(1), 35-45. Retrieved March 10, 2006, from http://llt.msu.edu/vol2num1/article2/.

Preece, J. (2000). Online communities: Designing usability, supporting sociability. Chichester, UK: Wiley.

Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human-computer interaction. Harlow, UK: Addison-Wesley.

Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction design: Beyond human-computer interaction. New York: John Wiley & Sons.

Shield, L., & Kukulska-Hulme, A. (2003a, September). Language learning websites: Designing for usability. Paper presented at EUROCALL'03: New Literacies in Language Learning and Teaching, University of Limerick, Ireland.

Shield, L., & Kukulska-Hulme, A. (2003b). The annual courses survey (2002): A summary of student responses to questions about their course-related websites. Open University Knowledge Network Internal Document No. 3852.

Shiratuddin, N., & Hassan, S. (2001). Web sites usability evaluation guidelines for teachers: A case of CikguNet on-line environment. Proceedings of the Society for Information Technology and Teacher Education International Conference 2001 (pp. 3074-3079). Retrieved March 10, 2006, from http://dl.aace.org/4154

Singh, N., & Baack, D. (2004). Website adaptation: A cross-cultural comparison of U.S. and Mexican web sites. Journal of Computer-Mediated Communication, 9(4). Retrieved March 10, 2006, from http://jcmc.indiana.edu/vol9/issue4/singh_baack.html#fourth

van Dijck, P. (2000). Cultural portals: What makes 'em tick? Evolt.org. Retrieved March 10, 2006, from http://www.evolt.org/article/Cultural_portals_what_makes_em_tick/20/3099/

Wade, V., & Lyng, M. (2000). An automated evaluation service for educational courseware. World Conference on the WWW and Internet 2000 (pp. 569-574). Retrieved March 10, 2006, from http://dl.aace.org/100

Weininger, M.J., & Shield, L. (2003). Promoting oral production in a written channel: An investigation of learner language in MOO. Computer Assisted Language Learning, 16(4), 329-349.

LESLEY SHIELD AND AGNES KUKULSKA-HULME

The Open University

UK

[e-mail]

[e-mail]

Table 1 Types of Research Question Asked by Different Categories of TELL
Professionals

Type of question     Researchers  Practitioners  Developers

CALL Effectiveness   4            0              0
Interaction issues   1            0              3
Effective Practice   3            3              0
Learner Variables    6            6              0
Descriptive studies  5            3              0

Table 2 Semi-Structured Interview Instrument Developed to Elicit
Language Learners' Views of Their Course Websites

1. How often do you use the course website? Why is that?
2. How effective do you think the course website is in helping you
   learn? Can you say something about this in relation to:
   * the stated learning outcomes of the course
   * more general aims of learning the language outside the particular
     aims of the course unit.
3. How enjoyable or engaging is the site?
4. Which parts of the site do you like best/least?--Why?
5. If you were able to redesign the site, would you do anything
   differently?-If so, what and why?
6. In general, what sorts of websites do you like or dislike?--Can you
   explain why?