Institutional use of learning objects: lessons learned and future directions.

A learning resource catalogue (currently LRC3) that comprises records of learning objects has been used by members of the Universitas 21 Consortium for three years. Five conceptually useful classes of learning objects are employed. While almost all faculty who were introduced to the LRC appreciate its value, need, and ease of use, few are willing to use the system for themselves. There are issues such as time to complete a record (possibly several minutes) and reluctance to make teaching materials public. Although there are acknowledged efficiency gains made by sharing and reusing learning objects, one reason for the slow uptake is the lack of a reward system that parallels rewards for publicising research. While improvements to the LRC continue to be made, including collaborative tools and in record creation, cultural changes in the adoption of educational technology and the recognition and reward for teaching seem to be the main reasons that the utilisation of learning objects will take time.

**********

Three years ago at ED-MEDIA, Koppi et al. (2000) described how Learning Objects could be made part of an institutional learning environment by including a learning resource catalogue (LRC). The idea of learning objects was seen as the key concept that could influence the authoring of learning resources and the subsequent development and use of an LRC. Learning objects were defined as discrete chunks of reusable learning materials or activities that can communicate with other learning objects to build a learning environment. This is consistent with the definition given by the IEEE (2004). Other examples of database systems that address learning objects are given by ARIADNE (2004) and MERLOT (2004).

There are many definitions of learning objects (Rehak and Mason, 2003) and the meaning is still open to debate. Reusability seems integral to the concept (unique developments hardly seem worthwhile) although it is conceivable that some learning materials or activities are so contextual that they are only used once. The notion that learning activities are not learning objects and are disposable (Wiley, 2003) is not a concept employed by the LRC. We hold that learning activities, while they may be contextual, nevertheless may represent considerable and painstaking instructional (learning) design input and, as such, the ideas may be useful to others. In a recent development at the Open University (Weller et al., 2003), the notion of reusable learning activities was extensively employed. The LRC has a class of learning objects called task or exercise and several such learning activities are described in the catalogue.

The ability of a learning object to communicate with other learning objects (articulation) is not a necessary property of the learning object; rather, articulation is something that the teacher does with the object by placing it into a context. The same learning objects in different contexts can be combined in different ways as the teacher (and/or learner) decides. Therefore, recontextualisable seems to be an integral part of the property of a learning object. The more inherently contextual an object is, the less reusable it may be; something already loaded with context may be difficult or impossible to reuse in a new context (Hodgins, 2002). The notion that a learning object needs to contain some learning ingredient (as advocated by Bradley and Boyle, 2003) is also debatable; for example, a plain X-ray of a chest can be considered to be a learning object in a raw sense, and it becomes a different kind of learning object once it has been annotated. In the LRC, we include both of these categories; hence a learning object need not inherently contain learning material but it is capable of being used in a learning context.

The essential part of the LRC was a system for enabling the description of learning objects in standard metadata terms through a Web application (IMS, 2001-2004 and superseded by the more recent IEEE 1484.12.1 LOM standard (IEEE, 2004)). The metadata would then be submitted to an online searchable database to facilitate the reuse of the learning objects in other combinations and contexts. Only the metadata resided in the catalogue; the digital learning objects themselves resided on various distributed servers.

The impetus for the development of the LRC originated in 1999 from a small group, the Learning and Teaching Technologies Working Party (six members) of the Universitas 21 (U21) Consortium. This consortium of international, research-intensive universities saw the benefit of identifying and describing the learning and teaching materials that were owned by its members in order to facilitate their reuse and make efficiency gains by minimising replication. The creation of this database was seen by senior management of the consortium as an important project. The LRC became operational in early 2000 and was enthusiastically received. The LRC was aimed at academic staff and had to have certain qualities:

* Availability through a standard Web browser 

* Intuitiveness--usable with little training required

* Standardization of metadata

* Rapid data entry--only a few minutes per record

* Simple and efficient search and retrieval of records

Each member institution added about 50 records during an evaluation phase. As a result, several changes were made to the second version of the LRC.

* Each institution had to have its own version of the LRC and could chose to make records available locally or throughout U21

* Records entered by any institution had to rapidly be made available to other institutions

* Records entered were of quality learning objects

* The type of learning object record had to be made obvious from the beginning

On this last point, the evaluation revealed that there was a great variety of learning object records added to the LRC such as images, learning activities, and whole courses. It could take considerable time in browsing through the data fields to identify the type of learning object. Over and above the IMS metadata system, it was decided to create a classification of learning objects that was based on more than granular considerations (such as advocated by Wiley et al., 1999); this was described by Koppi and Hodgson (2001) at ED-MEDIA. Essentially the five types of learning objects can be illustrated by the following examples:

1. Raw asset, e.g., an image

2. Learning asset, e.g., annotated image

3. Task or exercise, e.g., research activity

4. Learning design with content (containing some or all of classes 1-3)

5. Generic learning design, e.g., a generic problem-based learning design

These are shown in Figure 1 as the drop-down list for resource description. This classification system proved to be convenient, allowing users to search the database for type of learning object. It is worth noting that this classification system yields metadata elements based on a vocabulary we developed that was specific and useful to our community of users. As such, this represents an example of a successful metadata implementation.

[FIGURE 1 OMITTED]

Figure 1 shows the manual data entry fields in LRC3 displaying the Required Fields and the drop-down list for the Learning Resource Category (type of learning object).

Relative to the requirement that each institution have its own version of the LRC and be able to choose to make records available locally or throughout U21, individual institutional databases were created (each with local and U21 access); they synchronised every 15 minutes. In other words, an approved record added to the University of Edinburgh LRC, and designated as a U21 record, would be visible to all the other 16 institutions 15 minutes later. In practice, all the different institutional versions of the LRC, with their own branding, were housed on the same server at UNSW and the speed of usage from other parts of the world was found to be satisfactory. More recently, the Universities of Melbourne and Hong Kong have installed their versions of the LRC on their local servers and the functionality remains the same. Other universities have indicated that they will follow suit.

Whether or not an institution chooses to make metadata records available to other U21 members, it has been recognised that there is value in identifying and cataloguing the learning and teaching resources within an institution. Such an endeavour was also noted by Collis (2001) and Collis and Strijker (2002) for the University of Twente.

Regarding the quality of learning objects referred to in the LRC, it was felt that only learning objects of high quality should be made available to members of the U21 consortium. This required human intervention and an approval process prior to making records visible to the whole consortium.

LESSONS LEARNED

Quality Control Implementation

Senior managers of U21 expressed the position that there should be a quality control process for the addition of records to the U21 pool (but not for local record additions) so that only the highest quality learning objects would be made available to the whole of U21. However, the actual development of a process was found to be problematic with no ready solution ever devised. It proved difficult to define quality. For example, a particular learning object may be worthless to one user (and hence be interpreted as low quality) but may be exactly what another user has been seeking. We did not solve the problem of who would police the uploading of records to U21 and what criteria would be used. The notion of only quality resources being present in U21 may also have acted as a deterrent to users, some of whom said that their materials were not good enough; but this was only based on their own undefined subjective criteria. It was decided to let the owner of the record make the decision (without formal criteria) as to whether or not the learning object would be submitted locally (only) or also to U21. In the end it was a matter of trust: we trust the teachers to provide their students with quality learning materials, so why should it be any different for the LRC records?

Data Entry

The intention from the outset was for the owners and creators of learning and teaching materials--the teachers themselves--to create the metadata records of their materials. Figure 1 shows the manual data entry fields displaying the Required Fields. Because the research and teaching staff are very busy, it was recognised that this process had to be quick and simple. This was a problem because of the approximately 90 metadata fields that describe learning objects using IMS Learning Object Metadata (LOM). Even a mandatory minimal data set of about 15 fields (used in LRC2) took several minutes--and possibly much longer depending upon whether or not it was possible to cut and paste from the digital object itself or whether entirely new entries had to be typed. This issue highlights the desirability of automated data whenever possible in the future. The number of required data fields of LRC3 (Figure 1) has been reduced to five because some fields, such as discipline and contact details, are automatically associated with the user. However, the resource (object) title, resource category (drop-down list in Figure 1), description, URL, and file type(s) still require manual entry. At this stage we can envisage more automatic metadata creation, such as with the title and file type(s); but the description will still need to be manually entered until we can find a way of generating this automatically. Other optional entries are concerned with creating a copyright license (under the optional secondary field tab); this is achieved by a direct link to the Creative Commons legal site (http://creativecommons.org/). This feature was requested by many users. The Access Management tab (Figure 1) also allows record owners to choose who can access the record, such as for personal use only, the institution, or the entire system. Therefore, data entry is still a manual process. However, the fields are few and give the user considerable control over how the learning object may be used and who can access it. Even that minimal data entry may still represent a barrier to individual teachers creating records of their own teaching materials. The ultimate goal would be to have automatic metadata generation of learning objects that did not involve any extra work for the owners of the materials--or for any other people such as those who may be currently delegated to create the metadata.

As long as data entry remains a manual process, it will remain problematic as to who should enter it. In several institutions the job has been passed to librarians who are perceived as people who do cataloguing. While this may seem convenient, it places the LRC one step further from the people who created and own the learning resources, i.e., the teaching staff. If they are distanced from the process of contributing their learning objects to the LRC, then it is less likely that the LRC will become a central part of their working lives. However, getting others to enter data may not be negative if there is a willingness to contribute and share learning resources by teachers. That willingness depends on other factors such as awareness and reward.

Building Awareness

The concept of learning objects may be well understood by some, even if the definition is unclear. However, many teachers are not aware of what it means and do not understand its value. In fact, the word object may itself be a deterrent because teachers do not think of their teaching materials, resources, lessons, and activities as objects. However, once they engage with the concept, and it becomes clear to them, the term object seems to be less of a concern and they start to use it. To think about teaching materials in terms of learning objects requires changes in thinking and in practice. To bring about change in thinking and practice requires an awareness and appreciation of the value of the change. Most busy teachers in research-intensive institutions (the universities of the U21 Consortium) have little time to invest in reorganising their teaching materials, particularly if it is seen as benefiting others rather than themselves. We recognise that publicising the LRC, encouraging contribution, and sharing of learning objects is necessary and that these must be coupled with rewards. We continually stress that control remains entirely in the hands of the owner of the learning object because all the owner really gives away is the description of the object and not the object itself. Nevertheless, rewards of some kind have to be obvious before we can expect widespread engagement with the concept or practice.

Reward

Closely linked to time investment of metadata record creation and learning object appreciation is the issue of reward. Reward was found to be desirable for two specifics--effort and ownership. There must be a good answer to the question teaching staff ask themselves: "Why should I bother?" The notion of using other people's teaching materials for the sake of efficiency or promoting one's own material does not by itself appear to be enough of an incentive or reward for everyone. Satisfying a social responsibility by making their teaching materials widely available is sufficient reward for some people. However, in general, teaching staff would like more reward that is tied to their prospects of self-perpetuation through channels such as recognition of their teaching that leads to promotion.

Reward in research-based academia is given by peer review and recognition, a practise that leads to publication and career progression. Taylor and Richardson (2001) have proposed a mechanism for peer reviewing different categories of learning objects. While this proposal has great merit, teaching staff may not take such a system seriously at this time because it does not yet have credible recognition. MERLOT also includes a discipline-based peer review process and 5-star rating system related to quality.

Establishing a credible reward system also requires change by senior academic administrators who write and administer the academic promotion criteria.

Academic Culture

While the notion of only quality learning objects being referenced in the LRC is a worthy aim, it has actually been a deterrent to staff members who may not feel that their own teaching materials would stand up to scrutiny. Academics have often commented on having many learning objects but that they do not consider them good enough for the LRC. In addition, a local LRC administrator may not be the best person to evaluate the quality of a learning object other than checking that the required data entry fields are complete to a reasonable degree of detail. Ironically, when academic groups have become involved in LRC population, high quality materials have been seen as not suitable for the LRC because teachers feel they are giving away their intellectual property (IP). Giving away IP does not worry them in the case of journal, book, and conference publications, presumably because there is clear reward attached to contributing to such publications.

Technology in learning and teaching is also still only appreciated by a minority of teaching staff and the majority of courses that utilise technology do so in an ancillary or support fashion (DEST, 2001). Experience in staff development and training in the use of WebCT at UNSW reveals that for most people simple use of technology is as far as they can go, or wish to go, under the circumstances of a research-intensive university where the on-campus experience is seen as paramount. For busy researchers and teachers, the creation of metadata records of their own teaching materials is often perceived as not providing a return on investment, at least not in the short term.

The big-picture (institutional and cross-institutional) view of the benefits of utilising a learning object approach and providing rewards for doing so, again rests with senior academic managers who defer the micro-management of learning and teaching materials to others. Within that culture, it is possible that the ownership of a tool such as the LRC would be lost.

Technological Uncertainty

A relatively new technology application such as the LRC, in an area not well known or understood by teaching staff, also brings with it feelings of uncertainty about its future and raises questions as to whether or not it will still be around in a few years and whether the investment in time will be worthwhile. This is coupled with the LRC link to the U21 Consortium, which itself represents a risk venture into future global online learning needs. The concepts of such online learning needs are somewhat remote from most campus-based teaching staff, especially those at research-intensive universities. These uncertainties cannot easily be addressed because there can be no guarantees about the success or growth of online learning.

Having been developed for the academic community, the LRC was aimed at a perceived need. Issues such as academic time availability and the need for reward to overcome these time and intellectual property barriers could not have been fully foreseen. There is little doubt that the benefits of sharing learning and teaching materials are still valid. However, it is evident that such a notion may only emerge once the issues of reward for effort and intellectual property contribution are overcome. It is also evident that this is an emerging area with considerable confusion as to perceived benefits; it is also an area that must be supported while the issues of reward are being solved. Further development of the LRC application is now addressing collaboration within the academic community as a missing link in the change process to learning object usage and development.

A Community Approach

To aid in the sharing and development of learning and teaching resources within and between institutions, the most recent version of the LRC3 contains, in addition to the catalogue tools, community tools. Examples include the following: My LRC (which can be customised), a communications centre for direct and email communication, group creation and management facilities, group file uploading, and survey and review creation capabilities. Figure 2 is a screen shot of My LRC and shows a list of groups of which Tony Koppi (TK) is a member, a calendar of events scheduled for October 2003, a clock that will show the time at other U21 institutions for immediate contact purposes, and other navigation options including the communications centre and the catalogue functions. Figure 2 shows that there are numerous groups to which TK belongs and that some of these groups are very active in discussions and sharing files (Figure 3). It is hoped that this community facility will encourage the development and creation of learning objects that can be added to the catalogue.

[FIGURE 2 OMITTED]

Figure 2 shows the community part of the LRC, such as My LRC and groups in which Tony Koppi is a member, an LRC diary of events, access to communications, and links to the cataloguing part of the LRC3.

Figure 3 shows the homepage of one very active community group with group function options and uploaded files in the left navigation panel. It may be observed that as yet no new learning objects have been created as a result of the group activities. Nevertheless, by promoting collaboration in a teaching environment it is hoped that the communities of practice supported in this way will have positive outcomes in directions that have yet to be evaluated.

[FIGURE 3 OMITTED]

Summary and Future Directions

To date, there have been three international meetings of U21 universities regarding the LRC: the first in Sydney, Australia in 2001; the second in Birmingham, UK in 2002; and the third in Hong Kong in 2003. These meetings have helped shape the future directions of the LRC, the current version of which has been named LRC3 to represent catalogue, cross-institutional, and collaboration.

LRC3 is a significant redevelopment. To address the time constraints that many academics face, the form for adding metadata has been improved, moving all of the mandatory fields to the front of a tabbed form. This also presents a less daunting task to the person completing the form. Instead of a long scrolling list of fields, the user now has only one page with the option of completing other fields, such as the educational and technical ones. Although users could save drafts of records and make templates from any of their records, the flexibility in the handling and sharing of these templates was limited. LRC3 introduces considerable flexibility in the sharing of draft and template records, effectively allowing users to create these records collaboratively and share them with each other.

The introduction of new flexible group-based collaborative tools enhances the possibilities offered by allowing the development of new collaborative data entry work flows. For instance, it will now be possible for an academic to work with an assistant in a private group environment and develop a database of records. Using flexible rights management options, records may be kept private within this group as drafts. Then, if so desired, once published, the access rights may be opened up to the whole system. To address concerns of Intellectual Property, users when publishing their records may keep read access to the records just to themselves or a selected group(s) of other users.

The new groups can be created by any users and are flexible enough to provide a range of uses from managing a small learning and teaching group to providing LRC level (cross-institutional) support functions. Collaborative tools available to groups include the ability to identify links, upload files, and identify learning objects to which members of the groups have read or write access. The discussion forum may be used as a place to discuss any of the resources, and effective internal communication tools have been added to facilitate easy communication. Users may send internal messages or external emails to any user within the system. This negates the need for the user to leave the application to communicate with others. In addition, large numbers of resources, such as web-links, learning objects, or files may be made available quickly and flexibly to all members of a group. Effectively, LRC3 could be used anywhere that a compatible browser is available.

The emphasis in developing collaborative tools is because of the ability to provide a support mechanism to new participants in the emerging world of educational technology. Support units, wherever they exist in the institutional structure, can use LRC3 to help them connect with academics, whether they are groups or individual users. Additionally, it transgresses the traditional community forums by providing new levels of security, effectively overcoming some of the intellectual property concerns whilst the change management issue of reward is being solved.

Developing a reward system that parallels the reward system for research is a slow process and requires institutional recognition of the value of developing and sharing learning objects for the benefit of student learning. At UNSW, it is intended that the development and use of learning objects will be used as evidence of teaching in the promotion process. A peer review system of learning objects will also add to the validity and credibility of these materials and therefore also provide a reward system. The development and acceptance of these reward systems will take time and demonstrates (yet again) that educational technology progress is a people issue.

The sustainable use of the LRC depends on the willingness of teaching staff to take ownership and utilise the system for their own benefit (which some are doing). Therefore, we have adopted a short-term strategy to help populate the LRC by creating records on behalf of the teaching staff and then assigning ownership of the record to the creator of the object who can then verify the record. It is intended that this scaffolding will be phased out once some sort of critical mass of records is attained and the reward system has become established.

A possible future strategy of automating the creation of metadata to a greater extent is presented by a possible functional link between the LRC3 and WebCT Vista which utilises the concept of learning objects. This convergence offers the opportunity of tagging learning objects (classes 1-3) as they are uploaded into courses. Lessons and courses (class 4) can be tagged when their creation is requested via web forms. In addition, establishing interoperability between distributed learning object catalogues and databases outside the LRC will add to the functionality and value of the LRC.

CONCLUSION

The value of a Learning Resource Catalogue (LRC), containing records of learning objects (created by teaching staff themselves) that can be reused and recontextualised, has been appreciated by almost all teaching staff that have been introduced to it. The Web-based system is simple to use with minimal training (if any) required. The concept of learning objects and how they can be utilised in the construction of learning environments is generally not well understood but the classification of five types of learning objects has aided in their conceptualisation and more rapid identification and retrieval from the database by users.

It is ironic how the issue of Intellectual Property (IP) is often used as a reason for not making teaching materials public when the same academics cannot give their IP away fast enough when it comes to publicising their research by way of publications and conferences. The issue of reward for publicising teaching and learning materials is of paramount importance to the success of a sustainable LRC where the teaching staff themselves take ownership of the system. This necessary change in the academic culture will be a slow one. Coupled with the simple use of educational technology on a large scale, this suggests an extended timeframe for widespread sharing and adoption of reusable learning objects.

References

ARIADNE (2004). ARIADNE Foundation for the European Knowledge Pool. Retrieved August 13, 2004, from: http://www.ariadne-eu.org/.

Bradley, C. & Boyle, T. (2003, June). The development and deployment of multimedia learning objects. Learning Objects Symposium, ED-MEDIA, Hawaii, USA.

Collis, B. (2001, June). Linking organisational knowledge and learning. EDMEDIA 2001, World Conference on Educational Multimedia, Hypermedia and Telecommunications, Tampere, Finland. Proceedings 311-316.

Collis, B. & Strijker, A. (2002, June). New pedagogies and re-usable learning objects: toward a different role for an LMS. ED-MEDIA 2002, World Conference on Educational Multimedia, Hypermedia and Telecommunications, Denver, Colorado, USA. Proceedings 334-339.

Creative Commons. Retrieved August 13, 2004, from: http://creativecommons.org/.

DEST (2001). Universities Online: A survey of online education and services in Australia. Commonwealth Department of Education Science and Training, Government of Australia. Retrieved August 13, 2004, from: http://www.dest.gov.au/highered/occpaper/02a/default.htm.

Hodgins, W. (2002). Learning by Design: Future of Learning Objects. AUTC Learning Objects Conference, UTS, Sydney, December. Retrieved August 13, 2004, from: http://www.iml.uts.edu.au/autc/pdf/AUTC hodgins.pdf.

IEEE's Learning and Technology Standards Committee. (2004). Retrieved August 13, 2004, from: http://ltsc.ieee.org.

IMS (2001-2004). IMS Global Learning Consortium, Inc. Retrieved August 13, 2004, from: http://www.imsproject.org/.

Koppi, A.J., Hodgson, L & Bayly, J. (2000, July). The often missing but essential component for online learning: a Learning Resource Catalogue. EDMEDIA 2000, World Conference on Educational Multimedia, Hypermedia and Telecommunications, Montreal, Canada. Proceedings, Vol. 1, 502-506.

Koppi, A. J. & Hodgson, L. (2001, June). Universitas 21 Learning Resource Catalogue using IMS Metadata and a New Classification of Learning Objects. EDMEDIA 2001, World Conference on Educational Multimedia, Hypermedia and Telecommunications, Tampere, Finland. Proceedings 998-1001.

MERLOT (2004). Multimedia Educational Resource for Learning and On-line Teaching. Retrieved August 13, 2004, from: http://www.merlot.org/.

Rehak, D. R. & Mason, R. D. (2003). Keeping the learning in learning objects. In: Littlejohn, A. (ed) Reusing online resources: a sustainable approach to elearning. Kogan Page, London.

Taylor, P. G. & Richardson, A. S. (2001). Validating Scholarship in University Teaching: constructing a national scheme for external peer review of ICT-based teaching and learning resources. Evaluation and Investigations Programme, Commonwealth Department of Education Science and Training, Government of Australia.

Weller, M., Pegler, C., & Mason, R. (2003, September). Working with learning objects--some pedagogical suggestions. Association for Learning Technology Conference Proceedings, 342-349. ALT-C, Sheffield, U.K.

Wiley, D. A. (2003, September). Prepare for IMPACT: Learning Objects, Learning Communities, and Standards. Presentation at Australasian WebCT Conference, Gold Coast, Queensland, Australia.

Wiley, D. A., South, J. B., Bassett, J., Nelson, L. M., Seawright, L. L., Peterson, T., & Monson, D. W. (1999). Three common properties of efficient online instructional support systems [Electronic Version]. The ALN Magazine, 3(2). Retrieved August 13, 2004, from: http://www.aln.org/alnweb/magazine/Vol3issue2/wiley.htm.

TONY KOPPI, LISA BOGLE (NEE HODGSON), AND NEIL LAVITT

University of New South Wales 

Australia

[e-mail]

[e-mail]

[e-mail]