Keeping faculty online: the case of Merlot.

Abstract

MERLOT is an online community of faculty with technology-enriched teaching as a shared goal. A survey of MERLOT Editorial Board members reveals key attributes of the academic community including on-going and sustained interactions, shared purpose or goals, reciprocity of information, support, and services, and conventions for establishing strong ties among participants. Patterns in the survey responses highlight camaraderie, leadership, and online community design as factors that support improved teaching. Benefits outweigh faculty time costs in accordance with sociological theory of distributed leadership.

Introduction

The Internet provides electronic support for virtual communities where people find others who share their interests. One study of online communities found 90 million Americans using the Internet to contact a group and referred to network users who nurture long-distance online community memberships as Cyber Groupies (Hoorigan et al, 2001). The subject of this paper is a survey of one particular higher education community of practice that has been functioning online for five years. Though it seems natural that academic communities would move some of their work to the virtual realm of online communities, not much is known about community attributes that can actually sustain the productivity of one type of online community in higher education: a faculty community of practice to improve effectiveness of teaching and learning. Indeed, many apparently consensus ideas about successful online communities remain to be tested for this particular group for whom information satisfaction and rigor are of particular concern.

Sociological theories suggest that building community requires on-going and sustained interactions, shared purpose or goals, shared resources with policies to control access, reciprocity of information and services, and protocols or conventions for establishing strong ties among participants (Kollock, 1998; Whittaker et al, 1997). Leimeister & Sidiris (2004) suggest that online communities are built on common interests or common tasks performed with both implicit and explicit codes of behavior. But very little published research explores the application of these theories to academic online communities in higher education. In fact, some proposed core attributes of online communities, such as the need for rules to govern the use of shared resources, seem largely hypothetical. Clearly, work in a virtual community of practice reduces threat of theft, but an idea that remains to be tested is the suggestion by Kollock (1998) and others that low risk makes a community dull and lacking in opportunity to build trust.

This study attempts to identify those elements that assist in building and sustaining an academic online community about teaching and learning. The context for the study is a nationwide community of Cyber Groupies who work together as Editorial Board members of the Multimedia Educational Resource for Learning and Online Teaching (MERLOT), a project that started in 1999. MERLOT is a free and open collection of online learning materials designed primarily for faculty and students of higher education. MERLOT is also a grassroots community of faculty who make the resource collection grow by contributing materials and adding assignments and comments. Results of this study question the need to control access to academic resources shared online. Results also suggest that for the MERLOT editors, benefits of membership in the online community outweigh faculty time costs in accordance with sociological theory of distributed leadership and in spite of little professional status or recognition. The lynchpin that sustains the MERLOT online community is the camaraderie members view as important support for their productivity. Results illuminate the openness of the MERLOT online community to welcoming new members. But relationships between size and productivity of an online community merit further investigation, since the MERLOT community depends upon personal contact for interactions among working members.

Overview of the MERLOT Project

MERLOT is an organization made up of higher education systems, consortia, and institutions. The MERLOT project provides tools and processes to enable faculty to identify and collect, peer review, and use high quality online learning materials effectively and efficiently. The mission is to improve the effectiveness of teaching and learning by building discipline-specific faculty communities around teaching and learning. As of September, 2004, MERLOT held over 10,000 online learning resource items and 1,400 peer reviews. Over 21,000 registered users average 15,000 visits per month. This survey study investigated the subset of MERLOT faculty who serve on Editorial Boards to triage and peer review materials. The peer reviews highlight the top quality resources for their academic disciplines.

Constituents of the MERLOT Online Communities Faculty members from four state institutions commenced the development of the MERLOT project in 1999. MERLOT's initial online communities represented four academic disciplines, Biology, Business, Physics, and Teacher Education, with 12 faculty members serving as Editorial Board members from each discipline. The project has grown to include 111 faculty members as Editorial Board members in 14 discipline communities. Each discipline community had two general goals. Identify high quality materials for collections of online teaching and learning resources. Develop and implement a peer review process for the online learning and teaching resources within MERLOT discipline-based collections.

Discipline Community Communication In order to facilitate both synchronous and asynchronous communication, the MERLOT Editorial Board of each discipline community was provided several communication tools: Phone teleconferencing tool, Listserv for email to the community, Internal MERLOT Web site to post links and documents, and Threaded discussion tool.

Methods

A survey of the MERLOT Editorial Boards was designed by the Teacher Education Editorial Board to investigate the definition of an online community and to identify those elements that assist in building and sustaining such a community. The Center for Usability in Design and Assessment (CUDA) at the California State University was approached to help create, distribute, and analyze the online survey. Survey questions were field-tested in the form of open-ended items. The items were edited for clarity. The Survey Questionnaire consisted of 15 questions and was administered in forced-choice format. The respondent was asked to choose three of each set of options. The options were derived from typical open responses to the items piloted in the field-test. One item described several core elements required to sustain an online community according to previous publications (Kollock, 1998; Whittaker et al, 1997). The respondent could choose very characteristic, somewhat characteristic, or not characteristic for each attribute. An open response option was included with each item.

Each MERLOT Editorial Board member from all 14 of the current discipline communities was emailed a request to respond and a link to the online survey. Board Members did not have to identify themselves, only the Editorial Board Community in which they belonged. A week later, a reminder email was sent to each Board Member who had not yet responded. Two weeks later, members of the Teacher Education Board phoned non-respondents requesting their participation and a third and final reminder was emailed. The last of the responses were received one month after the first request for response to the survey.

Respondents A total of 91 responses, approximately 82% of the 111 current Editorial Board members, were received. The response rates for each Board are listed in Table 1. [Table ONE] See issue website http://rapidintellect.com/AEQweb/win2004.htm

Rank or Position Nearly three-fourths of the respondents were tenured faculty members or administrators as listed in Table 2. The remainder consisted of junior faculty, lecturers, and a few instructional technologists. [Table TWO]

Length of Time on the Editorial Board About half of the respondents (51%) were Editorial Board discipline community members since the planning and implementation stages of MERLOT between 1998 and 2000. Table 3 shows that nearly a fourth (23%), were new Editorial Board members who had joined their MERLOT community within the past year. [Table THREE] See issue website http://rapidintellect.com/AEQweb/win2004.htm

Results

Figures 1 through 6 show the frequency of respondents who selected various forced-choice items on the survey. The actual survey questions used and the options are displayed with each figure. Figure 1 compares the motivating factors that convinced each board member to first join MERLOT with the factors that were important for their continued participation. Panel A shows their responses to the question, What top three factors first motivated you to participate on a MERLOT Editorial Board? and Panel B shows responses to the question, What top three factors contribute to your continued participation on a MERLOT Editorial Board? Most of the Board members decided to join MERLOT at the request of an administrator, but significantly fewer continue their participation due to an administrator's request (Chi square equaled 12.88 with one degree of freedom, therefore the p value was less than 0.001). A sense of community and networking became significantly more important as a reason for continued participation (Chi square equaled 11.75 with one degree of freedom, therefore the p value was less than 0.001). Sharing expertise also became more important to respondents for their continued participation than it was initially (Chi square equaled 3.88 with one degree of freedom, therefore the p value was 0.05). Commitment to teaching with technology was the most frequent motivation for joining MERLOT and for continued participation. [Figure ONE] See issue website http://rapidintellect.com/AEQweb/win2004.htm

Several factors contribute to an editorial board's productivity from a combination of individual and collaborative work done by each discipline community as shown in Figures 2 to 4 and Tables 4 and 5. Three factors were chosen most often as contributors to productivity: camaraderie, interest in the subject area, and frequent communication. Comments under the option other for Figure 2 recognized good support from leaders as a key to productivity. Figure 3 indicates lack of time as an important impediment to productivity for nearly all respondents. Comments about collaborating highlight the need for regular telephone conference meetings to establish assignments, to plan and for support, to discuss policies and goals, and to carry out the shared task of writing composites of double-blind peer reviews (Table 4). Comments about working independently highlight the writing of reviews or carrying out other assignments. Table 5 shows a preponderance of positive comments about the Editorial Boards as online communities. Nearly all of the positive remarks were about the leadership or the people involved. The few concerns expressed revolved around lack of time, need for more members, need for face-to-face contact, and inadequate rewards for the work. Of 143 comments written by survey respondents in addition to their forced choices, only one comment addressed any concern with or difficulty in navigating, interacting with, writing reviews, locating information, or any other usability feature in MERLOT. [Figures TWO, THREE, and FOUR. Tables FOUR and FIVE] See issue website http://rapidintellect.com/AEQweb/win2004.htm The frequency of scheduled contact for most Editorial Boards shown in Figure 5 was once or twice a month (83%). Nearly all of the respondents (89%) selected phone conference calls as a communication choice. Email followed as a choice by 83% of the respondents. Less prominent choices included listserv or email broadcast tools (25%), face-to-face workshops (14%), individual phone calls between members (11%), documents posted to Web sites (7%), and video conference calls (5%). None of the respondents chose online threaded discussions (0%) as one of the three most important communication forms upon which they depend. [Figure FIVE The elements chosen by at least half of the respondents as very characteristic of an Editorial Board Community shown in Figure 6 include shared goals, context, reciprocity, and repeated active participation. Access to shared resources was selected by only 40% as very characteristic of their Board. [Figure SIX] See issue website http://rapidintellect.com/AEQweb/win2004.htm

When asked, "What top five benefits (personally and professionally) do you receive from participation as a member of a MERLOT Editorial Board?" The second highest response, at a rate of 66%, was "Keeping current with the growing importance of online learning resources." The third most frequent response, at 63%, was "Increased competence with learning resources." It should be noted the most frequent response, at 68%, was "Networking with faculty from other colleges/universities."

Emergent Themes

Five themes emerge from the data as important attributes of the online MERLOT Editorial Board Communities: the importance of shared goals and commitment, the role of strong leadership support, high rankings of collegiality and networking, the building of trust and a close team for productivity, and continued participation in spite of weak or non-existent professional recognition. These themes are illuminated by typical responses to the open items.

Shared Goals and Commitment MERLOT Editorial Board members clearly feel a sense of commitment to one another for active participation as a result of shared goals as shown in

Figures 1 and 6 and expressed in their own words:

We have a good relationship as a group and have managed to be fairly productive, primarily due to member dedication to our shared goals and purpose (Mathematics).

We discuss issues collaboratively and set goals collaboratively ... shared sense of commitment to each member both professionally and personally (Teacher Education).

Even though we are small in number, the active participants are truly dedicated to this work (Information Technology).

Further, Figure 4 illustrates that most of the Boards' goals were accomplished through a combination of collaborative and independent work.

Leadership Support The data clearly show an important role for leadership as shown by the fact that most respondents first joined MERLOT when requested by an administrator (Figure 1). Nearly half of the respondents (42%) first got involved in MERLOT because they were directed or requested to do so by an administrator, but this remained an important factor for the continued participation of only 16% of the respondents. Editorial Board members express a high degree of satisfaction with MERLOT's leadership. From the highest level of administration down through the ranks to the discipline team Co-Editors, Editorial Board members express confidence in their leaders. The leadership has a significant impact on the actual scholarly work of MERLOT members:

The Business team leader has been very effective in staying on top of the activities of the Board Members. She has played an instrumental role in our success to date ... we have a lot of autonomy to achieve our individual targets (Business).

Some specific leadership styles were touched upon in members' comments:

We work around those who don't produce but we work collaboratively to do so (Music).

Collegiality and Networking The comments shared by MERLOT Editorial Board members and a conclusion that can be drawn from Figure 2 show that this this is a happy and collegial group of education professionals, even with scheduled communication only once or twice a month, usually through a telephone conference call. Many statements express overall satisfaction with the people in the discipline group:

Great bunch of folks! (Business).

It's alive, vibrant, caring, and a community I depend on for professional and personal growth (Teacher Education).

The MERLOT experience has been one of the highlights of my professional career (World Languages).

Others offered specific examples of how their group has emerged as a true team.

There is a history with our board among several of us who have stayed since 1999. This core has provided continuity, support, and open arms for all who have joined. They, too, then model this for even newer members. My MERLOT Editorial Board provides an arena for me to truly discuss, debate, and investigate what is happening in my academic profession. I really get to feel like I am a professor (Teacher Education).

Members of several teams explain different ways of making MERLOT tasks a collaborative endeavor:

The most common modus operandi is that in which one board member writes an initial draft of a review and then his/her co-reviewer(s) make additions, modifications, etc. that are mutually shared and revised until all agree on a final draft (Mathematics).

Two reviewers work on the same object independently and then a composite is made out of their individual reviews. If there are points of disagreement, these are discussed until a solution is found (World Languages).

We plan as a group collaboratively, do assigned individual tasks, but work in teams, and then discuss/assess progress as a whole group. This is a very worthy, comfortable, and supportive approach (Teacher Education).

I'm happy to say that through our collaborative work and attendance at the conferences, I now consider the members of my group to be friends as well as professional colleagues: that's a wonderful benefit (Music).

Trust and Commitment as Factors for Productivity MERLOT Editorial Board members clearly feel a sense of commitment to one another and their trust contributes to productivity. In fact, one of the few negative comments was from a Board member who complained about lack of members who could be counted on (Figure 2). Other comments were positive:

We have a good relationship as a group and have managed to be fairly productive (Mathematics).

Typically, we divide up work and complete it that way. We collaborate on what we want end-products to look like and how we want to shape our discipline in MERLOT (History).

Everyone on the Editorial Board is a committed individual. One often sees individuals from other disciplines in our World Languages Team that reach out constantly to reach our established goals (World Languages).

But personal contact is important. Board meetings at the MERLOT International Conference have been energizing factors--spending time together personally makes it easier to maintain our shared commitments during the rest of the year (Mathematics).

Lack of Professional Recognition for the Time Commitment One unfortunate theme emerged from the qualitative data for this survey. The commodity of time, including too few contributing members and lack of professional recognition for the amount of dedication required, was a serious issue for most MERLOT Editorial Board members (Figure 3). Although Board members enjoy their participation in this virtual community and value their colleagues and the contributions they are making to the field, the cost in terms of inadequate rewards for the time invested is quite steep; fortunately, many feel their participation is worth the cost:

Our main problem is that people are tapped for services everywhere and consequently all of us are short on time (Mathematics).

This requires lots of time and commitment. It is not clear that this is rewarded adequately and we are also finding that larger schools, more research active universities, reward it least of all (Chemistry).

The benefits tar outweigh the costs in terms of time commitment. MERLOT specializes in college teaching in a way that is very similar to the professionalization of research through my discipline-based professional organization. I depend upon my MERLOT colleagues to improve my teaching in the same way that I depend upon my professional discipline colleagues to improve my research (Teacher Education).

Discussion

Previous studies suggest that building community requires on-going and sustained interactions, shared purpose or goals, shared resources with policies to control access, reciprocity of information, support, and services, and protocols or conventions for establishing strong ties among participants (Dibben, 2003, Kollock, 1998, Whittaker et al, 1997). The findings reported here support the idea that frequent communication among members of an online community helps to sustain the longevity of the community. Time spent together at meetings and through phone conferences made it easier to maintain the shared commitments of the tasks to be accomplished, in this case, to identify high quality materials for collections of online learning and teaching resources, and to develop and implement a peer review process for the resources in MERLOT discipline-based collections. Respondents expressed satisfaction with the collaborative work and most of the respondents highlighted the value of other members of the community being considered as friends as well as professional colleagues. Results confirm reports that some settings are more effective than others in supporting a virtual collective group when online transactions involve complex sense-making (Desanctis et al, 2003). Although MERLOT provides defined protocols, individual groups came up with slightly different conventions for accomplishing their tasks. Regardless of the conventions used, the commitment to technology, a sense of community, and shared expertise for the aim of more effective teaching were all prominent reasons to continue participation as a MERLOT Cyber Groupie. The importance of shared purpose and strong social ties in sustaining online community agrees with much of what has been published by others about the characteristics of online communities (Desanctis et al, 2003, Hall & Graham, 2004, Leimeister & Sidiras, 2004).

Whitaker et al, (1997) reported that key characteristics to building and sustaining online communities are integrally tied to the design principles of online communities. They also reported how their own "hotly debated" discussion attempts to prioritize critical design issues to support central characteristics of online communities. The debate centers around the difference between those believing support for social interaction is the primary design feature versus those who feel that the crucial issues concern conventions that lead to interaction and sharing. The design of MERLOT is strongly aligned with the following design principles of Whitaker et al (1997). Activity and Object Building--allowing users to have their own space to construct and modify objects; examples include MERLOT collections, assignments, and reviews. Navigation-ease in maneuvering and using various sites within MERLOT. Culture and Policies--the commitment to shared resources and MERLOT "help" documents.

Comments from MERLOT faculty suggest the online design itself builds and sustains online community. The lack of comments regarding usability features in MERLOT may indicate the importance of design principles in building and sustaining online communities. New members may be more aware of how the design of MERLOT supports their involvement. Initially, the design features structure a user's primary interaction with MERLOT. The second largest percentage of participants, 23%, became members of editorial boards since 2003; 37% of participants are 'new" since 2001. The gradual addition of new members to editorial boards, together with the debate of design versus social interaction, suggests the need for further analysis into possible variance between new and long term members in terms of factors they consider important to build and sustain online communities.

A prominent role of personal connections for building and sustaining the community is a clear conclusion from this study when responses are pooled from both new and persistent Editorial Board members. Results show that phone teleconferences were as important as email for academic Cyber Groupies of the MERLOT community as a whole. Scheduling of phone conferences once or twice a month was a small commitment. Since phone and face-to-face meetings were among the most valued interactions for these academic Cyber Groupies, an implication from this apparent need for human contact is the possibility that academic community size matters for online collaboration. Feasibility of academic collaboration may be limited without structures for personal contact instituted, supported, and maintained to achieve comfort among group members as suggested by Hall & Graham (2004) and Desanctis et al (2003). Leimeister and Sidiras (2004) claim that technical platforms do not yet support the communication of implicit codes of behavior needed to build trust required for collaborative work. However, their study of both commercial and non-commercial online communities ranked support in the form of regular real-world meetings much lower. For the academic faculty surveyed here, the discipline community meetings at the annual MERLOT International Conference and periodic phone conferences were important mechanisms provided by MERLOT leadership to support the personal contact, ongoing existence, and productivity of the Editorial Boards.

An interesting observation from the results presented here concerns the absence of a graduated system of sanctions to enforce norms of behavior found to be important to other communities (Kollock, 1998). With MERLOT, no defined boundaries prevent individuals from entering, making use of MERLOT resources, and then departing without contributing. No system is used to monitor and sanction MERLOT members' behavior. A reason for this difference may stem from the fact that a MERLOT resource is not a good to be consumed. There is no risk from the exploitation of MERLOT by other people and no threat of loss or scarcity. An online collection like MERLOT has resources available to be used over and over again. In fact, "lurking" is not a problem because the structure of MERLOT allows for any user to contribute or improve a resource by describing what is valuable or by adding assignments. Thus, the collection has the capacity of growing, but not of diminishing with use. Kollock (1998) presents theories about behavior monitoring as a challenge that sustains a community, and he suggests that a vibrant community requires a challenge. But instead of policing and designing access policies, the challenge for the MERLOT Editorial Boards is to identify high quality and assess value of contributed MERLOT resources, a task that requires collective expertise. Peer reviews are produced through virtual work together. The complex task of peer review is to find and describe those elements most valuable to the teaching and learning process, thus making the selection of good resources more efficient for other faculty.

The community aspect of building the MERLOT collection appears to fit a distributed leadership perspective. A distributed perspective on leadership considers educational resources as falling into three categories: (1) material resources, (2) human capital, and (3) the sociological concept of social capital (Spillane et al, 2001). While consumptive goods, including time, must clearly be used for good instruction, human and social capital are more durable assets that contribute to the process. Human capital refers to peoples' skills and abilities that may be used for leading change to improve instruction. Social capital refers to the collective value of relations among individuals in a community such as the trust and collaboration that makes them able to accomplish more than they could without the community. Human capital can be created by social capital when the community makes the individual capable of greater contributions (Spillane et al, 2001). Social capital facilitates the transfer of knowledge among people within the MERLOT community resulting in the development of human capital to expand the resource base available to the higher education systems contributing as MERLOT partners.

The work of the MERLOT Editorial Board community as a whole can be viewed as an investment in social capital and building of human capital, rather than as a creation of material for consumption. While investments in physical capital depreciate, investments in social capital generate human resources. Time is a resource of concern for the Editorial Board members who contribute peer reviews. But the time spent on MERLOT is an investment in assets that become more valuable with peer reviews. It is not yet clear whether the peer reviews will depreciate. The technology used for a resource could become outmoded, or pedagogical theories of teaching and learning could lead to more effective instructional approaches. Indeed, participation in the MERLOT community should accelerate the development of improved instructional technology, so depreciation of some resources is not unexpected. Evolving views of what quality means for online instructional material is an expected benefit distributed among faculty users who join the MERLOT community at no cost and at no risk. This result reinforces the view of distributed social resources and knowledge capital as an outcome of online communities in agreement with reports by Mason (2003), Desanctis et al (2003) and by Hall & Graham (2004).

In summary, the MERLOT Editorial Boards of academic Cyber Groupies are a vibrant and productive community with technology-enriched teaching as a shared goal, a warm and collegial network under effective leadership with the aim of better teaching as an outcome that outweighs the cost in terms of time commitment. It should be noted that this paper discusses factors from the perspective of Editorial Board members. The survey does not reflect factors from the user or "lurker" perspective. Academic leaders have called for mechanisms to promote more effective teaching in higher education (Association of American Colleges and Universities, 2002). Since the MERLOT community as a whole has the potential to improve teaching in the same way that professional disciplines improve research, it would seem logical to recognize and adequately reward time spent on MERLOT work. People who participate in networked communities bring more value to their campus and their discipline than those who teach in isolation. Recognition of MERLOT as a tool for distributed leadership highlights such merits and should encourage others to become academic Cyber Groupies who will join the charge to professionalize university teaching through online community environments.

References

Association of American Colleges and Universities. (2002). Greater expectations: A new vision for learning as a nation goes to college. National panel report. Washington, D. C. Retrieved October 31, 2004 from http://www.greaterexpectations.org

Desanctis, G., Fayard, A., Roach, M., & Jiang, L. (2003). Learning in online forums. European Management Journal, 21,565-577.

Dibben, K. (2003). Making online communities work. Educause Conference paper. Retrieved October 19, 2004 from http://www.educationau.edu.au/papers/

Hall & Graham (2004). Creation and recreation: motivating collaboration to generate knowledge capital in online communities, International Journal of Information Management, Volume 24 (3), 235-246.

Horrigan, J. B., Rainie, L., & Fox, S. (2001). Online communities: Networks that nurture long-distance relationships and local ties. Pew Internet and American Life Project. Retrieved June 20, 2004, from http://www.pewinternet.org

Kollock, P. (1998). Design principles for online communities. PC Update, 15(5), 58-60. Leimeister & Sidiris (2004) Success factors of virtual communities from the perspective of members and operators: An empirical study. In Proceedings of the 37th Hawaii International Conference on System Sciences.

Mason, J. (2003). Trust, collaboration, e-learning and organizational transformation. International Journal of Training and Development, 7, 259-270. Retrieved October 19, 2004, from http://www.educationau.edu.au/papers/

Spillane, J. P., Diamond, J. B., Walker, L. J., Halverson, R., & Jita, L. (2001). Urban school leadership for elementary science instruction: Identifying and activating resources in an undervalued school subject. Journal of Research in Science Teaching, 38(8), 918-940.

Whittaker, S., Isaaes, E., & O'Day, V. (1997). Widening the net: Workshop report on the theory and practice of physical and network communities. SIGCHI Bulletin 29(3). Retrieved June 20, 2004, from http://www.acm.org/sigchi/bulletin/1997.3/whittaker.html

Nancy J. Pelaez, California State University Fullerton

Tamarah M. Ashton, California State University Northridge

Connie Pollard, Black Hills State University

Jane Moore, National-Louis University 

Cris Guenter, California State University Chico

David Wicks, Seattle Pacific University 

Diane Judd, Valdosta State University 

Darrell Pearson, Troy State University

Richard Staley, SUNY College at Oneonta

Melanie J. Wetzel, California State University

Ashton, Guenter, Judd, and Pelaez were among the twelve founding faculty members of the MERLOT Teacher Education Editorial Board. Moore, Pearson, Pollard, Staley, and Wicks joined the Teacher Education Editorial Board as representatives of institutions that subsequently joined the MERLOT partnership. Pearson is on the MERLOT Discipline Editors' Council.