Information metacognition: a course case study.

Abstract

LI 500 Foundations of Information was a new course designed in two formats: an online course using BlackBoard and a largely face-to-face course. The two options provide choice for students as well as a basis for evaluating the effect of delivery on student achievement and satisfaction.

Background

As the coordinator of the Library Media Teacher (LMT) program at California State University Long Beach (CSULB) developed a master's degree program, I created an introductory course to provide the theoretical underpinnings of the profession. This course, LI 500 Foundations of Information, provides a foundation for library and information science, and underlies the societal context of information and technology.

In the process of designing the course, I wanted to examine the impact of delivery mode, comparing face-to-face delivery (with intervening online activity) and a totally online environment. The delivery mode of the course addressed the need to incorporate technology to optimize access to relevant resources (many of which in information science occur only in electronic form) as well as foster telecommunications and collaboration. It also addressed the need for alternative means to reach students with transportation or time constraints. Additionally, by providing two delivery methods for the same section of the same course, it was thought that students would have greater choice in their education.

Additionally, this commingling of course delivery methods is apparently unusual; most courses have separate sections for face-to-face and online learners. (Boettcher 2001; Leh 2002; Stair 1997). The educational model that appears to best accommodate student choice in varying delivery methods is facilitation, which establishes a resource-rich learning environment and focuses on optimizing group dynamics as students meet desired outcomes (Schwartz 1994); its technology-based counterpart has been labeled as e-moderation by Salmon (2000). (This approach is used mainly in adult training and workplace learning where students are more self-directed and purposeful. (Smith 2003) In brief, systematic analysis of the impact of computer-enhanced/blended (i.e., face-to-face activities supplemented with online assignments) and online delivery modes is lacking in the professional literature, and this project provides a solid case study of that issue.

As this new course was being developed, it provided an optimal situation for analyzing the effects of course delivery. As such, the following aspects were noted: (1) Two versions of LI 500 were developed and offered: a blended delivery and a completely online delivery. (2) Students could choose a blended delivery or online delivery of the same class: LI 500 (3) The coordinator measured, analyzed and drew conclusions about the impact of delivery mode (blended vs. online) of LI500 on student achievement and satisfaction; and on faculty instruction and satisfaction.

To evaluate the project's effectiveness, a number of assessment measurements were utilized:

* Database of students who enroll in course: quantity, demographics, majors, GPA, course grade

* Electronic tracking of Blackboard use

* Collection of representative student work demonstrating information literacy competence

* Student self-assessment of information literacy skills based on existing rubrics. The assessment results can help determine the viability of other courses being offered in these delivery modes, help refine course design, and also determine the course's adoption by other programs.

Demographics

Nine students enrolled in LIT500 Foundations of Information: two online and seven face-to-face. One online student was male; all the others were female. The male student was under 30 years old. Most of the other students were in their forties. Seven students worked as the LMT in their school; one other (a female who took the course online) was a substitute teacher, and the other was an English high school teacher. Two students, the male and the English teacher, were still working on their LMT services credential; the others had completed that part of the program and were continuing their work toward a master of arts degree in education, with an option in school librarianship.

Students who chose to take the course online did so based on transportation issues. One student was an experienced online student, and the other online student preferred this delivery method because her English was limited, and she could easily consult a dictionary in an online environment--and respond more effectively. Classes were held on six Saturdays from 8:30am to 5pm approximately every three weeks. Although it was intended that online students would not attend any of the sessions face-to-face, a mandatory master's orientation meeting and writing proficiency examination required their presence for one Saturday morning; one online student left after the master's activities, and the other online student stayed for the rest of the day.

Online Assessment Instruments

Assignments consisted of an information flowchart, a lesson plan, a multimedia presentation, and written papers based on clinical observations, interviews, and content analysis.. No significant correlation existed between online and face-to-face student work. The major factor in lower grading was incomplete assignments.

Students used the major areas of BlackBoard (nicknamed BeachBoard at the university): communications (57%), main content areas (41%), group areas (1%), and student areas (1%). For one assignment students had to develop a joint white paper on a controversial information issue; one of the face-to-face students suggested the use of group pages to facilitate collaboration so that area was established about midway into the semester. No significant correlation existed between the areas used and final grades.

In terms of accessing BlackBoard, there was no significant correlation between the number of times that a student logged on and his/her final grade. The number of logons for any particular student ranged from 207 to 676; the average number of logons was 361. Students accessed the online site throughout the week, the two highest days being Sunday and Friday (each 18%), and the lowest day being Monday (8%). Likewise, they accessed the site throughout the day, the peak hours being 7 to 10pm. Again, no significant correlation was found in terms of user patterns and final grade.

Students carried out online activities and posted their assignments using BlackBoard's discussion board. In a couple of cases, students were required to respond to another student's work, but in general they were flee to comment on their peer's writing or not. Students tended to decrease their responses to peers as the semester progressed. One student replied to the instructor's entries as much as all the others combined; she also responded to peers more than anyone else besides the instructor.

Journaling Assessments

A major assessment tool to measure student learning was journaling. Library Media Teacher (LMT) students were required to write weekly self-reflections about information within the course's context. Students could draw upon coursework, career efforts, and personal experience. This activity not only served as a way to engage students in the flow of information, which was a course concept, but it also provided a rich means to have students self-assess their learning. For developing a thick dataset of course-centric impact, this kind of critical autographical representation was an effective assessment approach for the instructor as well.

David Boud (1985, 1994) and Donald Schon (1983) have been pioneers in using journal writing to enhance reflective practice and professional development. Reflective journaling also serves as a central component of service learning, and is highly touted in library science studies. (Yontz and McCook 2003) This practice represents one procedure under the research methodology narrative inquiry, which uses stories to understand and analyze contextualized experience. (Burner 1991; Clandinin & Connelly 1999; Schostak 1993) The basic premise is that the social context of one's behavior provides influences their perception and learning.

The following aspects were identified for journal entries: date of entry, entry which replied to a peer's entry, entry which replied to the instructor's entry, entry in reply to a peer, entry in reply to the instructor, source of inspiration (class, online chat, readings, peer's comments, work site, assignment, personal), contribution of source, insight on information (description, affective reflection, application, implication, theorizing), ethics and values, LMT role, action, learning/metacognition.

Even though students were not graded on the quantity or quality of their journal entries (beyond having a minimum of fifteen entries), there was a direct positive correlation between the number of entries and the grade, with a natural "break" at fifty entries. Additionally, those students who wrote consistently throughout the semester tended to get better grades. In a couple of cases, students wrote regularly but posted their entries infrequently so others could not provide timely feedback. Interestingly, online students did not comment on the lack of face-to-face interaction, but their counterparts missed their physical presence, wanting to share their insights more often in class. One student commented:

   I was curious to read his journal entries, especially in light
   that he took the class online. I wondered would his entries be
   markedly different from the rest of the class? Surprisingly, I
   didn't find that to be the case. In fact, in many of his entries
   he applies his reading, assignments and insights to his real life
   experiences. After reading them, I regret that he wasn't able to
   participate more in person because he has so much to offer.

Students tended to "clump" their entries: just before classes, on Sundays and Mondays; and at the semester's end. Students who took the course online tended to clump their entries more than face-to-face students.

Students tended to use the readings as their journal "touchstones": 47 entries. 38 entries built on site experiences, 20 responded to class experiences, 13 entries were based on peers' entries, 8 entries referred to the class's online chats, and 26 entries related to other resources: television, past experiences, or other self-identified resources. Towards the end, students started to suggest other sources for consideration. As students became more comfortable with each other in class, they tended to respond to peer issues face-to-face, and write about their responses to readings online. That is, they self-determined the most effective environment in which to communicate on different sources of information.

Some students wrote topically, using one theme to reflect their week's experiences relative to information: censorship, access, and "rabbit holes." Others used a daily log approach. One student always wrote in response to another student's journal. Their approach did not affect their grade. However, those students who tended to generalize their insights about information tended to demonstrate deeper understanding, which was reflected in their assignments. Thus the journal mirrored their intellectual understanding of coursework.

At the beginning of the semester, students tended to respond to their source material by describing it and sometimes giving an emotional response to it. Later, they began to write about ways to apply their insights about information. Even later they focused on theory and the implications of information's characteristics. Ethics and values of information were seen to be addressed directly after a class session that included that topic. Some students' comments included:

   In working on the group case study, I have had lots of time to
   reflect on the issue of privacy. I have never considered privacy
   to be a basic right ... What I've come to realize over the last
   week is that public libraries, even though they are public, serve
   as vital information centers, and under no circumstances do I want
   people not using it because of privacy issues. Thus, regarding
   libraries, I have totally reversed my opinion, and now advocate for
   the protection of privacy of patron records.

   As much as I had fought going back to school after getting my B.A.
   and teaching credential, I must say that this might be a great
   opportunity to get a master's. The report (research proposal) I
   read was right down our "alley" with the theme I've always been
   a proponent for bilingual education. Maybe I could use that theme
   to create a project to fit into the library/information literacy
   area. I need to start thinking about it now.

On the other hand, the LMT's role was noted throughout the semester on a fairly even time distribution. Calls to action, or acting on information, grew over the semester and peaked about the same time that students "grasped the idea" about the role of information in their lives: just before the third class, which occurred midway into the semester. Students wrote:

   I was looking at an episode of Enterprise that dealt with time
   travel. The captain and a time traveler were about to enter a
   library and their concern was that the device they had would not
   be able to be used to give them the info they needed because the
   library probably did not have the technology to play it. As they
   entered the building with its dusk and decay they were shocked to
   see books made of paper! This blew me away because I had just
   finished reading Rubin a few days before where he writes about the
   rush to embrace new ways to store information but future
   generation might not have access to it be cause they machines no
   longer exist that can make it readable.

   Do you find that since the class started everything seems to come
   back to information? I think this class has awakened my
   consciousness about information. Before we have all (at least I
   did) taken it for granted. Now I'm looking at it in different ways
   and appreciating it more. Recently, I mentioned the importance of
   tacit knowledge in a conversation with my principal and, because
   I better understood why I was able to do what I do, I was able to
   better articulate the unique and valuable role the librarian plays
   in a library media center.

Interestingly, students did not comment on one another's breakthrough on the importance and ubiquity of information--or its transformation in terms of LMT mindset.

Discussion

Over time, students tended to become more theoretical and action-oriented with respect to information. They tended to grow from a consumer-based, emotional response to information, relating it to personal experience, to a more objective and abstract construct with a greater impetus for pro-active change. Students responded significantly to issues of ethics, and became more committed to engagement in information flow as a result of such awareness. It appears that introducing this topic later into the semester was an appropriate strategy since students had already gained a better perspective about the creation and dissemination of information by that time.

Students who made these generalizations and strategic plans tended to earn higher grades, probably because they were able to synthesize and apply theory more effectively; the sophistication of their insights were reflected in their assignments. It should be noted that both online students earned Bs. Four face-to-face students earned As, and three other face-to-face students earned Bs. A key factor was consistent and timely effort, regardless of delivery format.

Students who took the course in a face-to-face manner thought that such a learning environment was more effective; they tended to interact more.

   Anyone who has tried to communicate by e-mail is aware that there
   are limitations due to the fact that it's difficult to interpret
   nuances of meanings and avoid misunderstandings due to word
   choices. I prefer printed material to online material only when I
   need to read, reread and reflect on the topic. However, if I am
   searching for factual information and using it to write, I don't
   need to have a printed copy of the document.

On the other hand, online students were satisfied with their own delivery choice. Interestingly, face-to-face students missed the online students more than the other way around. The key element seemed to be student choice. Based on this experience, the following recommendations are made:

* Encourage students to participate online weekly.

* Provide more explicit structure relative to readings: require students to respond to a set number of readings each week. This approach runs counter to the instructor's preferred method of teaching but responds to the learning nature of most of the program's students.

* Explicitly encourage students to write about the LMT's role and their own learning about the nature of information in their journaling. Such direction should optimize student metacognitive activity.

* Encourage students to reflect on their assignments in their journals. Instead, students tended to process their experiences face-to-face. In that respect, students tended to self-categorize the content that they would write about--and would talk about. They intuitively decided what concepts were best treated in face-to-face conversation, and which lent themselves better to online journaling. Indeed, they did a remarkable job of connecting coursework to personal experience as a way to make sense of the course topics. While the instructor told students to make such connections from the start, it was heartening to see that approach carried out naturally.

* Explicitly instruct on content analysis and growth over time. This was incorporated into the follow-up capstone course on action research at our institution. The instructor had students assess each other's journals at the end of the semester. Most tended to describe the content, and a few generalized the insights or related them to their own experiences, but only one analyzed the entries in terms of growth in knowledge about information.

* Ask students if this course should be incorporated into the credential program. Responses were mixed. While all appreciated this course and found it to be a very effective way to synthesize their prior coursework, they cautioned the instructor to allow only students at the end of their credential program to take the course because of its high level of synthesis. They did think that it was a vital foundation course for the master's degree program, however.

Next Steps

This study is limited to a very small population. However, the critical autobiographical research method proved to be an effective way to garner a thick, authentic, descriptive dataset of information to analyze. Another self-determining process is critical incident analysis, which the instructor is using in another current course. (Mezirow 1990)

The use of journaling and content analysis of journal entries can be extended to other courses and other populations to draw more general conclusions.

It would also be interesting to research the long-term impact of information concepts. Even in the following semester, both the instructor and students found themselves less intensely involved in information per se, although "residue" thinking about such topics continued. Thus, the issue of professional dispositions arises. Do students "turn a comer" relative to their attitudes about information and sustain that changing belief and value into their career?

Works Cited

Boettcher, J. (2001, Feb.). New classics and personal favorites: An annotated bibliography for online teaching and learning. Syllabus, 14(7), 36-38.

Boud, D., Keough, R. & Walker, D. (Ed.) (1985). Reflection: Turning experience into learning. London: Kogan Page.

Boud, D. & Knights, S. (1994). Designing courses to promote reflective practice. Research and Development in Higher Education, 16, 229-234.

Bruner, J. (1991). The narrative construction of reality, Critical Inquiry, 18, 1-21.

Clandinin, D., & Connelly, F. (1999). Narrative inquiry: Experience and story in qualitative research. San Francisco: Jossey-Bass.

Leh, A. (2002, Mar.). Action research on hybrid courses and their online communities. Educational Media International, 39(1), 31-8.

Mezirow, J. et al (Eds.), (1990). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco: Jossey-Bass.

Salmon, G. (2000). E-moderating: The key to teaching and learning online. London: Kogan Page.

Schon, D. (1983). The reflective practitioner. London: Temple Smith.

Schostak, J. (1993). Dirty marks: The education of self, media and popular culture. London: Pluto.

Schwarz, R. (1994). The skilled facilitator. San Francisco: Jossey-Bass.

Smith, P. (2003). Workplace learning and flexible delivery. Review of Educational Research, 73(1), 53-88.

Starr, Robin M. (1997 May). Delivering instruction on the World Wide Web: Overview and basic design principles. Educational Technology, 37(3), 7-15.

Yontz, E. & McCook, K. (2003). Service-learning and LIS education. Journal of Education for Library and Information Science, 44(1), 58-68.

Lesley S. J. Farmer, California State University Long Beach

Dr. Farmer coordinates the Library Media Teacher program at CSULB. She earned her doctorate from Temple University. Her research interests include information literacy, collaboration, and technology. Dr. Farmer has written fourteen professional books, and presents regularly.