Enhanced informatics instruction in evidence-based medicine. (The scholarship of teaching and learning).

Abstract

The future of medical education may be dependent upon our ability to create a curriculum for adult learners that integrates competencies in the use of technology, evidence-based medicine and communication skills. This paper reports the development and implementation of an evidence-based medicine curriculum designed to teach not only traditional evidence-based medicine (EBM) skills, but also to enhance student use of the computer for communication and life-long learning and clinical problem solving. The EBM curriculum incorporates principles of teaching and learning including two instructional and four learning processes. These principles, utilizing a project-based curriculum with computer-mediated support, enhances self-directed learning and student mastery of informatics skills thereby expanding mechanisms for collaboration, communication, support and assessment.

**********

The future of medical education may be dependent upon our ability to create a curriculum for adult learners that integrates competencies in the use of technology, evidence-based medicine and communication skills. This paper reports the development and implementation of an evidence-based medicine curriculum designed to teach not only traditional evidence-based medicine (EBM) skills, but also to enhance student use of the computer for communication and life-long learning, and clinical problem solving. A project-based curriculum utilizing computer-mediated support enhances self-directed learning and student mastery of informatics skills, thereby expanding mechanisms for collaboration, communication, support and assessment.

Students learn best by observing and doing work, by engaging in a community of knowledge-seekers, and by being challenged to actively solve problems that are engaging and important to them. Medical educators are challenged to create an academic environment where students obtain knowledge and build skills by participating in authentic, context-based inquiry. In the clinical years this challenge is easily addressed by the student's active pursuit of knowledge in the clinical setting, where real patient problems are the focus of learning and mastery. In the didactic setting, this challenge proves to be more difficult, although some programs have attempted to address this issue through the instructional methodology of problem-based learning where real patient scenarios guide the student through a process of self-directed inquiry. Unfortunately, traditional structures and practice often do not support or encourage the mastery of context-based informatics skills or the practice of EBM, as much of the knowledge, skills and attitudes we hope to impart to our students do not yet exist either in the clinical or didactic setting.

Medical informatics is the rapidly developing scientific field that deals with resources, devices and formalized methods for optimizing the storage, retrieval and management of biomedical information for problem-solving and decision-making (1). There is increasing awareness that biomedical knowledge and clinical information about patients can no longer be managed and accessed by traditional paper and pencil strategies. The emergence of advanced technologies for management and retrieval of the rapidly-changing information data base that forms the basis of medical practice has led to a strong conviction that education in knowledge retrieval and expert decision-making are of utmost importance in the practice of medicine. In fact, mastery of this "process" is now seen at least as equally important to practice as the fact base on which clinical decisions are made. This process of knowledge retrieval is highly relevant to the dynamic practice of evidence based medicine.

Traditional EBM instruction focuses on the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Traditional instruction focuses on constructing researchable clinical questions, searching for relevant literature and applying that information to patient care. As such, one area of emphasis in the traditional EBM curriculum is to increase expertise in searching and evaluating existing literature as it relates to a clinical question, often in the area of etiology, diagnosis, therapy and prognosis. Informatics skills are often enhanced through student mastery of advanced OVID and medline search strategies.

The aforementioned skills, while important, fail to address the broader role that informatics will play in the life of the practitioner of the future. In a recent statement, the Association of American Medical Colleges identified five major roles played by the future medical provider--all of which included a vital informatics component--the life-long learner, the clinician, the educator/communicator, the researcher and the manager (2). This expanded definition of role and concomitant emphasis on new informatics skills goes beyond that of information retrieval and evaluation. An evidence-based medicine course provides an ideal forum for enhancing student knowledge, skills and attitudes about the use of technology across these roles. A project-based curriculum and self-directed inquiry provides an appropriate vehicle to practice and master these life-long skills.

This paper describes a course of instruction directed towards the process of teaching and learning informatics and EBM skills through a project-based curriculum with computer-mediated support. Wiggens (3) in arguing for authentic assessment practices, stated that " ... we cannot be said to understand something unless we can employ our knowledge wisely, fluently, flexibly and aptly in particular and diverse contexts." Authenticity is supported by: (a) investigations that are open-ended; (b) answers that are not predefined; (c) student construction of meaning; and (d) student use of scientific tools and techniques involving them in scientific discourse and collaboration (4). Project-based learning is a form of contextual instruction that encourages student problem-finding and framing which is often carried out over an extended period of time. "Students pursue solutions to authentic problems by asking and refining questions, debating ideas, making predictions, designing plans and/or experiments, gathering information, collecting and analyzing data, drawing conclusions, and communicating their ideas and findings to others." (5)

There were many challenges to developing a project-based, evidence based medicine curriculum that would tie informatics objectives and instructional strategies to teach new skills in a traditional setting using an authentic context for learning. An early needs assessment survey revealed that some faculty and students were uncomfortable using the computer for even the most basic informatics tasks. It was difficult to convey the importance or relevance of these skills in a purely academic setting. In clinical rotations, preceptors were ill prepared to model or reinforce skills of evidence-based practice. The challenge confronting the faculty was to develop a new approach to teaching and learning, one that would be authentic and project-based to enhance student mastery of requisite, knowledge, skills and attitudes. Teachers would need to shift their support from traditional teaching roles to emphasize modeling, facilitation, in-depth assessment of student understanding and coordination of various types of in-person and computer-mediated support. Teachers would have to manage multiple student projects, including those in which they had limited expertise. They would need to provide specific feedback, promote teamwork and assist students in working together to master the process of evidence-based enquiry.

Students would need support for taking on an independent project, not just carrying out specific learning tasks assigned by a professor. Students would be expected to draw upon their personal interest and experience to identify a researchable project that would have to match the curriculum objectives. They would need to organize and work through the steps of a project; collaborate with peers, and work with available mentors and resources to achieve a quality outcome. They also would need to interpret and share their efforts by disseminating their results to their professional colleagues and peers. Project-based learning with a computer-mediated support system provides current students with opportunities to practice and master important informatics skills while at the same time, enhancing and supporting clinical problem-solving and communication skills. Table 1 identifies the various learning issues and opportunities that students encounter over the course of instruction. See issue's website <http://rapidintellect.com/AEQweb/spri02.htm>

The EBM curriculum includes support for two instructional and four learning processes. The instructional processes are (a) scaffolding and Co) coaching. The learning processes are (a) planning and resourcefulness Co) knowledge representation, (c) communication and collaboration, and (d) reflection.

Scaffolding

Scaffolding is defined as structural supports to assist novice learners in the performance of tasks for which they would otherwise be unable to complete It is any tool, procedure or aspect of the learning environment that specifically assists learners in performing unfamiliar tasks. Scaffolding might be a suggestion or discourse-based assistance or a tool or special device to assist learning (like training wheels on a two-wheeled bicycle) (6). The project-based EBM curriculum provides both explicit and implicit forms of scaffolding. Interaction with a mentor or instructor provides direct instructional scaffolding. The computer-mediated support system (CMSS) acts as a form of implicit scaffolding. Students work through a set of procedures that includes setting goals, breaking down complex tasks of critical appraisal and clinical problem-solving into achievable objectives, planning specific time periods for this to be accomplished and anticipating and planning for the resources that will be used to ensure the objectives are reached. Students also must communicate their work, ideas and findings on-line to their peers and work as part of a team to provide feedback and information. These skills, which overlap with the five major roles of the future medical provider, may be argued to be fundamental to building and refining new knowledge and participating in a community of life-long learners. By providing a framework for these processes, the CMSS scaffolds some of the work that must take place using available informatics methods and strategies for instruction.

Coaching

Coaching typically includes a variety of activities including modeling, demonstrating techniques, giving feedback, challenging the learner, providing hints, encouraging, providing reminders and diagnosing problems (7). Coaching involves situation-specific responses designed to bring learner outcomes closer to ultimate performance expectations. It is critical that feedback be provided quickly in accordance with specific task mastery as students attempt to complete individual tasks. Coaching can also be used to help prepare a student for task completion through guidance, structure and modeling. The EBM curriculum implements coaching both through its structure (completion of a project-based inquiry with specific objectives and tasks) as well as through constant feedback and guidance. This feedback occurs both before and after completion of tasks but is always delivered quickly at the time it is needed. While some of the coaching occurs in the didactic classroom environment, much of the coaching is provided through the use of the CMSS. For example, on-line support assists students in completing representational tasks such as a concept-map or critical appraisal matrix for their presentation. This assistance is available to the student in the CMSS environment whenever their individual project or the classroom environment requires it. Coaching is also supported by the CMSS in that student work is communicated to the professor as well as mentor-group peers in ways that supports immediacy in the feedback process. The CMSS supports communication and collaboration between students and professor/mentors. Using an Internet browser students and faculty can review student work, view previously posted comments and create new feedback for discussion. Thus, the CMSS helps represent student work quickly and immediately facilitating the coaching process.

Planning and Resourcefulness

Students are often unfamiliar with the process of managing a complex multistep project. In addition, they have little practice working through a clinical decision-making process on a patient-oriented project with many components that require specific attention. To assist them in the complex management of their authentic patient-based project the CMSS provides students with organizational and management tools for project planning and implementation. Given that each patient project is individualized, open-ended tools enable the student to make appropriate modifications. Tools assist the student in setting objectives for the project as well as specifying time lines, steps for completion, and available resources

Knowledge Representation

Constructing and representing knowledge is essential to learning and exemplifies higher-order thought (8). A representation can be seen as a cognitive artifact and can function as a future tool for thought. The act of creating a final project (manuscript and presentation) requires deep processing and reflective thought. The final product or presentation becomes a learning tool for colleagues. The process of creating the final project requires that a number of cognitive steps occur. This includes students assimilating new information, integrating this into what they already know, building new connections and restructuring their understanding into a new conceptual framework (9). These activities are essential for learning. CMSS supports this process by assisting learners in organizing their thoughts into a final product and presentation that includes an abstract, literature review, methods, findings and discussion chapter where they speculate about implications for future clinical practice and research. Students also track and analyze their progress in completing objectives and meeting timelines, encouraging a metacognitive analysis of their work in process. Their faculty mentor also tracks their progress through these notes.

Communication and Collaboration

Practitioners communicate with each other about their hypotheses and diagnostic considerations for medical treatment. These communities are bound by a medical culture that values experiential as well as empirical knowledge. Enabling students to establish communities for learning while engaged in a project-based curriculum is important to providing an authentic context for future clinical collaboration and inquiry. On-line discourse can provide important opportunities for growth as students are intellectually challenged by faculty and colleagues to think more deeply about their project and to reflect on the integration of new knowledge and ideas. This process also promotes a culture of telementoring, where colleagues can collaborate across great distances. CMSS provides an opportunity for topic-based asynchronous discussion groups. Each mentor group is provided with a site where students and faculty mentor can communicate and track several discussion areas. Mentor groups can elect to share some of their discussion with other groups, allowing students from different groups to share information electronically. E-mail is also available. Files can be uploaded to the group members and links to resources and distant sites can be customized by each group.

Reflection

Although students are asked to complete a final project, that does not automatically mean that students will be reflective in their work or critically appraise their own product with respect to the work of others. Students often have great difficulty with editing their own thinking or work. This kind of "reification" or "abstracted replay" (10) enables a learner to systematically reflect on the problem-solving process and to make new conclusions about clinical problems and their own work. Self-monitoring is recognized in the literature to be an acquired skill. Specific focus and organized, formal systems for self-monitoring help students master the covert and overt components of the skill they are developing (11). CMSS supports reflection and mastery of self-monitoring by requiring students to post their work to their mentor group, facilitating critique from fellow students and faculty mentors, and by making it easy for the student to revise and incorporate these comments as they reflect on their own work. By encouraging on-line discourse and collaboration, CMSS increases the likelihood that students' work will be revised by tracking and storing revisions of student work. On-line checklists also provide a structure for students to check their work against providing a self-editing tool.

Conclusions

The project-based EBM curriculum was first implemented two years with ago using BlackBoard instructional software with a current enrollment of 31 Physician Assistant students per year. Summative and formative evaluations of the process and the use of on-line discussion groups have revealed strengths and weaknesses of the curriculum based on student feedback and performance. No differences were found on student performance based on pre-and post testing. Students enrolled in on-line asynchronous discussion groups were compared with those using the tools only in in-person discussion settings. A focus group with faculty mentors from both platforms revealed that faculty perceived on-line groups as being an effective method of communicating with students and providing immediate feedback. Faculty did not feel that the groups took any more time than in-person contacts and that asynchronous groups eliminated the problem of scheduling conflicts. Faculty also appreciated the opportunity to be able to conduct groups from a distant site.

Students initially felt inexperienced in using on-line technology and skills as measured by a pre-class survey on use of technology. They were apprehensive about using the on-line tools and supports that the CMSS provided. At the conclusion of the course they demonstrated mastery of performance based informatics learning objectives and reportedly felt more comfortable using technology as a resource for practice. However, students did report frustration with delivery system problems and access at times to course materials. Students in on-line asynchronous discussion groups reported feeling that they spent more time thinking about what they would say before communicating with other students. Faculty confirmed that the quality of on-line exchange was richer in content and critical appraisal than the in-person counterparts. Most students, however, viewed the tools initially as extra work and had difficulty in perceiving the benefit of scaffolding and structural supports until late in the process as they were pulling their final project together for dissemination. In part, however, this is consistent with the nature of the tools, which were still under development and may have lacked evidence of supports that would be helpful early in the process.

Future Directions

Despite the difficulties sited above, we are encouraged by student performance in the EBM project-based curriculum. Student mastery of evidence-based concepts and their ability to apply critical appraisal skills to a real patient problem has dramatically increased, reinforcing the paradigm shift occurring in medicine to evidence-based practice. It is clear that project-based learning with a CMSS provides an authentic context for students to master new knowledge and informatics skills as well as the principles of EBM. A computer-mediated support system enables both students and faculty to collaborate in problem solving as part of a community of learners, outside the traditional classroom environment. The CMSS thereby supports mentoring and modeling the skills and attitudes that will be essential to the practitioner of the future. Future efforts will be directed towards improving and expanding the CMSS with the possibility of expanding access to distant students by creating a totally on-line interactional environment.

Notes

(1.) Shortliffe, E. What is medical informatics? http://camis.stanford.edu/whatisinformatics.htm. accessed 1/5/98. Stanford University, 1995

(2.) Association of American Medical Colleges. Medical School Objectives Project: Medical Informatics Objectives. http://www.aamc.org/meded/msop/informat.htm. accessed 10/11/2000. Association of American Medical Colleges, 1999

(3.) Wiggins, G.P. Assessment: Authenticity, context, and validity. Phi Delta Kappan, 1993, November: 200-214.

(4.) Spitulnik, J., Struder, S., Finkel, E., Gustafson, E., Laczko, J., & Soloway. Toward supporting learners participating in scientifically-informed community discourse. Paper presented at the Computer Support for Collaborative Learning Conference, Bloomington, IN: Indiana University. October, 1995.

(5.) Krajcik, J., Blumenfeld, P., Marx, R., & Soloway, E. A collaborative model for helping middle grade science teachers learn project-based instruction. The Elementary School Journal, 1994, 94(5), 483-497.

(6.) Laffey, J., Tupper, T., Musser, D., Wedman, J. A computer-mediated support system for project-based learning. Education Technology, Research and Development, 1998; 46, No. 1, 73-86.

(7.) Collins, A. Design issues for learning environments. In S. Vosniadou, E. De Corte, R. Glaser, & H. Mandl (Eds.), International perspectives on the design of technology supported learning environments. Hillsdale, NJ: Lawrence Erlbaum Associates; 1996, 347-362.

(8.) Norman. A. Things that make us smart: Defending human attributes in the age of the machine. Reading, MA: Addison-Wesley Publishing Company; 1993.

(9.) Spitulnik, J., Struder, S., Finkel, E., Gustafson, E., Laczko, J., & Soloway. Toward supporting learners participating in scientifically-informed community discourse. Paper presented at the Computer Support for Collaborative Learning Conference, Bloomington, IN: Indiana University. October, 1995.

(10.) Collins, A., & Brown, J.S. The computer as a tool for learning through reflection. In H. Mandl & A. Lesgold (Eds.), Learning issues for intelligent tutoring systems. 1-18. New York: Springer-Verlag; 1998

(11.) Zimmerman, B., Paulsen, A.S. Self-monitoring during collegiate studying: An invaluable tool for academic self-regulation. New Directions for Teaching & Learning. No 63 13-27. Fall 1995.

Anita Duhl Glicken, University of Colorado Health Sciences Center Carol Kamin, University of Colorado Health Sciences Center Gerald Merenstein, University of Colorado Health Sciences Center

Ms. Glicken is Associate Professor, Pediatrics. Her 20 year experience in medical education has focused primarily on the psychosocial aspects of health care, the practice of evidence-based medicine and the educational, personal and professional development of the health care provider. Dr. Kamin is a medical educator with the Department of Pediatrics. Her current efforts are directed towards the use of technology in medical education. Dr. Merenstein is Senior Associate Dean of Medical Education and Director of the Child Health Associate Physician Assistant Program at the Health Sciences Center.